Part 2: Storage on bootc — NFS Mounts and First-Boot Data Disks
Storage is where bootc's rules bite first, and the error messages do not point you at the real cause.
The /mnt symlink trap
My first instinct was mkdir /mnt/depot in the Containerfile. It failed with cannot create directory '/mnt': File exists. On fedora-bootc, /mnt is a symlink to /var/mnt. You cannot mkdir over it, and hand-managing mount points fights the image anyway. The answer is to stop using fstab and mkdir entirely and let systemd own the mounts.
Mounts are systemd .mount units
A systemd .mount unit's filename has to match the escaped mount path exactly, or it silently will not work. /var/mnt/depot becomes var-mnt-depot.mount. systemd auto-creates the mount-point directory for you, which is the whole reason the mkdir was pointless.
NFS example (the shared depot, baked into itg-base):
[Unit]
Description=NFS depot
After=network-online.target
Wants=network-online.target
[Mount]
What=10.10.10.2:/mnt/fast-vmstore/depot
Where=/var/mnt/depot
Type=nfs
Options=_netdev,noatime
[Install]
WantedBy=remote-fs.target
Two things matter: NFS is a network filesystem, so it goes to remote-fs.target with _netdev, not local-fs.target. A local disk goes to local-fs.target.
The nosharecache gotcha
When one host mounts two exports from the same NFS server with different options (for me, the depot and a downloads share, one of which needed a context= mount option and one did not), NFS throws busy or already mounted or sharecache fail. The fix is to add nosharecache to the mount options so the kernel does not try to share a superblock cache between mounts with mismatched options.
SELinux on NFS
You cannot relabel an NFS mount the normal way. Instead you pass the label as a mount option: context=system_u:object_r:container_file_t:s0. That is how the book host's containers can write to an NFS downloads share.
First-boot data disks
For local app data I attach a second virtual disk and format it exactly once, on first boot, with a oneshot service guarded by conditions so it never runs twice.
The format service:
[Unit]
Description=Format dataBook disk on first boot
ConditionPathExists=/dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi1
ConditionPathExists=!/dev/disk/by-label/dataBook
[Service]
Type=oneshot
ExecStart=/usr/sbin/mkfs.xfs -L dataBook /dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi1
The two ConditionPathExists lines are the safety: run only if the raw disk is present AND a filesystem labeled dataBook does not already exist. After it formats, the label exists, so it never fires again. Then a plain mount unit brings it up by label:
[Mount]
What=/dev/disk/by-label/dataBook
Where=/var/lib/book
Type=xfs
[Install]
WantedBy=local-fs.target
Mounting by label rather than by device path means it survives the disk moving around.
Lessons
- Never mkdir mount points on bootc. Name a
.mountunit after the escaped path and let systemd build the directory. - NFS is
remote-fs.target+_netdev; local disks arelocal-fs.target. nosharecachewhen the same server is mounted twice with different options.context=for SELinux on NFS, since you cannot relabel it.- Format data disks once with a condition-guarded oneshot, then mount by label.
Runbooks
The build
- Image Mode & Base
- Storage
- Hostnames & DNS
- Registry
- Actions Runner
- The Pipeline
- Nightly Auto-Deploy
- First Workload
- VSCode Cockpit
- Hugo Auto-Deploy
- Cloudflare & Kobo
- Runner Provisioning
- Tailscale Router
- SWAG & Website
- Split-DNS
- Plex
- Media Library Support
- The Day After
- Self-Hosted Media
- Hypervisor Joins the Fleet
- The Matrix Homeserver
Reference