Part 4: Moving the Container Registry to Forgejo

I started with a plain HTTP registry on Coulson (10.10.10.162:5000) and migrated to my Forgejo instance's built-in registry at forge.itguyeric.com, served over HTTPS through SWAG. This is where the pipeline images live.

Why move

The HTTP registry meant every host needed an insecure-registry exception, and bootc pulling over plain HTTP is exactly the kind of thing you do not want as the foundation of an auto-updating fleet. Forgejo already hosts my git and Actions, and it has an owner-scoped package/container registry built in, fronted by the same SWAG cert as everything else. One HTTPS endpoint, real auth, done.

The HTTP interim (for reference)

While I still had the old registry, bootc needed to be told it was allowed to talk HTTP. That lives in a registries config drop-in (/etc/containers/registries.conf.d/) marking 10.10.10.162:5000 as insecure. Useful to know it exists; the goal was to delete it.

Forgejo registry auth

Create an access token in Forgejo with write:package and read:package scopes. Then log in. The catch that bit me: podman stores credentials as JSON in an auth file, not as a raw token. Writing the token straight into /root/.config/containers/auth.json gives you invalid character 'b' type errors. Use podman login and let it write the file:

podman login forge.itguyeric.com -u itguyeric -p <TOKEN>

Root vs user auth stores are separate. The pipeline runs pushes under sudo, so the credentials have to exist in root's store (/root/.config/...), not your user's. If a push fails auth in a job, check which user's auth store you actually logged into. More than once the fix was sudo rm -f /root/.config/containers/auth.json to clear a malformed leftover and log in cleanly as root.

The bootc pull secret

For a host to bootc upgrade/switch from an authenticated registry, bootc needs its own pull secret at /etc/ostree/auth.json. Same JSON format. Without it, the host cannot pull its own new image.

DNS so the hosts can reach forge

The internal hosts have to resolve forge.itguyeric.com to SWAG (10.10.10.200). I added an Unbound override on OPNsense for that, and had to flush the resolver's negative cache afterward, because it had already cached the earlier "does not resolve" answer.

Repointing hosts to the new registry

Once images were building to Forgejo, each host gets repointed with bootc switch to the new image ref, then upgrades from there on out. bootc switch <ref> repoints, bootc upgrade pulls and stages, and a reboot activates it.

Lessons

• Auth files are JSON. Use podman login, never hand-write the token.
• Root and user have separate credential stores; match the store to whoever runs the push.
• bootc's pull secret is its own file at /etc/ostree/auth.json.
• After adding a DNS override for a name that recently failed, flush the negative cache or you will chase a ghost.