Over this last Christmas break I had nothing but time as I was stuck down with covid. All my Christmas and new years plans were abruptly cancelled or moved and I was bed-ridden for a week. So, I decided to make good on a promise that I made myself some time ago to refresh my homelab. It’s something I’ve been meaning to do for a while as I had some 13 different VMs some of which running a single service, many of which could have simply been containers instead. And now I finally had the time to commit to such an endeavour.

One of the virtual machines in my homelab (I named it bristlecone for it’s age and gnarled nature) has been around as long as I’ve been playing with linux. At some point in 2016 I took the old family desktop and installed Ubuntu server on it. Originally I only used it as a web server, but as time went on I installed ownCloud, which I later upgraded to Nextcloud, then I added this service, then I added that. Before long I wanted more than just a single logical machine to play with. My solution was to get a new HDD, install Proxmox on it, and then image the original Ubuntu installation onto a virtual disk to run as a VM. This was probably a year or so after the initial install, for context. So, for the past 9 years (8 as a virtual machine) this Ubuntu server installation has trudged along. A port forward pointing port 80 and 443 at this machine has existed in at least 5 different routers as I’ve moved around.

Yesterday, I shut it down.

I’ve managed to pull all but a couple of services in my homelab onto a single machine to be run as podman quadlets, a ingenious little piece of technology I learnt about from a lovely French Canadian colleague of mine recently. I had used docker in the past to run services like Bitwarden (vaultwarden) and Gitlab but, although I’d heard of it, I’d never used podman. I had certainly never come across quadlets anyhow. So I dove in, starting with the previously linked documentation (which is a wonderous resource if you’re planning on doing anything like what I’ve done), and then with some good old fashioned googling. I figured that I could simply as I pleased, set up quatlets for the services that I wanted, and then migrate my old running instances across to the new containers. And that’s basically how it went down.

I’d like to outline the basic premise, how I went about setting everything up, and some of the tips and tricks I learnt along the way. Also I want to be clear that my exposure to podman has been almost solely through quadlets. I have had to interact with the podman cli somewhat throughout this but primarily I’ve been defining configs for quadlets in files and then using systemd to orchestrate them.

I’m running on podman version 5.4.1, which is the version available in the Ubuntu 25.04. I have 24.04 installed and will until 26.04 is released as I want to tie myself only to LTS releases. However, 24.04 has podman 4.x.x which is missing some substantial feature development for quadlets as quadlets are still under active development. To solve this problem I found and reviewed a script, see plucky-pinning.sh below, which pins podman and a collection of dependencies to the 25.04 releases.

One of the features that I was particularly interested in was podman quadlets, as I’m sure is clear. A quadlet is a systemd unit which defines a container or some other piece of podman infrastructure (network, pod, volume, etc). These unit files can then be symlinked into a set of predefined locations and systemd can then be used to start, stop, inspect, etc, them. You can even use journalctl to view logs. The primary ways I set up services were through standalone containers, and pods. Pods, which I was under the impression I needed podman v5+ for, are a way to logically group containers and make it easier to manage groups of containers supporting the same service. The general idea is, as far as I engaged with it, that you create a my-service.pod file in which you can specify your network and published ports, among other things. Again, I pushed this as far as networking as that is all I needed, I’m sure there is more to learn here.

For example, the following is the definition for my Planka service that I’m running.

planka.pod:

[Pod]
PodName=planka
Network=planka.network
PublishPort=1337:1337

planka-server.container:

[Unit]
Description=Planka - Server
Requires=planka-db.service
After=planka-db.service

[Container]
Pod=planka.pod
ContainerName=planka-server
Image=ghcr.io/plankanban/planka:2.0.0-rc.4

Environment=BASE_URL=planka.example.com
Environment=DATABASE_URL=postgresql://<planka-db-user>:<planka-db-user-password>@planka-db:5432/<planka-db-name>
#                                         this is the host name of the db container. ^ 
Environment=SECRET_KEY=<64-character-secret-key>

Volume=/<data-location>/favicons:/app/public/favicons
Volume=/<data-location>/user-avatars:/app/public/user-avatars
Volume=/<data-location>/background-images:/app/public/background-images
Volume=/<data-location>/attachments:/app/private/attachments

[Service]
Restart=always
TimeoutStartSec=300

[Install]
WantedBy=default.target

planka-db.container:

[Unit]
Description=Planka - DB

[Container]
Pod=planka.pod
ContainerName=planka-db
Image=docker.io/postgres:16-alpine

Environment=POSTGRES_PASSWORD=<planka-db-user-password>
Environment=POSTGRES_USER=<planka-db-user>
Environment=POSTGRES_DB=<planka-db-name>

Volume=/<data-location>/postgresql:/var/lib/postgresql/data
Volume=/etc/timezone:/etc/timezone:ro
Volume=/etc/localtime:/etc/localtime:ro

...
<homepage-configuration>
...

[Service]
Restart=always
TimeoutStartSec=300

[Install]
WantedBy=default.target

planka-network.container:

[Unit]
Description=Planka network
After=network-online.target

[Network]
NetworkName=planka-network
Subnet=10.1.0.0/24
Gateway=10.1.0.1
DNS=

[Install]
WantedBy=default.target

Thees files were soft linked with ln -s <original> <soft-link> into the $HOME/.config/containers/systemd directory. After which they could be started with systemd commands.

While setting all this up I ended up using some commands a lot, for instance systemctl --user restart <unit-name> so I ended up creating a list of aliases to help with this, these are for bash:

alias \
    usta="systemctl --user start" \
    usto="systemctl --user stop" \
    ures="systemctl --user restart" \
    ustatus="systemctl --user status" \
    ureload="systemctl --user daemon-reload" \
    usvc="systemctl --user --type=service" \

One of the services I’m running is homepage which integrates with podman to get container information directly. You can specify homepage related data as Labels, this is not dissimilar to how one would do the same with docker I believe. For instance, the following is what I had for Planka:

Label=homepage.group=Productivity
Label=homepage.name=Planka
Label=homepage.icon=planka.png
Label=homepage.href=planka.example.com
Label=homepage.description="Kanban Board"

This makes it very easy to keep track of your services, all in one place, each service defining its important data itself.

In my setup I had all my containers running in userspace as rootless containers. The only time this became an issue was when a container needed access to the podman (or docker) socket. This usually requires root privileges. Fortunately, podman comes with a solution to this problem.

You can make a userspace version of the socket available with the followig commands:

systemctl --user enable podman.socket

# check with 
systemctl --user status podman.socket

The socket should then be available to be mounted as in the following:

Volume=/run/user/1000/podman/podman.sock:/run/podman/podman.sock

In my case my user has the uid 1000.

All in all, I am very happy with how this turned out. I have all my original services setup as quadlets and their data migrated across (with basically no issues at all, Nexcloud was the worst but nothing broke), and I have a host of new services to try out. It’s going to be very simple for me to add services in the future, and manage those that I have set up and I’m very happy about it.

There are however a couple of caveats to this setup. The first is that every container, thus every service, is tied to the same machine. Admittedly, that machine has a low upgrade surface area so updates and reboots should be infrequent, but regardless, when the machine goes down, so do ALL of the services. I see this as an acceptable price to pay for the ease of setup and administration though. Secondly, is backups. I have setup a restic profile for the data diretories of the containers which I run as a cronjob every week but for instance, an image of a running database is not a backup. I’m going to need to investigate how to properly automate backups of this data using a method that doesn’t leave room for error. That being said, all of the configuration for the containers (to get them set up and running) is stored in a git repository, and most of the data is contained in the files one way or another which is all being backed up as well. So, I’m not overly worried right now.

The machine that all of my new services ended up on has been named rimu as an ode to my kiwi heritage and the fact that now instead of having multiple small VMs hosting separate services, I now have one towering VM to host all my services.

P.S. In the future, I will migrate the containers to using environment files and then publish a sanitised copy of the repository for public perusal.

• plucky-pinning.sh

#!/bin/bash

# Must be run as root
if [ "$EUID" -ne 0 ]; then
  echo "Please run as root (e.g., sudo $0)"
  exit 1
fi

# Define file paths
PINNING_FILE="/etc/apt/preferences.d/podman-plucky.pref"
SOURCE_LIST="/etc/apt/sources.list.d/plucky.list"

# Write Plucky APT source list
echo "Adding Plucky repo to $SOURCE_LIST..."
echo "deb http://archive.ubuntu.com/ubuntu plucky main universe" > "$SOURCE_LIST"

# Write APT pinning rules
echo "Writing APT pinning rules to $PINNING_FILE..."
cat <<EOF > "$PINNING_FILE"
Package: podman buildah golang-github-containers-common crun libgpgme11t64 libgpg-error0 golang-github-containers-image catatonit conmon containers-storage
Pin: release n=plucky
Pin-Priority: 991

Package: libsubid4 netavark passt aardvark-dns containernetworking-plugins libslirp0 slirp4netns
Pin: release n=plucky
Pin-Priority: 991

Package: *
Pin: release n=plucky
Pin-Priority: 400
EOF

# Update APT cache
echo "Updating APT package list..."
apt update

echo "Plucky pinning setup complete."