Notes on systemd I keep forgetting
I write a new .service file maybe once a month, which is exactly the
wrong frequency: often enough to need it, rarely enough to forget the details. So this
is my own cheat sheet, written down so I stop re-deriving it.
A unit that just works
Most of my services want the same handful of things: start after the network is up, run as an unprivileged user, restart on failure. That's almost the whole file:
[Unit]
Description=My service
After=network-online.target
Requires=network-online.target
[Service]
User=myapp
ExecStart=/usr/local/bin/myapp
Restart=on-failure
RestartSec=2s
[Install]
WantedBy=multi-user.target
Binding to low ports without root
The thing I always forget: you don't need to run as root just to listen on port 80 or 443. Grant the one capability and keep the unprivileged user:
AmbientCapabilities=CAP_NET_BIND_SERVICE
This is so much nicer than the old setcap dance on the binary, because
it survives the binary being replaced on upgrade.
Cheap hardening
A few directives buy real isolation for free. ProtectSystem=full makes
most of the filesystem read-only to the service, and PrivateTmp=true gives
it a private /tmp so it can't see or be seen by other processes there.
I add both reflexively now.
Reading the logs
Everything goes to the journal, and the journal is better than the flat files I used to grep. The two invocations I use constantly:
journalctl -u myapp -f # follow live
journalctl -u myapp --since today
The reload reflex
After editing a unit file, systemctl daemon-reload before
restart, every time. I have lost more minutes than I'd like to admit to
"my change isn't taking effect" only to realise systemd was still running the old
definition. Muscle memory, eventually.
None of this is advanced. It's just the subset I reach for again and again, finally in one place.