Everything in the provisioning script arrived on the new host, and three things that were not in it did not. A provisioning script describes configuration and a host…
A major distribution upgrade, by rebuild, with no changes to the provisioning script at all. A provisioning script that is exercised monthly is a script that works when…
A deploy script rewritten twice — into a compiled language and back — and stable for eighteen months at a hundred and eighty lines. Shell is the right…
A burst limit configured in 2023 as a precaution, which took three years to be needed. Without the burst limit the worker would have restarted every five seconds…
A systemd burst limit configured in 2023 as a precaution, which took three years to be needed. Without the limit the worker would have restarted every five seconds…
A systemd unit enabled and running for five years, consuming a gigabyte, serving a feature that was removed in 2021. Nothing alerts on a service that is working…
A year of eight-hour certificates rather than an authorized_keys file, and the operational reality of it. The mid-incident expiry is the failure mode that theory does not predict,…
A service-to-service certificate issued by an internal authority in 2022, with a three-year validity nobody had recorded. A certificate with a three-year validity is a reminder set three…