linux – Page 4 – Türker Yıldırım

A systemd timer instead of a cron entry, at last

NoteFebruary 22, 20242 min read

A cron entry that had been failing silently for six weeks, and a timer that would have said so. Persistent=true runs a missed occurrence after a reboot, which…

Reproducing a build from six months ago

Deep diveJanuary 4, 202414 min read

A security question about a release from July, and no way to rebuild it. The same commit produces a different image, and neither of them is wrong.

Journald rate limiting, and the log line that vanished

NoteOctober 19, 20231 min read

A worker logging a line per job hit the journal’s rate limit during a backlog, and the lines that were dropped were the ones describing the backlog. The…

The alert that fired for two years and meant nothing

Deep diveOctober 18, 202310 min read

Thirty-one alert rules, a channel muted by everybody, and 412 alerts in a month of which four corresponded to a real problem.

Import blocks, and the resources we had been pretending to manage

Deep diveSeptember 21, 202310 min read

Terraform 1.5 in June, and an import that is part of the plan rather than a side effect. Fourteen resources created by hand in 2019.

Debian 12 on a host that had been on 10

NoteAugust 8, 20232 min read

Two major versions in one jump, which the documentation says not to do and which is what the situation actually was. A rebuild is the honest test of…

The state file two people wrote to at once

Deep diveJuly 14, 202310 min read

Two pipelines, one state file, and a plan that proposed deleting a database. A lock held by a run cancelled forty minutes earlier.

Rebuilding the deploy as something a stranger could run

Deep diveMay 16, 202314 min read

A deploy that one person can run, and a holiday booked for August. A runbook of 22 steps, four of which say “check it looks right”.

nginx 1.24, and a config I had not read since 2016

NoteMay 2, 20232 min read

Upgrading forced a read of a configuration file that had been copied forward through three servers and four PHP versions. The cipher list is the line worth deleting…

A backup we restored on purpose, every month, forever

Deep diveApril 5, 20238 min read

A backup that had run nightly for four years and had never been restored. The first restore produced a database missing one table.