Four servers, eleven log files, and an incident reconstructed from three of them. Shipping first and parsing second, structured events, and the correlation id that makes it useful.
With binary log file-and-position replication, promoting a replica means finding the equivalent coordinate on every other replica by hand, at the moment nobody wants to be doing arithmetic.…
Daily indices grow linearly and nothing removes them, so a logging cluster works beautifully for four months and then stops accepting writes with a disk watermark error at…
Deploying by rsync on a Friday afternoon, and what replaced it: lint, test, build, stage, promote — with a rollback that is the same mechanism rather than a…
Redis has two persistence mechanisms and the documentation presents them as a choice, which invites picking one. They fail differently, and on anything that matters both are usually…
Free certificates went GA this month. Automating issuance and renewal, the rate limits that catch staging first, and the monitoring that has to outlive the automation.