logging

  • try/catch around await catches a rejection

    A rejected promise inside an async function behaves like a thrown exception, which means error handling stops being a .catch() bolted onto the end of a chain and…

  • A circuit breaker before the third-party outage

    A payment provider was slow and the whole site went down with it. A timeout bounds one request; it does nothing about how many are waiting at once.

  • journald rate limiting drops your logs silently

    systemd-journald discards messages from a service exceeding its burst limit and notes the fact in a single line that is easy to miss — so a debugging session…

  • Alerting that people do not learn to ignore

    340 alerts in a month, four of which mattered. An alert channel nobody reads is the same as no alerting, and the cause is alerting on causes.

  • Delete Elasticsearch indices older than 30 days

  • Logging drivers, and why json-file fills the disk

    The default driver writes every line to a JSON file with no rotation, so a chatty container quietly consumes the host disk and nothing points at it —…

  • Centralised logging that answers a question

    Four servers, eleven log files, and an incident reconstructed from three of them. Shipping first and parsing second, structured events, and the correlation id that makes it useful.

  • Every threshold needs a duration

    An alert on an instantaneous value fires on the spike that a deploy, a cron job or a single slow request produces, so the first thing anyone does…

  • The runbook link belongs in the alert

    An alert saying HighErrorRate on shop-web at three in the morning is a puzzle. The person receiving it may not have written the check, and the context lives…

  • Structured logs are the whole decision

    A log line written as a sentence has to be parsed back into fields by a regular expression somebody maintains, and that expression breaks whenever the sentence changes.…