Türker Yıldırım

monitoring

  • A pipeline that deploys on merge, and the four gates before it

    Deep dive10 min read

    Eight deploys a week performed by a person, an hour after merge, by whoever was free. Continuous deployment needs confidence rather than automation.

  • An API versioned for the first time in six years

    Deep dive14 min read

    Six years of additive-only changes, and a change that cannot be additive. A field whose meaning must change, with four consumers reading it.

  • The VPS that had been running since 2019

    Deep dive14 min read

    A host provisioned in 2019, upgraded in place three times, and a provisioning script that has never been run against an empty machine.

  • Eleven things installed by hand during incidents

    Note2 min read

    A diff between what the provisioning script installs and what is actually on the host. Six of the eleven are diagnostic tools somebody installed at three in the…

  • The four things a drill found, all of them access

    Note2 min read

    An onboarding drill for a third responder, which found nothing about knowledge and four things about permissions. Access is the failure mode that a knowledge-focused onboarding misses entirely,…

  • Structured logs, and the field that was never queried

    Note2 min read

    Eleven fields on every log line, of which four have never appeared in a query. A structured log invites adding fields because each one is cheap, and the…

  • An on-call rota of three

    Deep dive10 min read

    A rota of two is not a rota. A third person joined in November, and the gap was everything learned in the shadowing month and never written down.

  • An incident drill on a Tuesday afternoon

    Note2 min read

    A simulated incident, announced in advance, run against production with a deliberately broken dependency. Announcing it in advance removes the value of surprise and removes the risk of…

  • A rota of three, and a handover note

    Note2 min read

    A third person on the rota, and the artefact that makes a weekly rotation work. Three sections and a five-minute write-up is the whole mechanism, and it is…

  • The endpoint we could not deprecate because nobody knew who used it

    Note1 min read

    An endpoint from 2019 with no documentation, no named consumer and four hundred requests a day. Two weeks of logging turned an unanswerable question into one email, and…