logging

  • real_ip_from, or the client address is whatever they say

    Behind a proxy, $remote_addr is the proxy — and reading X-Forwarded-For instead is correct only if you also refuse to believe it from anyone but the proxy. This…

  • An index template before the first document

    Field types are decided by whichever document arrives first, so a field that is an integer in one service and a string in another is rejected in the…

  • An SLO is a number somebody agreed to

    Everyone wanted the site to be fast and nobody could say what fast meant. An indicator, an objective and a window — and the error budget is what…

  • A for clause on every rule, or it fires on noise

    A threshold evaluated instantaneously fires on every transient spike, and transient spikes are constant — which is how a channel accumulates three hundred alerts a month. The for…

  • Redis 6 and the shared password that was everywhere

    One password, six services, and FLUSHALL available to all of them. 6.0 landed in April with users — and the keyspace pattern is where the isolation actually is.

  • Log rotation on json-file, which has no default limit

    The default logging driver writes to a file that grows without bound, so a chatty container fills the host disk over a fortnight and nothing warns first. Setting…

  • An SLI is good events over valid events, and both are choices

    The numerator and the denominator are both decisions, and getting the denominator wrong is what makes an objective meaningless. Excluding health checks and monitoring traffic is uncontroversial and…

  • An HTTP client that is part of the framework

    Four services, four hand-rolled clients, four retry policies. 7.0 wraps Guzzle, and the wrapper is the point rather than the transport.

  • A runbook is more useful for what it says not to do

    An alert that states a number crossed a threshold hands the receiver a research project, and at three in the morning three sentences beat any precision in the…

  • Multi-window burn rate, and the two alerts it replaces

    A single burn-rate window is either too twitchy or too slow, and pairing a long window with a short one gives both sensitivity and stability. The short window…