Uncategorized

  • A Node script in a PHP repository, and where it lives

    A build script written in JavaScript because it processes the asset manifest, in a repository where everything else is PHP. The Makefile is the interface and the language…

  • An environment with a required reviewer on production

    A deploy job that anybody with write access could trigger, on a repository where write access was granted to six people for unrelated reasons. Scoping the secrets to…

  • The unique index we could not add, and the duplicates we had to find first

    A constraint that should have existed since 2016, and 1,240 rows that violate it. Support merging customer records by hand, monthly.

  • Log levels agreed in writing, finally

    Eight severity levels, used by four people with four different interpretations, producing a log where error meant nothing in particular. The column that resolved the arguments is the…

  • A deploy that rolls back on a health check

    A deploy that succeeds because the files copied is a deploy that has verified nothing. Twenty seconds of retrying before giving up, and a rollback that is the…

  • The online DDL that was not online

    ALGORITHM=INPLACE does not mean lock-free, and the operation that needed a table rebuild took a metadata lock at the end anyway. Specifying LOCK=NONE turns an unpleasant surprise into…

  • A replica that fell behind because of one long transaction

    Replication lag of eleven minutes on a replica with no load, caused by a single transaction on the primary that took eleven minutes. The instant drop is the…

  • A compose profile for the services nobody runs locally

    A stack of eleven services where four are needed by two people occasionally, started by everybody constantly. Memory is the reason: the search pair was 2.4 GB on…

  • A PSR-3 context array nobody was reading

    Every log call carried a context array, the log format rendered it as a string, and the aggregator indexed it as one blob. The context array is the…

  • HPOS, and reading the migration before enabling it

    High-Performance Order Storage moves orders out of the posts table, and the compatibility mode that writes to both is the part to plan around. The four queries were…