testing

  • Testing a block theme without a browser

    A theme with no PHP has nothing a unit test can call, and the only test is a person looking at a page.

  • Elasticsearch 8 turns security on and removes types

    February, and a major that changes the default posture rather than the API. The cluster starts and nothing can connect to it.

  • A theme.json schema check in CI

    A typo in a theme.json key is silently ignored, which means a setting that does nothing looks exactly like a setting that works. The published schema is versioned…

  • An audit log that a lawyer could read

    A dispute, and a log that recorded that something changed. “Order updated” with no before, no after and no actor.

  • A test that queries for data it did not create

    A test calling first() depends on whatever happens to be in the database, which is an ordering dependency nobody wrote down. The failure only appears under a randomised…

  • A visual baseline belongs in a container, not on a laptop

    Font rendering, device pixel ratio and browser version all differ between a developer machine and a CI runner, and a screenshot comparison notices all three. A zero pixel…

  • A block theme is a directory of HTML files

    5.9 on 25 January, and templates/ replaces the template hierarchy. A theme with 41 PHP templates and an editor that cannot see any of them.

  • Freezing time, and the code that calls time() directly

    A test freezing the framework clock still fails against code calling time() or new DateTime(), because those bypass it entirely. The database clock is the one that catches…

  • A randomised test order, in CI, on every push

    Tests that pass in file order and fail in any other order are testing the order, and the coupling is invisible until something is added. The seed being…

  • Enums, and the string column that was pretending

    8.1 on 25 November, and the feature PHP had been simulating with class constants for a decade. A status column holding three spellings of the same thing.