testing

  • A quarantined test, three deadlines, and a real race

    A test skipped since 2023 under a policy with deadlines, extended three times, covering a payment path. A quarantine deadline that can be extended is a policy with…

  • A review agent, and what it is allowed to say

    The only agent permitted near a pull request, constrained to a specific kind of comment. Restricting it to conventions is what makes it useful — the convention findings…

  • A thousand runs, and what that proves

    A test run a thousand times to establish that a flake is fixed, and what that number actually supports. A thousand clean runs bounds the failure rate rather…

  • One flaky test split into three

    A test asserting three things across two systems, failing intermittently for three years, split into three tests that each assert one. Splitting is diagnosis rather than a fix…

  • The command bus we removed

    A command bus introduced in 2021, and 88 commands with one handler each. A stack trace with four frames of dispatch before the work.

  • The test that had been quarantined since 2023

    A quarantine list with a deadline, three extensions, and one test covering a payment path that had been skipped for three years.

  • Coverage on changed lines only

    A merge gate on the coverage of lines the pull request touched, which is a different and better question. A global coverage threshold is satisfiable by testing the…

  • A contract fixture that a consumer stopped publishing

    Four consumer-authored fixtures fetched nightly, and one that stopped updating in November. A fetch that falls back to a cached copy on failure is being helpful and is…

  • Reusable skills and agent definitions, and what they are actually for

    turkerdev/ai-skills and turkerdev/ai-agents, and eighteen months of retyping the same instructions. Context is per-session and knowledge is not.

  • Deleting a module

    A module built in 2022 for a product line discontinued in 2024. Eight thousand lines, a database schema and two scheduled jobs, all still running.