code-generation

  • A skill that was too long to be read carefully

    A two-thousand-word document describing this application’s architecture, which produced worse output than no document. A long document produces confident output that follows some of it, which is worse…

  • Reusable skills and agent definitions, and what they are actually for

    turkerdev/ai-skills and turkerdev/ai-agents, and eighteen months of retyping the same instructions. Context is per-session and knowledge is not.

  • A conventions file, versioned with the code

    A document describing this codebase’s conventions, read by a tool and by people, kept in the repository. The file is the single most effective thing we have done…

  • Reviewing code I did not write

    Review as the skill that matters, and a volume of code that needs it. A change that looks right, passes, and is subtly wrong.

  • A test written by a tool, and read as review

    A generated test for an existing class, which is the case model assistance is genuinely good at, with one caveat. The second test asserts that assignment works, which…

  • Three suggestions that would have been vulnerabilities

    Six months of review findings, filtered to the ones with a security consequence. All three are patterns that appear constantly in public code, which is presumably why they…

  • A year of an assistant in the editor, measured

    A year of using a model-backed assistant, and an attempt to say something specific about it. Every claim about this is a demonstration or a complaint.

  • Acceptance rate, edit distance, and what neither measures

    Two metrics that are available, neither of which answers the question anybody is asking. A high acceptance rate is compatible with a tool that produces plausible code and…

  • A tool that does not know this codebase’s conventions

    Suggestions that are idiomatic for the language and wrong for this application, which is the most expensive kind of wrong. Both suggestions are what most PHP looks like,…