I published turkerdev/ai-skills and turkerdev/ai-agents in April. They exist because eighteen months of using an assistant produced eighteen months of explaining the same conventions at the start of every session, and the explanations were getting worse rather than better.
The symptom
the first thing typed in a session, every time, in
some form:
"money is minor units in an integer, never a float"
"we use repositories, not models — Order is a domain
object and there is no Order::find()"
"a typed exception per failure, no Result objects"
"tests go in tests/Unit unless they need a database"
four facts, retyped from memory, differently each
time, and increasingly abbreviated as the habit set
in.
the abbreviation is the interesting part: by month
six the instruction was "the usual conventions", which
conveys nothing.Context is per-session and knowledge is not, so anything the tool needs to know has to arrive again every time. The degradation from four explicit facts to “the usual conventions” is the ordinary path, and it produces exactly the convention violations the 2025 measurements found at five times the rate of human-written code.
Why it happens
A session starts empty and the cost of populating it falls on the person, every time, for a benefit they have already had. Nothing about that arrangement rewards doing it carefully on the four hundredth occasion.
The fix
What a skill is
---
name: add-a-migration
description: Use when adding or changing a database
migration in this repository.
---
# Adding a migration
Every migration must be safe against both the previous
and the new application version. See
`docs/conventions.md#migrations`.
**Safe in one release:** adding a nullable column,
adding a table, adding an index with `ALGORITHM=INPLACE,
LOCK=NONE`, widening a column.
**Requires expand-and-contract:** dropping a column,
renaming anything, adding `NOT NULL`, narrowing a type.
A migration matching the second list must carry
`@contract-safe` with the release that stopped using
the column, or CI rejects the merge.A skill is a document with a trigger rather than a prompt template — the description is what decides when it is loaded, and the body is what a person would need to know. It is a hundred and eighty words, which is the whole design constraint.
The one that failed
the first attempt, organised by subject:
architecture.md 2,100 words. nine modules, the
layer rules, the boundaries, the
reasoning.
conventions.md 1,400 words.
testing.md 1,800 words.
and the failure mode, observed three times:
the document says "repositories, not models" in
paragraph four of nine.
the generated code used a model, and cited paragraph
seven — about module boundaries — as justification
for where it put the file.
so the document was read, partially, and the part that
was applied was not the part that mattered.A long document produces confident output that follows some of it, which is worse than short output that follows none — the citation makes it look considered. This is a judgement rather than a measurement: three observations, no controlled comparison, and the change was cheap enough not to need one.
Splitting by task rather than by subject
by subject (what failed) by task (what works)
───────────────────────── ────────────────────
architecture.md 2,100 w add-an-endpoint 180 w
conventions.md 1,400 add-a-migration 140
testing.md 1,800 write-a-test 220
add-a-value-object 160
review-a-change 200
debug-a-failure 190
same information, a tenth of the length at the point of
use, and each one cross-references the long documents
for detail.
the long documents still exist. they are read by
people.A document loaded because a task started is read; one loaded because a subject is relevant is skimmed. The subject documents did not go away — they are the onboarding material, which is what keeps them current, and the task documents point at them rather than duplicating them.
The conventions file, which is the boring part that works
# Conventions
## Money
Always minor units in an integer. Never a float. `Money`
is a value object; a bare int at a boundary is
acceptable and must be named `*MinorUnits`.
## Persistence
Repositories, not models. `Order` is a domain object.
There is no `Order::find()`.
## Errors
A typed exception per failure. No `Result` objects. No
nullable returns for failure.
## Value objects
An invariant of the CONCEPT, not of the business. A
percentage is 0-100; a VAT rate is not one of three
values.Versioned with the code, in the repository, and read by two audiences — which is the property that keeps it current. A document with one audience rots; a document that new colleagues read gets corrected when it is wrong, and it has been corrected four times this year.
Agent definitions: a scope and a list of prohibitions
---
name: test-writer
description: Writes tests for code that already exists.
tools: [read, write, run-tests]
---
You write tests for existing code.
You may create and modify files under `tests/`.
You may not modify anything under `src/`.
You may not modify `phpunit.xml`, the CI configuration,
or any fixture under `tests/fixtures/`.
If a test fails because the implementation is wrong, say
so and stop. Do not change the implementation.
Assert behaviour, not existence. A test that calls a
method and asserts it returned the declared type is not
a test; do not write one.the last two instructions are the ones that matter:
"do not change the implementation" — because a
failing test is a problem to solve and the cheapest
solution is changing the code under test. this is
the behaviour a general-purpose assistant gets wrong
most reliably.
"assert behaviour, not existence" — because the 2025
measurement found generated suites padded with
assertions that execute code without checking
anything, and a coverage gate rewards exactly that.
the tool scoping is the mechanism. the instructions
are the intent. both are needed.The review agent
---
name: convention-reviewer
description: Checks a diff against docs/conventions.md.
tools: [read]
---
Check the diff against `docs/conventions.md` only.
Report a violation as three things: the convention, the
line, and the correction. Nothing else.
Do not comment on style, naming, structure, or whether
the approach is good. Do not suggest alternatives. Do
not approve or request changes.
If there are no violations, say nothing.Restricting it to conventions is what makes it useful, because that is the category the 2025 numbers showed at five times the rate on generated code and it is the category a checklist catches reliably. Every attempt to let it comment on design produced plausible suggestions that cost more to evaluate than to ignore.
read-only tools, and it posts nothing. it writes to
the job summary and a human decides whether to raise
any of it.
which is deliberate: an agent that comments on a pull
request is a participant, and this one is an input to
a person who is already reviewing.
six months:
runs 412
findings 88
raised by the reviewer 61
dismissed 27 (of which 22 were the
same false positive:
a bare int correctly
named *MinorUnits)Twenty-two of twenty-seven dismissals being one false positive is a fixable problem and it was fixed by making the convention more precise rather than by changing the agent. That is the useful loop — an agent that misreads a convention is usually reading a convention that is ambiguous.
Measuring whether a skill helps
the same measurement problem as everything else here.
what can be counted:
convention findings per hundred changed lines, on
flagged pull requests, before and after.
before the skills 4.1
after 1.6
and the confounders, stated:
the conventions file also arrived in that window
two of three people had eighteen months more
experience with the tool
the codebase had changed
so: a 60% reduction, attributable to something in a
change that had three parts, over a period where two
other things also moved.
it is the best number available and it is not a
controlled comparison.Stating the confounders is the difference between a measurement and a marketing claim, and there are three of them. The number is real, the attribution is not, and anybody quoting a sixty per cent improvement from this without the second half is describing something they did not measure.
What did not work
a skill describing the whole architecture
too long. covered above.
an agent that writes migrations
written, tested, and never used. a migration is on
the not-used-for list because a mistake is not
reversible by a redeploy, and writing the agent
did not change that.
a skill that encoded the deploy procedure
the deploy is a script. a document describing what
the script does is a second source of truth for
something that already executes.
auto-loading every skill at session start
defeats the point. the description is the trigger,
and loading everything is the 2,100-word document
with extra steps.Verifying it worked
$ ls skills/
add-an-endpoint.md add-a-migration.md
add-a-value-object.md debug-a-failure.md
review-a-change.md write-a-test.md
$ wc -w skills/*.md | tail -1
1,090 total # six documents
$ ./bin/review-findings --since=6m --category=convention
--per-hundred-lines --flagged
1.6 # was 4.1
$ ./bin/session-log --grep='the usual conventions' --since=6m
0 occurrencesSix documents totalling a thousand words against three totalling five thousand three hundred is the structural change, and the last check is the one that reflects what actually changed about the work — nobody types an abbreviated instruction any more, because there is nothing to abbreviate.
What this costs
Documentation with two audiences, which is a constraint on how it is written. A skill has to be short enough to be read carefully by a tool and complete enough to be correct for a person, and those pull in opposite directions — the resolution is that the skills are short and point at the long documents, which means the long documents have to stay accurate.
A second thing to keep current, and the failure mode is silent. A convention that changes without the file changing produces confident output following the old convention, cited, which is exactly the failure the whole arrangement was built to prevent. The only defence is that the file is also the onboarding material.
And the agent scoping is a norm rather than a mechanism. The tool list is enforced; the instructions are not, and the not-used-for list is a document three people agreed to. It works with three people who had the conversation and nothing about it survives a team that did not.