The ignoreErrors block in phpstan.neon had grown to sixty-one patterns over two years. Nobody knew which were still needed, three of them were broad enough to silence real errors, and pruning them meant running the analyser and reading four thousand lines. 0.12 shipped in December with a generator, and the whole block became a file nobody edits.
The symptom
$ wc -l phpstan.neon
78 phpstan.neon
$ grep -c ' - .#' phpstan.neon
61
$ head -12 phpstan.neon
parameters:
level: 4
paths: [src]
ignoreErrors:
- '#Access to an undefined property#' ← everywhere. everything.
- '#Call to an undefined method [a-zA-Z\\]+::[a-z]+()#'
- '#Parameter #1 .* expects array, mixed given#'
- '#Cannot call method [a-z]+() on mixed#'
$ vendor/bin/phpstan analyse --no-progress 2>&1 | grep -c 'was not matched'
19The first pattern silences undefined property access across the entire codebase, which is one of the most valuable things the analyser detects. Nineteen patterns match nothing at all any more, which means the code they existed for has been rewritten and nobody removed them. This is not configuration; it is sediment.
Why it happens
A mature codebase produces thousands of errors on the first run, and there is no way to act on four thousand errors. The two available responses are to lower the level until the number is small — which makes the tool useless — or to suppress by pattern, which is what everyone does.
Patterns are the wrong unit. An error is specific to a file, a line and a message, and a pattern is a generalisation over all three — so every suppression is broader than the thing it was written for, and the breadth is invisible. A pattern written to silence one legacy class silences the same mistake in code written next year.
The fix
Generate it, commit it, and stop reading it
$ vendor/bin/phpstan analyse --level=5 --generate-baseline
Note: Using configuration file phpstan.neon.
[OK] Baseline generated with 3,884 errors.
$ head -10 phpstan-baseline.neon
parameters:
ignoreErrors:
-
message: "#^Access to an undefined property App\\Order::\\.$#"
count: 4
path: src/Order.php
-
message: "#^Method App\\Billing\\Vat::rate\(\) has no return type specified\.$#"
count: 1
path: src/Billing/Vat.php# phpstan.neon — what is left of it
includes:
- phpstan-baseline.neon
parameters:
level: 5
paths: [src]Six lines of configuration and a generated file. The message is anchored, the path is exact and the count is recorded, so a suppression covers precisely the errors that existed when it was generated and nothing else — which is the property patterns could never have.
The count is what makes it a ratchet rather than a suppression file. Fixing three of four occurrences in a file fails the build until the count is updated, so partial progress is recorded rather than silently absorbed. That is the mechanism, and it is the whole reason this works.
A debt register, not configuration
# what is actually in there, by error type
$ grep 'message:' phpstan-baseline.neon | sed 's/.*#^//;s/ .*//'
| sort | uniq -c | sort -rn | head -6
1204 Method
881 Access
612 Parameter
402 Cannot
288 Property
194 Call
# and by directory, which is the more useful cut
$ grep 'path:' phpstan-baseline.neon | awk -F/ '{print $2}'
| sort | uniq -c | sort -rn
2841 Legacy
602 Http
288 Billing
153 DomainSeventy-three percent of the debt is in one directory, which changes what to do about it: the answer is not a gradual cleanup across the codebase but a decision about Legacy/, and that decision has a business case attached rather than a technical one.
Reading the baseline this way — once, when it is generated — is worth an hour. Reading it any other time is not, and that is the point of committing it and then ignoring it.
The ratchet, and the rule that stops it growing
#!/usr/bin/env bash
set -euo pipefail
# fails on anything not already in the baseline
vendor/bin/phpstan analyse --no-progress
# and the check that stops somebody regenerating it larger
before=$(git show "origin/${CI_DEFAULT_BRANCH}:phpstan-baseline.neon"
| grep -c 'message:' || echo 0)
after=$(grep -c 'message:' phpstan-baseline.neon)
if [ "$after" -gt "$before" ]; then
echo "baseline grew: $before → $after"
exit 1
fi
echo "baseline: $before → $after"The first command alone is most of the value: new code is analysed at the chosen level and old code is ignored, so the analyser is useful from the first day rather than after a cleanup project. The second is what stops the obvious workaround, which is regenerating the baseline to make a red build green — and that workaround is always available and always tempting at five o’clock on a Friday.
Printing the two numbers in every build makes the trend visible without anybody having to look for it, and a number going down without being asked to is the most reliable evidence that a quality initiative is actually working.
Levels, and deciding out loud where to stop
0-2 names that do not resolve. free, and every fix is an improvement.
3 return and property types the code does not state.
4 dead code, always-true conditions. mostly real findings.
5 argument types at every call site. a lot of docblocks.
6 a typehint on everything. months, on a mature codebase.
7-8 union handling and nullability. worth it for a library.
9 mixed is forbidden. rarely worth it for an application.
we stopped at 5, deliberately, and wrote down why.Level 3 is the first that asks for information the codebase does not contain, and the annotations it wants are usually describing a structure that has outgrown being an array — so the errors point at something real even when the fix is a docblock. Level 6 on this codebase was 3,880 additional errors representing months rather than days, and it was left.
Writing the chosen level and the reasoning into the configuration file as a comment is what stops the next person quietly raising it and generating a much larger baseline. Deciding out loud that a level is not worth reaching is a better outcome than a project that stalls at 4 while everyone assumes it is heading for 8.
What it does not find, which is most bugs
The claim static analysis attracts is that it replaces tests, and it does not — the two find disjoint sets of problems, and being clear about the boundary is what stops the tool being oversold internally and then distrusted.
finds does not find
----------------------------------------------------------
a method that does not a method that does the wrong
exist thing
an argument of the wrong an argument in the wrong order,
type both being strings
a null that is not guarded a rounding error
dead code a missing authorisation check
an unreachable branch an N+1 query
a typo in a property name a race between two requestsEvery item in the right column is a test’s job, and several of them are nobody’s job until a customer finds them. The correct framing is that the analyser raises the floor: the class of bug it removes is the one that used to be found by a user at the cost of an incident and is now found in nine seconds by a machine.
The suite has not become less important. It has stopped being the only thing checking anything, which frees it to be about behaviour rather than about whether the code runs at all — and that is a genuine improvement in what tests get written.
Extensions, for what the analyser cannot see
final class MacroMethodsExtension implements MethodsClassReflectionExtension
{
public function hasMethod(ClassReflection $class, string $name): bool
{
return $class->getName() === Collection::class
&& Collection::hasMacro($name);
}
// getMethod() returns a reflection describing the macro's signature
}
// phpstan.neon
// services:
// - class: AppPHPStanMacroMethodsExtension
// tags: [phpstan.broker.methodsClassReflectionExtension]Frameworks built on __call and facades produce hundreds of findings that are all the same false positive, and putting them in the baseline hides real errors of the same shape. The community extensions for Laravel and Symfony already do this, and installing one is the right first move — writing your own is for a magic method specific to your codebase.
On this project the Laravel extension removed 1,102 errors from the baseline, which is a quarter of it and none of it was debt. That is worth doing before generating the baseline rather than after, or the file records a great deal of noise as though it were work to be done.
Verifying it worked
$ git rm phpstan.neon && git checkout -- phpstan.neon # the 61 patterns, gone
$ wc -l phpstan.neon phpstan-baseline.neon
6 phpstan.neon
15,536 phpstan-baseline.neon
# the assertion that matters: a real bug, caught before review
$ git commit -m 'add refund handling'
$ vendor/bin/phpstan analyse --no-progress
------ ---------------------------------------------------------
Line src/Billing/Refund.php
------ ---------------------------------------------------------
41 Call to an undefined method AppOrder::totalCents()
(did you mean total()?)
# six months later
# baseline: 3,884 → 2,140That finding is the one that justifies the whole exercise: a method name that did not exist, on a class the author had not written, caught by a machine in nine seconds rather than by a customer in a fortnight. It is the class of bug the old pattern list had been silencing since 2017.
The baseline shrinking by 1,744 over six months without anybody being assigned to reduce it is the other outcome, and it happened because touching a file means the errors in it become visible during review — which is a much better forcing function than a cleanup ticket.
What this costs
A fifteen-thousand-line generated file in every diff that touches it, which makes some pull requests unreviewable in the usual sense. The mitigation is that nobody should read it — a diff that only changes the baseline is a diff where the interesting part is the count, and marking the file as generated in .gitattributes collapses it in most review tools. That is worth setting up on day one, before the first complaint.
The deeper risk is that a baseline makes the debt comfortable. Four thousand errors in a file nobody reads is easier to live with than four thousand errors in the build output, and a project that generates a baseline and never shrinks it has bought silence rather than quality. The ratchet in CI is what makes that visible, and the number in the build output is what makes it uncomfortable — both are the point, and a baseline without either is the same pattern list with better ergonomics.