Three client teams were building against a document written the previous January. It described eleven endpoints, of which two no longer existed, one had a renamed field and one had changed its error format. Nobody had lied; the document was written once, carefully, and then the API carried on.
The symptom
# what the document says
$ jq -r '.paths | keys[]' docs/api.json
/orders
/orders/{id}
/orders/{id}/lines ← removed in August
/customers
# what the application has
$ php artisan route:list --path=api --json | jq -r '.[].uri' | sort
api/orders
api/orders/{order}
api/orders/{order}/refunds ← added in November, undocumented
api/customersTwo discrepancies in four endpoints, in opposite directions. The removed endpoint was still being called by one client, which had been getting a 404 and retrying. The added one was being used by another client that had discovered it by reading the source.
Why it happens
A hand-written document is a copy of the implementation, and every copy needs somebody to keep it in step. The person changing the endpoint is usually not the person who wrote the document, the document lives in a different place, and nothing in the review process notices that one changed and the other did not.
The structural fix is not more discipline. It is removing the copy: if the document is derived from the code, it cannot describe something the code does not do.
The fix
What to generate from
annotations verbose, next to the code, describes everything
including the response shape. drifts only if
somebody edits the code and not the comment —
which review can catch, because they are adjacent.
the route table plus request classes
free, always accurate about paths and parameters,
says nothing about responses.
the test suite accurate about everything the tests cover, and
silent about everything they do not.
we used annotations, and generated the paths from the router
as a cross-check.Generating from two sources and diffing them is the arrangement that caught the most: the router knows every path that exists and the annotations know what each one does, so a path with no annotation is a missing document and an annotation with no path is a stale one. Neither source alone reports that.
/**
* @OAGet(
* path="/api/orders/{id}",
* summary="Fetch one order",
* tags={"orders"},
* @OAParameter(name="id", in="path", required=true,
* @OASchema(type="integer", format="int64")),
* @OAResponse(response=200, description="The order",
* @OAJsonContent(ref="#/components/schemas/Order")),
* @OAResponse(response=404, description="No such order",
* @OAJsonContent(ref="#/components/schemas/Problem"))
* )
*/
public function show(int $id): JsonResponse
{
// ...
}The schemas are declared once as components and referenced, which is what stops the document being forty repetitions of the same object. Putting the schema annotation on the API resource class rather than in the controller keeps it next to the thing that produces the shape — which is where it will be noticed when the shape changes.
The document as a review artefact
Generating it is half the work. Committing the generated file is what turns an API change into something a reviewer can see without reading the controller.
- name: The API document is current
run: |
vendor/bin/openapi app -o /tmp/openapi.json
diff <(jq -S . public/openapi.json) <(jq -S . /tmp/openapi.json)
|| { echo 'run: composer docs:api'; exit 1; }# and what a reviewer now sees in the pull request
$ git diff public/openapi.json
- "contact_email": { "type": "string" },
+ "email": { "type": "string" },
# a field rename. visible. in the diff. before it ships.Sorting with jq -S before diffing removes key-order noise, without which the check fails on every run for no reason. A generated file in the repository is duplication and is worth it precisely here: the pull request that renames a field now shows the rename in a document rather than only in a resource class four files away.
Automated breaking-change detection is the next step and the tooling for it is immature in 2020 — openapi-diff exists and its classification of what is breaking is not reliable enough to gate a build. A human reading the diff is the mechanism, and it works because the diff is now small and legible.
Validating requests against the schema
A document claiming a field is required and an endpoint that accepts it missing is a bug the clients will find first, and there is a middleware that closes the gap.
final class ValidateAgainstSchema
{
public function handle($request, Closure $next)
{
if (app()->environment('production')) {
return $next($request);
}
$this->requestValidator->validate($this->toPsr($request));
$response = $next($request);
$this->responseValidator->validate($operation, $this->toPsr($response));
return $response;
}
}Validating the response is where this earns its place. The request side duplicates what the form request already does; the response side checks a claim nothing else checks, and it found four discrepancies on the first run — three nullable fields the document said were required, and one integer being returned as a string.
Running it outside production is the pragmatic arrangement, because full schema validation costs more than the request in some cases. Running it in the test suite instead is the version that scales: every feature test that hits an endpoint now also asserts that the response matches the document, at no extra cost in production.
Serving it, and the endpoint that must not be public
Route::middleware(['auth', 'can:view-api-docs'])->group(function () {
Route::view('/docs', 'swagger');
Route::get('/openapi.json', fn() => response()->file(
public_path('openapi.json'),
['Content-Type' => 'application/json']
));
});Serving the interactive documentation from the application is convenient and publishes a complete map of every endpoint, parameter and error code to anybody who finds the URL. For an internal API that is a poor default, and for a public one it is the entire point — the decision is per API rather than a habit.
The try-it button is the part that gets overlooked: it sends real requests with whatever credentials the browser has, against whatever server the document names. Pointing it at production from a documentation page is how somebody cancels an order while reading about how to cancel orders. Setting the servers block to staging only is the mitigation.
Generating a client, and when not to
A document that machines can read makes a generated client possible, and the two client teams that wanted one had very different reasons — which turned out to be the deciding factor rather than the technology.
$ openapi-generator generate -i public/openapi.json -g php -o ./client
$ find client -name '*.php' | wc -l
214
$ openapi-generator generate -i public/openapi.json
-g typescript-fetch -o ./client-ts
$ find client-ts -name '*.ts' | wc -l
38The PHP output is 214 files for eleven endpoints, in a style nobody chose, with a serialisation layer that duplicates what the framework already does. The TypeScript output is 38 files and is mostly interfaces, which is genuinely useful because the alternative was hand-written types that drift.
The rule that emerged is that generating types is nearly always worth it and generating a whole client rarely is. A generated interface file is a compile-time check with no runtime footprint; a generated client is a dependency with an opinion about error handling, retries and authentication, and all three of those were already decided.
# what we actually shipped for the TypeScript client
$ npx openapi-typescript public/openapi.json --output src/api/schema.d.ts
# 1 file. types only. no runtime code at all.
# the fetch wrapper stays hand-written and stays ours.One generated file of types alongside a hand-written client is the arrangement that gets the compile-time safety without importing somebody else’s architecture. It also regenerates cleanly, which a customised generated client never does — the first local modification is the last regeneration.
The parts a generator cannot produce
A schema describes shapes and says nothing about meaning, sequence or intent — and those are what a client integrator actually needs.
generated paths, methods, parameters, schemas, status codes
written by hand authentication: how to get a token, how long
it lasts, what happens when it does not
pagination: which style, and why
rate limits: the numbers, and the headers
idempotency: which endpoints, which header
a worked example: place an order, end to end
the changelog
the second list is a page of prose and is what people read first.Keeping that page in the same repository, in Markdown, next to the generated document is the arrangement that works — it is small enough to maintain and it is the thing an integrator opens before anything else. Trying to express it in annotations produces a document with a large description field nobody renders properly.
Verifying it worked
# the cross-check, which is what found the two discrepancies
$ diff
<(jq -r '.paths | keys[]' public/openapi.json | sed 's|^/||' | sort)
<(php artisan route:list --path=api --json
| jq -r '.[].uri' | sed 's|^api/||' | sort)
# and the suite, now asserting the response matches the document
$ php artisan test --filter Api
Tests: 84 passed
$ php artisan test --filter Api 2>&1 | grep -c 'schema mismatch'
0The path cross-check is the assertion that the document is complete rather than merely accurate, and it is the one that catches an endpoint added without documentation. Running both in CI means the failure arrives at the person who made the change rather than at a client three weeks later.
The check worth adding after a month is whether anybody is reading it: a request count on /openapi.json and on /docs. On this project the answer was that the three client teams read it constantly and internal developers never did, which is the correct outcome and is worth knowing before spending more on it.
What this costs
Annotations everywhere, and they are genuinely ugly. A controller method with a twenty-line docblock above six lines of code reads badly, and there is no version of this that does not — attributes in 8.0 improve the syntax and not the volume. The mitigation is putting the schema annotations on the resource classes and keeping the controller annotations to paths and responses, which halves it and is still a lot.
The second cost is a generator in the dependency tree that has to keep up with the specification. OpenAPI 3.1 arrives in 2021 with JSON Schema alignment, and a generator that lags means the document cannot use it — which is a dependency on somebody else’s release schedule for a document format. That is a smaller risk than a document nobody maintains, and it is worth naming rather than discovering.