A search index that had been incrementally updated for three years, with no test that the increments were correct.
# monthly, into a new index behind an alias
$ ./bin/reindex --into=products-2024-03
documents: 412,884
duration: 22m
$ ./bin/index-diff products products-2024-03
missing from live: 41
stale in live: 188
extra in live: 7 (deleted products)
$ ./bin/alias-switch products products-2024-03The diff is the point rather than the rebuild — two hundred and thirty-six documents wrong out of four hundred thousand is a drift rate, and it comes from failed increments nobody retried. Rebuilding without diffing would have quietly fixed it every month and told us nothing about whether the incremental path was getting worse.