Alerting on every cause — disk, memory, queue depth, replica lag, a failed cron — produces an alert per cause and a channel nobody reads. Alerting on what…
A 400ms pause on every request, once an hour, caused by a cache cleanup nobody thought of as expensive. Redis is single-threaded, and freeing memory is work.
Every project has eight levels available and uses three, inconsistently, so error means “something a developer should look at” in one service and “a customer typed the wrong…
A shipper that follows a file by descriptor keeps reading the rotated-away file and never sees the new one; a shipper that reopens by name misses whatever was…
Daily indices grow linearly and nothing removes them, so a logging cluster works beautifully for four months and then stops accepting writes with a disk watermark error at…
SSH and HTTP basic auth are covered by shipped filters. A brute force against the application’s own login form is invisible to both, because it is a 200…
Centralised logs from four services are four streams in one place until something ties a single request together across them. That something has to be generated at the…