An alert saying HighErrorRate on shop-web at three in the morning is a puzzle. The person receiving it may not have written the check, and the context lives…
The synchronous call between two services was the reason both were down. Publishing an event decouples their availability — and makes eventual consistency a business decision.
A log line written as a sentence has to be parsed back into fields by a regular expression somebody maintains, and that expression breaks whenever the sentence changes.…
Alerting on every cause — disk, memory, queue depth, replica lag, a failed cron — produces an alert per cause and a channel nobody reads. Alerting on what…