system-design – Page 24 – Türker Yıldırım

A read model is a copy, and copies need a story about staleness

NoteOctober 17, 20192 min read

Replicating the data a second service reads removes the shared schema and introduces a lag, and the lag is a product decision rather than a technical detail. Storing…

Fatal errors that reach a person before they reach a customer

Deep diveOctober 2, 201910 min read

The error tracker had 400,000 events and nobody had opened it in a month. Alert fatigue, in the one tool that exists to prevent it.

Kubernetes is not the answer to this question

Deep diveSeptember 16, 201915 min read

The question was how to deploy four services, and the answer arrived before it. Naming what you actually need — scheduling, health, rolling deploys, secrets.

Alert on symptoms; put the causes on a dashboard

NoteAugust 13, 20192 min read

CPU, memory and disk are causes, and there are unlimited causes — most of them entirely compatible with a healthy system serving every request correctly. The test for…

The queue worker that leaked, and the memory limit that hid it

Deep diveJuly 15, 201910 min read

A worker restarting every eleven minutes, reported as healthy. Finding the leak is not the same as bounding it, and the seatbelt hides the next one.

Messenger, and the queue that is part of the framework

Deep diveJune 25, 201910 min read

4.3 made Messenger stable, and dispatch stopped being the same decision as transport. A message, a handler, and routing declared in configuration.

A cache that survives the thing it is caching

Deep diveMay 8, 201914 min read

The cache expired at noon and the database fell over at 12:00:01. A lock, a two-tier expiry, jittered TTLs, and versioned keys — in that order of importance.

Elasticsearch 7 removed mapping types

Deep diveApril 30, 201910 min read

One type per index in 6, none at all in 7, and every query URL changes. The reindex cannot be done in place — and the default shard…

Timeouts are a concurrency limit only if you also cap concurrency

NoteApril 16, 20191 min read

Setting a thirty-second timeout on an HTTP call feels responsible and is not a limit on anything that matters when the upstream is merely slow rather than down.…

Bulkheads: one slow dependency should not take the pool

NoteMarch 20, 20192 min read

Fifty workers all waiting on the same upstream is a site that is down because of a service most of its pages do not use. A timeout bounds…