queues – Page 5 – Türker Yıldırım

Bus::batch, and the completion callback that is not a job

NoteSeptember 6, 20221 min read

A batch gives progress and a completion hook for a set of jobs, and the callbacks run on the worker that finished the last one. The callbacks are…

A read-through cache that stampedes exactly once

Deep diveAugust 24, 202210 min read

A cache entry expired at 09:00 and four hundred requests recomputed it. A latency spike every hour, exactly on the hour.

A GDPR deletion request across eleven tables

Deep diveJuly 4, 202214 min read

A request arrived and nobody could say where the data was. Personal data in eleven tables, three of them denormalised copies.

A backed enum’s value is a contract once it reaches a queue

NoteJune 21, 20222 min read

An enum serialised into a queued job payload is a value that must still be resolvable when the job runs, which may be after the next deploy. This…

A queued closure is a serialised closure, with everything that implies

NoteJune 14, 20222 min read

Dispatching a closure is convenient and stores executable code in the queue, which is a different security and deployment posture from a job class. Running old code after…

The queue table that became a bottleneck at four million rows

Deep diveJune 3, 202210 min read

A database-backed queue that worked for four years. SELECT FOR UPDATE SKIP LOCKED, and a lock wait timeout.

Disabling an endpoint that has failed for a week

NoteMay 26, 20221 min read

Retrying forever against a dead endpoint is a queue that grows without bound, and disabling silently is worse than disabling loudly. The re-enable link in the email is…

A queue table grows because reserved_at never clears

NoteApril 29, 20221 min read

A database-backed queue marks a job reserved while it runs, and a worker killed mid-job leaves the row reserved forever. The retry-after window has to exceed the longest…

A Messenger middleware is where the transaction belongs

NoteApril 27, 20222 min read

Wrapping a handler in a transaction inside the handler couples the persistence decision to the business logic; a middleware applies it uniformly. The ordering is the part that…

Sharded pub/sub, and the cluster that could not broadcast

Deep diveApril 25, 20229 min read

A broadcast that goes to every node on every publish. Classic pub/sub does not scale in a cluster, and 7.0 gives channels that respect slots.