A queue table grows because reserved_at never clears

A database-backed queue marks a job reserved while it runs, and a worker killed mid-job leaves the row reserved forever.

SELECT COUNT(*), MIN(reserved_at) FROM jobs
WHERE reserved_at IS NOT NULL;
-- | 8104 | 2021-11-02 |

-- the release, which the framework does on a timer and
-- which is disabled surprisingly often:
UPDATE jobs SET reserved_at = NULL, attempts = attempts + 1
WHERE reserved_at < UNIX_TIMESTAMP() - 90;

The retry-after window has to exceed the longest job or a running job is released and picked up by a second worker, which is a duplicate execution caused by a timeout setting. A count of long-reserved rows is the metric that catches both this and a worker that is silently dead, and it is one query.