Journald rate limiting, and the log line that vanished

A worker logging a line per job hit the journal’s rate limit during a backlog, and the lines that were dropped were the ones describing the backlog.

# /etc/systemd/journald.conf defaults
RateLimitIntervalSec=30s
RateLimitBurst=10000

# which is per service, and generous until a queue drains
# 41,000 jobs in 30 seconds = 31,000 lines discarded,
# and journald says so, once:
#   Suppressed 31204 messages from /system.slice/worker.service

The suppression notice is itself a log line and it is the only evidence, so a search for the missing entries finds nothing and a search for “Suppressed” finds everything. The fix was not raising the limit — it was that the worker should not log a line per successful job. Per-job logging that exists for debugging becomes noise at volume, and a counter with a periodic summary carries the same information.