Celery Celery worker OOM due to unbounded prefetch on eta

The Fix

pip install celery==5.6.0

Based on closed celery/celery issue #9849 · PR/commit linked

Production note: This usually shows up under retries/timeouts. Treat it as a side-effect risk until you can verify behavior with a canary + real traffic.

Jump to Verify Open PR/Commit

@@ -342,6 +342,7 @@ def __repr__(self):
         proc_alive_timeout=Option(4.0, type='float'),
         prefetch_multiplier=Option(4, type='int'),
+        eta_task_limit=Option(None, type='int'),
         enable_prefetch_count_reduction=Option(True, type='bool'),
         redirect_stdouts=Option(

repro

this bug.

verify

Follow the reproduction steps, confirm the failure, apply the fix, and repeat the same steps to verify the behavior changes.

fix.md

Option A — Upgrade to fixed release\npip install celery==5.6.0\nWhen NOT to use: This fix is not appropriate if your application requires unlimited prefetching of ETA tasks.\n\n

Why This Fix Works in Production

Trigger: Celery worker OOM due to unbounded prefetch on eta tasks
Mechanism: Celery workers experience Out Of Memory issues due to unbounded prefetching of ETA tasks
Why the fix works: Introduces a new configuration option to limit the number of ETA tasks a worker can hold in memory, preventing Out Of Memory issues when fetching a large number of ETA tasks. (first fixed release: 5.6.0).

Production impact:

If left unfixed, this can cause silent data inconsistencies that propagate (bad cache entries, incorrect downstream decisions).

Why This Breaks in Prod

Celery workers experience Out Of Memory issues due to unbounded prefetching of ETA tasks
Production symptom (often without a traceback): Celery worker OOM due to unbounded prefetch on eta tasks

Proof / Evidence

GitHub issue: #9849
Fix PR: https://github.com/celery/celery/pull/9853
First fixed release: 5.6.0
Reproduced locally: No (not executed)
Last verified: 2026-02-08
Confidence: 0.70
Did this fix it?: Yes (upstream fix exists)
Own content ratio: 0.70

Discussion

High-signal excerpts from the issue thread (symptoms, repros, edge-cases).

Jump to Sources Open on GitHub

“we already got a contribution merged for this issue. do you have any other suggestion for this?”

@auvipy · 2025-12-17 · confirmation · source

“Memory exhaustion from backlogs is an interesting failure mode — it’s not just about correct scheduling but about resource control when there’s a big backlog”

@Caoska · 2025-12-16 · source

Failure Signature (Search String)

Celery worker OOM due to unbounded prefetch on eta tasks
We are encountering Out Of Memory infinite loop issues on celery workers, when our redis queue is suddenly loaded with either lot of failed tasks, or lot of scheduled tasks.

Copy-friendly signature

signature.txt

Failure Signature
-----------------
Celery worker OOM due to unbounded prefetch on eta tasks
We are encountering Out Of Memory infinite loop issues on celery workers, when our redis queue is suddenly loaded with either lot of failed tasks, or lot of scheduled tasks.

Error Message

Signature-only (no traceback captured)

error.txt

Error Message
-------------
Celery worker OOM due to unbounded prefetch on eta tasks
We are encountering Out Of Memory infinite loop issues on celery workers, when our redis queue is suddenly loaded with either lot of failed tasks, or lot of scheduled tasks.

Minimal Reproduction

this bug.

What Broke

Workers crash due to memory exhaustion when fetching too many scheduled tasks.

Why It Broke

Celery workers experience Out Of Memory issues due to unbounded prefetching of ETA tasks

Fix Options (Details)

Option A — Upgrade to fixed release Safe default (recommended)

pip install celery==5.6.0

When NOT to use: This fix is not appropriate if your application requires unlimited prefetching of ETA tasks.

Use when you can deploy the upstream fix. It is usually lower-risk than long-lived workarounds.

Fix reference: https://github.com/celery/celery/pull/9853

First fixed release: 5.6.0

Last verified: 2026-02-08. Validate in your environment.

When NOT to Use This Fix

This fix is not appropriate if your application requires unlimited prefetching of ETA tasks.

Verify Fix

verify

Follow the reproduction steps, confirm the failure, apply the fix, and repeat the same steps to verify the behavior changes.

Did This Fix Work in Your Case?

Quick signal helps us prioritize which fixes to verify and improve.

Prevention

Track RSS + object counts after deployments; alert on monotonic growth and GC pressure.
Add a long-running test that repeats the failing call path and asserts stable memory.

Version Compatibility Table

Version	Status
5.6.0	Fixed

Related Issues

No related fixes found.

Cluster: celery:data-consistency Celery hub Celery best practices All hubs All clusters

Related clusters: Configuration error Duplicates Duplicate processing

Sources

We don’t republish the full GitHub discussion text. Use the links above for context.

Celery Celery worker OOM due to unbounded prefetch on eta tasks (Fix)