Celery Workers retry Redis connection when task result is

The Fix

pip install celery==5.1.0b1

Based on closed celery/celery issue #6533 · PR/commit linked

Production note: This usually shows up under retries/timeouts. Treat it as a side-effect risk until you can verify behavior with a canary + real traffic.

Jump to Verify Open PR/Commit

@@ -853,7 +853,11 @@ def _store_result(self, task_id, result, state,
             return result
 
-        self._set_with_state(self.get_key_for_task(task_id), self.encode(meta), state)
+        try:
+            self._set_with_state(self.get_key_for_task(task_id), self.encode(meta), state)

repro.py

#!/usr/bin/env python3

from celery import Celery

app = Celery(
    'tasks',
    broker='amqp://user:***@**:5672/**',
    backend='redis://:**@**:6379/1',
)

@app.task(ignore_result=False)
def test(*args, **kwargs):
    return 'x' * 536870911  # 512MB

verify

Re-run the minimal reproduction on your broken version, then apply the fix and re-run.

fix.md

Option A — Upgrade to fixed release\npip install celery==5.1.0b1\nWhen NOT to use: This fix is not applicable if task results are expected to exceed 512MB.\n\n

Why This Fix Works in Production

Trigger: raise ConnectionError("Error %s while writing to socket. %s." %
Mechanism: The system retries Redis connection when task result exceeds 512MB, causing unnecessary retries
Why the fix works: Raises a BackendStoreError when attempting to insert a Redis string value larger than 512MB, addressing issue #6533. (first fixed release: 5.1.0b1).

Production impact:

If left unfixed, retry loops can amplify load and turn a small outage into a cascade (thundering herd).

Why This Breaks in Prod

The system retries Redis connection when task result exceeds 512MB, causing unnecessary retries
Surfaces as: raise ConnectionError("Error %s while writing to socket. %s." %

Proof / Evidence

GitHub issue: #6533
Fix PR: https://github.com/celery/celery/pull/6629
First fixed release: 5.1.0b1
Reproduced locally: No (not executed)
Last verified: 2026-02-09
Confidence: 0.85
Did this fix it?: Yes (upstream fix exists)
Own content ratio: 0.60

Discussion

High-signal excerpts from the issue thread (symptoms, repros, edge-cases).

Jump to Sources Open on GitHub

“ # Checklist  - [x] I hav”

Issue thread · issue description · source

Failure Signature (Search String)

raise ConnectionError("Error %s while writing to socket. %s." %

Error Message

Stack trace

error.txt

Error Message
-------------
raise ConnectionError("Error %s while writing to socket. %s." %
redis.exceptions.ConnectionError: Error 32 while writing to socket. Broken pipe.

Minimal Reproduction

repro.py

#!/usr/bin/env python3

from celery import Celery

app = Celery(
    'tasks',
    broker='amqp://user:***@**:5672/**',
    backend='redis://:**@**:6379/1',
)

@app.task(ignore_result=False)
def test(*args, **kwargs):
    return 'x' * 536870911  # 512MB

What Broke

Workers fail to store task results larger than 512MB, leading to task failures.

Why It Broke

The system retries Redis connection when task result exceeds 512MB, causing unnecessary retries

Fix Options (Details)

Option A — Upgrade to fixed release Safe default (recommended)

pip install celery==5.1.0b1

When NOT to use: This fix is not applicable if task results are expected to exceed 512MB.

Use when you can deploy the upstream fix. It is usually lower-risk than long-lived workarounds.

Fix reference: https://github.com/celery/celery/pull/6629

First fixed release: 5.1.0b1

Last verified: 2026-02-09. Validate in your environment.

When NOT to Use This Fix

This fix is not applicable if task results are expected to exceed 512MB.

Verify Fix

verify

Re-run the minimal reproduction on your broken version, then apply the fix and re-run.

Did This Fix Work in Your Case?

Quick signal helps us prioritize which fixes to verify and improve.

Prevention

Make timeouts explicit and test them (unit + integration) to avoid silent behavior changes.
Instrument retries (attempt count + reason) and alert on spikes to catch dependency slowdowns.

Version Compatibility Table

Version	Status
5.1.0b1	Fixed

Related Issues

No related fixes found.

Cluster: celery:retry-storm Celery hub Celery best practices All hubs All clusters

Related clusters: Configuration error Data consistency Duplicates

Sources

We don’t republish the full GitHub discussion text. Use the links above for context.

Celery Workers retry Redis connection when task result is larger than 512MB - retry is useless as (Fix)