Celery Chords contained in a group raise `AttributeError`s

The Fix

pip install celery==5.0.1

Based on closed celery/celery issue #6341 · PR/commit linked

Production note: This usually shows up under retries/timeouts. Treat it as a side-effect risk until you can verify behavior with a canary + real traffic.

Jump to Verify Open PR/Commit

@@ -1047,8 +1047,16 @@ class group(Signature):
     @classmethod
     def from_dict(cls, d, app=None):
+        # We need to mutate the `kwargs` element in place to avoid confusing
+        # `freeze()` implementations which end up here and expect to be able to
+        # access elements from that dictionary later and refer to objects

repro.py

import celery

app = celery.Celery("app", backend="redis://")

@app.task
def foo(*_):
    return 42

@app.task(bind=True)
def replace_with(self, sig):
    assert isinstance(sig, dict)
    sig = celery.Signature.from_dict(sig)
    raise self.replace(sig)

if __name__ == "__main__":
    sig = celery.group(
        celery.group(foo.s()),
    )
    res = sig.delay()
    print(res.get())

    sig.freeze()
    res = replace_with.delay(sig)
    print(res.get())

verify

Re-run the minimal reproduction on your broken version, then apply the fix and re-run.

fix.md

Option A — Upgrade to fixed release\npip install celery==5.0.1\nWhen NOT to use: Do not apply this fix if the signature structure is not guaranteed to be valid.\n\n

Why This Fix Works in Production

Trigger: [2020-09-08 12:44:05,453: DEBUG/MainProcess] Task accepted: app.replace_with[dcea02fd-23a3-404a-9fdd-b213eb51c0d1] pid:453431
Mechanism: The signature reconstruction from serialized dictionaries does not fully deserialize nested signatures
Why the fix works: This patch ensures that group tasks are deeply deserialized, fixing the issue where chords contained in a group raise AttributeErrors during freezing. (first fixed release: 5.0.1).

Production impact:

If left unfixed, this can cause silent data inconsistencies that propagate (bad cache entries, incorrect downstream decisions).

Why This Breaks in Prod

Shows up under Python 3.8 in real deployments (not just unit tests).
The signature reconstruction from serialized dictionaries does not fully deserialize nested signatures
Surfaces as: [2020-09-08 12:44:05,453: DEBUG/MainProcess] Task accepted: app.replace_with[dcea02fd-23a3-404a-9fdd-b213eb51c0d1] pid:453431

Proof / Evidence

GitHub issue: #6341
Fix PR: https://github.com/celery/celery/pull/6342
First fixed release: 5.0.1
Reproduced locally: No (not executed)
Last verified: 2026-02-09
Confidence: 0.85
Did this fix it?: Yes (upstream fix exists)
Own content ratio: 0.40

Discussion

High-signal excerpts from the issue thread (symptoms, repros, edge-cases).

Jump to Sources Open on GitHub

“This patch seems to fix the issue for the test case, and the idea might just need to be replicated to other signatures types that…”

@maybe-sybr · 2020-09-08 · source

Failure Signature (Search String)

[2020-09-08 12:44:05,453: DEBUG/MainProcess] Task accepted: app.replace_with[dcea02fd-23a3-404a-9fdd-b213eb51c0d1] pid:453431

Error Message

Stack trace

error.txt

Error Message
-------------
[2020-09-08 12:44:05,453: DEBUG/MainProcess] Task accepted: app.replace_with[dcea02fd-23a3-404a-9fdd-b213eb51c0d1] pid:453431
[2020-09-08 12:44:05,457: ERROR/ForkPoolWorker-8] Task app.replace_with[dcea02fd-23a3-404a-9fdd-b213eb51c0d1] raised unexpected: AttributeError("'dict' object has no attribute '_app'")
Traceback (most recent call last):
  File "/home/maybe/tmp/capp/venv/lib64/python3.8/site-packages/kombu/utils/objects.py", line 41, in __get__
    return obj.__dict__[self.__name__]
KeyError: 'app'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/maybe/tmp/capp/venv/lib64/python3.8/site-packages/celery/app/trace.py", line 409, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/home/maybe/tmp/capp/venv/lib64/python3.8/site-packages/celery/app/trace.py", line 701, in __protected_call__
    return self.run(*args, **kwargs)
  File "/home/maybe/tmp/capp/app.py", line 13, in replace_with
    raise self.replace(sig)
  File "/home/maybe/tmp/capp/venv/lib64/python3.8/site-packages/celery/app/task.py", line 894, in replace
    sig.freeze(self.request.id)
  File "/home/maybe/tmp/capp/venv/lib64/python3.8/site-packages/celery/canvas.py", line 1302, in freeze
    self.tasks = group(self.tasks, app=self.app)
  File "/home/maybe/tmp/capp/venv/lib64/python3.8/site-packages/kombu/utils/objects.py", line 43, in
... (truncated) ...

Minimal Reproduction

repro.py

import celery

app = celery.Celery("app", backend="redis://")

@app.task
def foo(*_):
    return 42

@app.task(bind=True)
def replace_with(self, sig):
    assert isinstance(sig, dict)
    sig = celery.Signature.from_dict(sig)
    raise self.replace(sig)

if __name__ == "__main__":
    sig = celery.group(
        celery.group(foo.s()),
    )
    res = sig.delay()
    print(res.get())

    sig.freeze()
    res = replace_with.delay(sig)
    print(res.get())

Environment

Python: 3.8

What Broke

AttributeErrors occur when attempting to freeze tasks with nested signatures, leading to task failures.

Why It Broke

The signature reconstruction from serialized dictionaries does not fully deserialize nested signatures

Fix Options (Details)

Option A — Upgrade to fixed release Safe default (recommended)

pip install celery==5.0.1

When NOT to use: Do not apply this fix if the signature structure is not guaranteed to be valid.

Use when you can deploy the upstream fix. It is usually lower-risk than long-lived workarounds.

Option D — Guard side-effects with OnceOnly Guardrail for side-effects

Mitigate duplicate external side-effects under retries/timeouts/agent loops by gating the operation before calling external systems.

Place OnceOnly between your code/agent and real side-effects (Stripe, emails, CRM, APIs).
Use a stable key per side-effect (e.g., customer_id + action + idempotency_key).
Fail-safe: configure fail-open vs fail-closed based on blast radius and spend risk.
This does NOT fix data corruption; it only prevents duplicate side-effects.

Show example snippet (optional)

onceonly.py

from onceonly import OnceOnly
import os

once = OnceOnly(api_key=os.environ["ONCEONLY_API_KEY"], fail_open=True)

# Stable idempotency key per real side-effect.
# Use a request id / job id / webhook delivery id / Stripe event id, etc.
event_id = "evt_..."  # replace
key = f"stripe:webhook:{event_id}"

res = once.check_lock(key=key, ttl=3600)
if res.duplicate:
    return {"status": "already_processed"}

# Safe to execute the side-effect exactly once.
handle_event(event_id)

See OnceOnly SDK

When NOT to use: Do not use this to hide logic bugs or data corruption. Use it to block duplicate external side-effects and enforce tool permissions/spend caps.

Fix reference: https://github.com/celery/celery/pull/6342

First fixed release: 5.0.1

Last verified: 2026-02-09. Validate in your environment.

When NOT to Use This Fix

Do not apply this fix if the signature structure is not guaranteed to be valid.
Do not use this to hide logic bugs or data corruption. Use it to block duplicate external side-effects and enforce tool permissions/spend caps.

Verify Fix

verify

Re-run the minimal reproduction on your broken version, then apply the fix and re-run.

Did This Fix Work in Your Case?

Quick signal helps us prioritize which fixes to verify and improve.

Prevention

Capture the exact failing error string in logs and tests so you can reproduce via a minimal script.
Pin production dependencies and upgrade only with a reproducible test that hits the failing path.

Version Compatibility Table

Version	Status
5.0.1	Fixed

Related Issues

No related fixes found.

Cluster: celery:data-consistency Celery hub Celery best practices All hubs All clusters

Related clusters: Configuration error Duplicates Duplicate processing

Sources

We don’t republish the full GitHub discussion text. Use the links above for context.

Celery Chords contained in a group raise `AttributeError`s during freezing if passed through a (Fix)