Celery Chord with a chord in header doesn't invoke on

The Fix

pip install celery==5.5.0

Based on closed celery/celery issue #9107 · PR/commit linked

Production note: This usually shows up under retries/timeouts. Treat it as a side-effect risk until you can verify behavior with a canary + real traffic.

Jump to Verify Open PR/Commit

@@ -2307,6 +2307,13 @@ def link_error(self, errback):
             )
 
+        # Edge case for nested chords in the header
+        for task in maybe_list(self.tasks) or []:
+            if isinstance(task, chord):

repro.py

app = Celery(
    'hello',
    broker='amqp://localhost//',
    backend='redis://localhost:6379/0',
    task_acks_late=True,
    task_reject_on_worker_lost=True,
)


@app.task(name='ok')
def ok(*args, **kwargs):
    print('ok')
    return 'ok'


@app.task(name='bad')
def bad(*args, **kwargs):
    print('bad')
    raise Exception('bad')


@app.task(name='on_error_handler')
def on_error_handler(request, exc, traceback):
    print('error detected')


@app.task(name='graph')
def graph():
    print('running graph')
    chord_inner = chord(
        (
            bad.s(),
            ok.s(),
        ),
        ok.s(),
    )

    chord_outer = chord(
        (
            bad.s(),
            chord_inner,
        ),
        ok.s(),
    )
    chord_outer.on_error(on_error_handler.s())
    chord_outer.delay()

verify

Re-run the minimal reproduction on your broken version, then apply the fix and re-run.

fix.md

Option A — Upgrade to fixed release\npip install celery==5.5.0\nWhen NOT to use: This fix should not be used if the application relies on the previous error handling behavior.\n\n

Why This Fix Works in Production

Trigger: [2024-06-27 12:39:39,979: INFO/MainProcess] Task bad[d285c927-aa7c-4167-9732-d914c9094498] received
Mechanism: The error callback was not invoked for nested chords in the header due to incorrect error linking
Why the fix works: Fixes the issue where a chord with a chord in the header doesn't invoke the error callback on inner chord header failure. (first fixed release: 5.5.0).

Production impact:

If left unfixed, this can cause silent data inconsistencies that propagate (bad cache entries, incorrect downstream decisions).

Why This Breaks in Prod

Shows up under Python 3.12 in real deployments (not just unit tests).
The error callback was not invoked for nested chords in the header due to incorrect error linking
Surfaces as: [2024-06-27 12:39:39,979: INFO/MainProcess] Task bad[d285c927-aa7c-4167-9732-d914c9094498] received

Proof / Evidence

GitHub issue: #9107
Fix PR: https://github.com/celery/celery/pull/9580
First fixed release: 5.5.0
Reproduced locally: No (not executed)
Last verified: 2026-02-09
Confidence: 0.95
Did this fix it?: Yes (upstream fix exists)
Own content ratio: 0.37

Discussion

High-signal excerpts from the issue thread (symptoms, repros, edge-cases).

Jump to Sources Open on GitHub

“Fixed in main. @naktinis @JerroldJV @crayola you’re welcome to test it out. Will be released soon in the next release candidate for v5.5.”

@Nusnus · 2025-02-24 · confirmation · source

“I also am seeing this issue, but also if the header tasks contain chains or groups”

@JerroldJV · 2024-07-31 · source

“I am also affected on 5.5.0rc2”

@crayola · 2024-11-28 · source

“> Is task_allow_error_cb_on_chord_header turned on? See docs”

@naktinis · 2024-08-14 · repro detail · source

Failure Signature (Search String)

[2024-06-27 12:39:39,979: INFO/MainProcess] Task bad[d285c927-aa7c-4167-9732-d914c9094498] received

Error Message

Stack trace

error.txt

Error Message
-------------
[2024-06-27 12:39:39,979: INFO/MainProcess] Task bad[d285c927-aa7c-4167-9732-d914c9094498] received
[2024-06-27 12:39:39,980: WARNING/ForkPoolWorker-1] bad
[2024-06-27 12:39:39,989: INFO/MainProcess] Task bad[b3e91204-905c-41e5-ae34-d398e69ce1a9] received
[2024-06-27 12:39:39,990: INFO/MainProcess] Task ok[9053fb18-f333-417c-bc7b-04c95dedfb1a] received
[2024-06-27 12:39:39,991: WARNING/ForkPoolWorker-9] bad
[2024-06-27 12:39:39,991: INFO/ForkPoolWorker-8] Task graph[a728c96b-5935-48ef-ad71-71cb6b50a58c] succeeded in 0.04279591701924801s: None
[2024-06-27 12:39:39,992: WARNING/ForkPoolWorker-2] ok
[2024-06-27 12:39:39,992: ERROR/ForkPoolWorker-1] Task bad[d285c927-aa7c-4167-9732-d914c9094498] raised unexpected: Exception('bad')
Traceback (most recent call last):
  File "/Users/foo/code/test-celery/venv/lib/python3.12/site-packages/celery/app/trace.py", line 453, in trace_task
    R = retval = fun(*args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^
  File "/Users/foo/code/test-celery/venv/lib/python3.12/site-packages/celery/app/trace.py", line 736, in __protected_call__
    return self.run(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/foo/code/test-celery/tasks.py", line 24, in bad
    raise Exception('bad')
Exception: bad
[2024-06-27 12:39:39,998: ERROR/ForkPoolWorker-9] Task bad[b3e91204-905c-41e5-ae34-d398e69ce1a9] raised unexpected: Exception('bad')
... (truncated) ...

Minimal Reproduction

repro.py

app = Celery(
    'hello',
    broker='amqp://localhost//',
    backend='redis://localhost:6379/0',
    task_acks_late=True,
    task_reject_on_worker_lost=True,
)


@app.task(name='ok')
def ok(*args, **kwargs):
    print('ok')
    return 'ok'


@app.task(name='bad')
def bad(*args, **kwargs):
    print('bad')
    raise Exception('bad')


@app.task(name='on_error_handler')
def on_error_handler(request, exc, traceback):
    print('error detected')


@app.task(name='graph')
def graph():
    print('running graph')
    chord_inner = chord(
        (
            bad.s(),
            ok.s(),
        ),
        ok.s(),
    )

    chord_outer = chord(
        (
            bad.s(),
            chord_inner,
        ),
        ok.s(),
    )
    chord_outer.on_error(on_error_handler.s())
    chord_outer.delay()

Environment

Python: 3.12

What Broke

Tasks in production failed to release external locks when header tasks failed, causing potential deadlocks.

Why It Broke

The error callback was not invoked for nested chords in the header due to incorrect error linking

Fix Options (Details)

Option A — Upgrade to fixed release Safe default (recommended)

pip install celery==5.5.0

When NOT to use: This fix should not be used if the application relies on the previous error handling behavior.

Use when you can deploy the upstream fix. It is usually lower-risk than long-lived workarounds.

Fix reference: https://github.com/celery/celery/pull/9580

First fixed release: 5.5.0

Last verified: 2026-02-09. Validate in your environment.

When NOT to Use This Fix

This fix should not be used if the application relies on the previous error handling behavior.

Verify Fix

verify

Re-run the minimal reproduction on your broken version, then apply the fix and re-run.

Did This Fix Work in Your Case?

Quick signal helps us prioritize which fixes to verify and improve.

Prevention

Capture the exact failing error string in logs and tests so you can reproduce via a minimal script.
Pin production dependencies and upgrade only with a reproducible test that hits the failing path.

Version Compatibility Table

Version	Status
5.5.0	Fixed

Related Issues

No related fixes found.

Cluster: celery:data-consistency Celery hub Celery best practices All hubs All clusters

Related clusters: Configuration error Duplicates Duplicate processing

Sources

We don’t republish the full GitHub discussion text. Use the links above for context.

Celery Chord with a chord in header doesn't invoke on_error on header failure (Fix)