Celery Duplicate executions and django-celery-results

The Fix

pip install celery==5.4.0

Based on closed celery/celery issue #8662 · PR/commit linked

Production note: This usually shows up under retries/timeouts. Treat it as a side-effect risk until you can verify behavior with a canary + real traffic.

Jump to Verify Open PR/Commit

@@ -958,6 +958,8 @@ def __or__(self, other):
             # unroll group with one member
             other = maybe_unroll_group(other)
+            if not isinstance(other, group):
+                return self.__or__(other)
             # chain | group() -> chain

repro.py

from celery import Celery
from celery.canvas import chain, group

app = Celery(
    "myapp",
    broker="amqp://",
    backend="redis://",
)


@app.task
def identity(x):
    return x


c = chain(
    group([identity.si(i) for i in range(5)]),
    group([identity.si(i) for i in range(10, 15)]),
    group([identity.si(i) for i in range(20, 25)]),
    identity.s(),
)

c.delay()

if __name__ == "__main__":
    app.start()

verify

Re-run the minimal reproduction on your broken version, then apply the fix and re-run.

fix.md

Option A — Upgrade to fixed release\npip install celery==5.4.0\nWhen NOT to use: This fix should not be applied if the upgrade mechanism is expected to handle other scenarios.\n\n

Why This Fix Works in Production

Trigger: Duplicate executions and django-celery-results backend exceptions when chaining 4x number of groups
Mechanism: The chain upgrade leads to consecutive chords, causing tasks to execute multiple times
Why the fix works: Eliminates consecutive chords generated by group and task upgrades, addressing the issue of duplicate executions. (first fixed release: 5.4.0).

Production impact:

If left unfixed, retries/timeouts can trigger duplicate external side-effects (double charges, duplicate emails, repeated writes).

Why This Breaks in Prod

The chain upgrade leads to consecutive chords, causing tasks to execute multiple times
Production symptom (often without a traceback): Duplicate executions and django-celery-results backend exceptions when chaining 4x number of groups

Proof / Evidence

GitHub issue: #8662
Fix PR: https://github.com/celery/celery/pull/8663
First fixed release: 5.4.0
Reproduced locally: No (not executed)
Last verified: 2026-02-09
Confidence: 0.75
Did this fix it?: Yes (upstream fix exists)
Own content ratio: 0.57

Discussion

High-signal excerpts from the issue thread (symptoms, repros, edge-cases).

Jump to Sources Open on GitHub

“Hey there, I am still studying this issue but I want to comment on the following: > Under the current behavior, these four groups are…”

@Nusnus · 2023-11-23 · source

“Holy moly bug confirmed ! @auvipy @thedrow FYI ## Test case Based on the unit test from #8663 ## WITH FIX FROM PR AS-IS ##…”

@Nusnus · 2023-11-23 · source

“I believe the root cause of the issue lies in the chain upgrade leading to consecutive chords”

@hann-wang · 2023-11-23 · source

“I am goin to check this and come back to you with my thoughts”

@auvipy · 2023-11-23 · source

Failure Signature (Search String)

Duplicate executions and django-celery-results backend exceptions when chaining 4x number of groups
- [x] I have included all related issues and possible duplicate issues

Copy-friendly signature

signature.txt

Failure Signature
-----------------
Duplicate executions and django-celery-results backend exceptions when chaining 4x number of groups
- [x] I have included all related issues and possible duplicate issues

Error Message

Signature-only (no traceback captured)

error.txt

Error Message
-------------
Duplicate executions and django-celery-results backend exceptions when chaining 4x number of groups
- [x] I have included all related issues and possible duplicate issues

Minimal Reproduction

repro.py

from celery import Celery
from celery.canvas import chain, group

app = Celery(
    "myapp",
    broker="amqp://",
    backend="redis://",
)


@app.task
def identity(x):
    return x


c = chain(
    group([identity.si(i) for i in range(5)]),
    group([identity.si(i) for i in range(10, 15)]),
    group([identity.si(i) for i in range(20, 25)]),
    identity.s(),
)

c.delay()

if __name__ == "__main__":
    app.start()

What Broke

Tasks in a chain are executed multiple times instead of once, leading to unexpected behavior.

Why It Broke

The chain upgrade leads to consecutive chords, causing tasks to execute multiple times

Fix Options (Details)

Option A — Upgrade to fixed release Safe default (recommended)

pip install celery==5.4.0

When NOT to use: This fix should not be applied if the upgrade mechanism is expected to handle other scenarios.

Use when you can deploy the upstream fix. It is usually lower-risk than long-lived workarounds.

Fix reference: https://github.com/celery/celery/pull/8663

First fixed release: 5.4.0

Last verified: 2026-02-09. Validate in your environment.

When NOT to Use This Fix

This fix should not be applied if the upgrade mechanism is expected to handle other scenarios.

Verify Fix

verify

Re-run the minimal reproduction on your broken version, then apply the fix and re-run.

Did This Fix Work in Your Case?

Quick signal helps us prioritize which fixes to verify and improve.

Prevention

Capture the exact failing error string in logs and tests so you can reproduce via a minimal script.
Pin production dependencies and upgrade only with a reproducible test that hits the failing path.

Version Compatibility Table

Version	Status
5.4.0	Fixed

Related Issues

No related fixes found.

Cluster: celery:duplicate-processing Celery hub Celery best practices All hubs All clusters

Related clusters: Configuration error Data consistency Duplicates

Sources

We don’t republish the full GitHub discussion text. Use the links above for context.

Celery Duplicate executions and django-celery-results backend exceptions when chaining 4x number of (Fix)