Celery regen() duplicates any non-list and non-tuple

The Fix

pip install celery==5.1.1

Based on closed celery/celery issue #6786 · PR/commit linked

Production note: This usually shows up under retries/timeouts. Treat it as a side-effect risk until you can verify behavior with a canary + real traffic.

Jump to Verify Open PR/Commit

@@ -241,12 +241,18 @@ def __bool__(self):
     @property
     def data(self):
-        try:
-            self.__consumed.extend(list(self.__it))
-        except StopIteration:

repro.py

In [1]: from celery.utils.functional import regen
In [2]: from collections import deque
In [3]: l = regen(deque([1]))
In [4]: l
Out[4]: [1]
In [5]: l
Out[5]: [1, 1]
In [6]: l
Out[6]: [1, 1, 1]

verify

Re-run the minimal reproduction on your broken version, then apply the fix and re-run.

fix.md

Option A — Upgrade to fixed release\npip install celery==5.1.1\nWhen NOT to use: This fix is not applicable if the `regen()` behavior is intentionally lazy.\n\n

Why This Fix Works in Production

Trigger: regen() duplicates any non-list and non-tuple sequence
Mechanism: The `regen()` instances were not marked as done, leading to element duplication
Why the fix works: Ensures that `regen()` instances are marked as done when concretised, preventing duplication of elements. (first fixed release: 5.1.1).

Production impact:

If left unfixed, retries/timeouts can trigger duplicate external side-effects (double charges, duplicate emails, repeated writes).

Why This Breaks in Prod

Triggered by an upgrade/regression window: 3.9.0 breaks; 5.1.1 is the first fixed release.
The `regen()` instances were not marked as done, leading to element duplication
Production symptom (often without a traceback): regen() duplicates any non-list and non-tuple sequence

Proof / Evidence

GitHub issue: #6786
Fix PR: https://github.com/celery/celery/pull/6789
First fixed release: 5.1.1
Affected versions: 3.9.0
Reproduced locally: No (not executed)
Last verified: 2026-02-09
Confidence: 0.85
Did this fix it?: Yes (upstream fix exists)
Own content ratio: 0.73

Discussion

High-signal excerpts from the issue thread (symptoms, repros, edge-cases).

Jump to Sources Open on GitHub

“Quick bisect suggest this has been broken since 998277302 when regen was first rewritten to make it lazy”

@maybe-sybr · 2021-05-28 · source

“Hey @Suor :wave:, Thank you for opening an issue”

@open-collective-bot · 2021-05-26 · source

Failure Signature (Search String)

regen() duplicates any non-list and non-tuple sequence
- [x] I have included all related issues and possible duplicate issues

Copy-friendly signature

signature.txt

Failure Signature
-----------------
regen() duplicates any non-list and non-tuple sequence
- [x] I have included all related issues and possible duplicate issues

Error Message

Signature-only (no traceback captured)

error.txt

Error Message
-------------
regen() duplicates any non-list and non-tuple sequence
- [x] I have included all related issues and possible duplicate issues

Minimal Reproduction

repro.py

In [1]: from celery.utils.functional import regen
In [2]: from collections import deque
In [3]: l = regen(deque([1]))
In [4]: l
Out[4]: [1]
In [5]: l
Out[5]: [1, 1]
In [6]: l
Out[6]: [1, 1, 1]

What Broke

Elements were duplicated when accessing the `data` property of `regen()` instances.

Why It Broke

The `regen()` instances were not marked as done, leading to element duplication

Fix Options (Details)

Option A — Upgrade to fixed release Safe default (recommended)

pip install celery==5.1.1

When NOT to use: This fix is not applicable if the `regen()` behavior is intentionally lazy.

Use when you can deploy the upstream fix. It is usually lower-risk than long-lived workarounds.

Fix reference: https://github.com/celery/celery/pull/6789

First fixed release: 5.1.1

Last verified: 2026-02-09. Validate in your environment.

When NOT to Use This Fix

This fix is not applicable if the `regen()` behavior is intentionally lazy.

Verify Fix

verify

Re-run the minimal reproduction on your broken version, then apply the fix and re-run.

Did This Fix Work in Your Case?

Quick signal helps us prioritize which fixes to verify and improve.

Prevention

Capture the exact failing error string in logs and tests so you can reproduce via a minimal script.
Pin production dependencies and upgrade only with a reproducible test that hits the failing path.

Version Compatibility Table

Version	Status
3.9.0	Broken
5.1.1	Fixed

Related Issues

No related fixes found.

Cluster: celery:duplicate-processing Celery hub Celery best practices All hubs All clusters

Related clusters: Configuration error Data consistency Duplicates

Sources

We don’t republish the full GitHub discussion text. Use the links above for context.

Celery regen() duplicates any non-list and non-tuple sequence (Fix)