The Fix
pip install celery==5.2.2
Based on closed celery/celery issue #5281 · PR/commit linked
Production note: This usually shows up under retries/timeouts. Treat it as a side-effect risk until you can verify behavior with a canary + real traffic.
@@ -198,7 +198,7 @@ STS token authentication
AWS STS authentication is supported by using the ``sts_role_arn`` and ``sts_token_timeout`` broker transport options. ``sts_role_arn`` is the assumed IAM role ARN we use to authorize our access to SQS.
-``sts_token_timeout`` is the token timeout, defaults (and minimum) to 900 seconds. After the mentioned period, a new token will be created.
+``sts_token_timeout`` is the token timeout, defaults (and minimum) to 900 seconds. After the mentioned period, a new token will be created::
repro.py
custom_properties = {
'MessageGroupId': 'YourMessageGroupId',
'MessageDeduplicationId': 'YourMessageDeduplicationId'
}
your_task.apply_async(**custom_properties)
verify
Re-run the minimal reproduction on your broken version, then apply the fix and re-run.
fix.md
Option A — Upgrade to fixed release\npip install celery==5.2.2\nWhen NOT to use: This fix is not applicable if using non-FIFO SQS queues.\n\n
Why This Fix Works in Production
- Trigger: Is there a way to propagate a value as the SQS `MessageDeduplicationId`? Kombu supports it as a message property, but I cannot see a way to propagate it from a…
- Mechanism: Lack of documentation on how to set MessageGroupId and MessageDeduplicationId for SQS in Celery
- Why the fix works: Added documentation on how to set additional message properties such as MessageGroupId and MessageDeduplicationId for SQS in Celery. (first fixed release: 5.2.2).
Production impact:
- If left unfixed, this can cause silent data inconsistencies that propagate (bad cache entries, incorrect downstream decisions).
Why This Breaks in Prod
- Lack of documentation on how to set MessageGroupId and MessageDeduplicationId for SQS in Celery
- Production symptom (often without a traceback): Is there a way to propagate a value as the SQS `MessageDeduplicationId`? Kombu supports it as a message property, but I cannot see a way to propagate it from a Celery task's enqueueing.
Proof / Evidence
- GitHub issue: #5281
- Fix PR: https://github.com/celery/celery/pull/7167
- First fixed release: 5.2.2
- Reproduced locally: No (not executed)
- Last verified: 2026-02-09
- Confidence: 0.85
- Did this fix it?: Yes (upstream fix exists)
- Own content ratio: 0.54
Discussion
High-signal excerpts from the issue thread (symptoms, repros, edge-cases).
“I'm wondering about this and MessageGroupId too”
“@ikonst I'm looking into the same issue now”
“@ant- I'm no longer involved with the project that needed that set up, but we did have multiple workers. Eventually we added a separate deduplication…”
“> @ant- I'm no longer involved with the project that needed that set up, but we did have multiple workers. Eventually we added a separate…”
Failure Signature (Search String)
- Is there a way to propagate a value as the SQS `MessageDeduplicationId`? Kombu supports it as a message property, but I cannot see a way to propagate it from a Celery task's
- Also, it could potentially be propagated from a Task Id, although I'm not sure whether Task Ids being de-duped on will break the existing contract.
Copy-friendly signature
signature.txt
Failure Signature
-----------------
Is there a way to propagate a value as the SQS `MessageDeduplicationId`? Kombu supports it as a message property, but I cannot see a way to propagate it from a Celery task's enqueueing.
Also, it could potentially be propagated from a Task Id, although I'm not sure whether Task Ids being de-duped on will break the existing contract.
Error Message
Signature-only (no traceback captured)
error.txt
Error Message
-------------
Is there a way to propagate a value as the SQS `MessageDeduplicationId`? Kombu supports it as a message property, but I cannot see a way to propagate it from a Celery task's enqueueing.
Also, it could potentially be propagated from a Task Id, although I'm not sure whether Task Ids being de-duped on will break the existing contract.
Minimal Reproduction
repro.py
custom_properties = {
'MessageGroupId': 'YourMessageGroupId',
'MessageDeduplicationId': 'YourMessageDeduplicationId'
}
your_task.apply_async(**custom_properties)
What Broke
Users unable to deduplicate messages in SQS, leading to potential duplicates.
Why It Broke
Lack of documentation on how to set MessageGroupId and MessageDeduplicationId for SQS in Celery
Fix Options (Details)
Option A — Upgrade to fixed release Safe default (recommended)
pip install celery==5.2.2
When NOT to use: This fix is not applicable if using non-FIFO SQS queues.
Use when you can deploy the upstream fix. It is usually lower-risk than long-lived workarounds.
Fix reference: https://github.com/celery/celery/pull/7167
First fixed release: 5.2.2
Last verified: 2026-02-09. Validate in your environment.
When NOT to Use This Fix
- This fix is not applicable if using non-FIFO SQS queues.
Verify Fix
verify
Re-run the minimal reproduction on your broken version, then apply the fix and re-run.
Did This Fix Work in Your Case?
Quick signal helps us prioritize which fixes to verify and improve.
Prevention
- Capture the exact failing error string in logs and tests so you can reproduce via a minimal script.
- Pin production dependencies and upgrade only with a reproducible test that hits the failing path.
Version Compatibility Table
| Version | Status |
|---|---|
| 5.2.2 | Fixed |
Related Issues
No related fixes found.
Sources
We don’t republish the full GitHub discussion text. Use the links above for context.