Celery dynamoDB result backend incorrect exception handling

The Fix

Fixes incorrect exception handling when checking for the existence of a DynamoDB table by using `describe_table` instead of `create_table`.

Based on closed celery/celery issue #6622 · PR/commit linked

Production note: Most teams hit this during upgrades or environment changes. Roll out with a canary and smoke critical endpoints (health, OpenAPI/docs) before 100%.

Jump to Verify Open PR/Commit

@@ -201,28 +201,25 @@ def _get_or_create_table(self):
         table_schema = self._get_table_schema()
         try:
-            table_description = self._client.create_table(**table_schema)
-            logger.info(
-                'DynamoDB Table {} did not exist, creating.'.format(

repro.py

"dynamodb:DescribeTable",
   "dynamodb:PutItem",
   "dynamodb:UpdateItem",
   "dynamodb:DeleteItem",
   "dynamodb:BatchWriteItem",
   "dynamodb:GetItem",
   "dynamodb:BatchGetItem",
   "dynamodb:Scan",
   "dynamodb:Query",
   "dynamodb:ConditionCheckItem"

verify

Re-run the minimal reproduction on your broken version, then apply the fix and re-run.

fix.md

Option A — Apply the official fix\nFixes incorrect exception handling when checking for the existence of a DynamoDB table by using `describe_table` instead of `create_table`.\nWhen NOT to use: Do not use this fix if the role has CreateTable permission, as it may lead to unnecessary complexity.\n\n

Why This Fix Works in Production

Trigger: dynamoDB result backend incorrect exception handling when table exists
Mechanism: The code attempts to create a DynamoDB table without the necessary permissions, leading to unexpected exceptions

Production impact:

If left unfixed, the same config can fail only in production (env differences), causing startup failures or partial feature outages.

Why This Breaks in Prod

The code attempts to create a DynamoDB table without the necessary permissions, leading to unexpected exceptions
Production symptom (often without a traceback): dynamoDB result backend incorrect exception handling when table exists

Proof / Evidence

GitHub issue: #6622
Fix PR: https://github.com/celery/celery/pull/7734
Reproduced locally: No (not executed)
Last verified: 2026-02-11
Confidence: 0.70
Did this fix it?: Yes (upstream fix exists)
Own content ratio: 0.68

Discussion

High-signal excerpts from the issue thread (symptoms, repros, edge-cases).

Jump to Sources Open on GitHub

“> Possible solution: > > * handle the check of the table existence with another boto3 call such as describe_table and then fall-back into the…”

@auvipy · 2021-02-01 · source

Failure Signature (Search String)

dynamoDB result backend incorrect exception handling when table exists
* handle that exception differently

Copy-friendly signature

signature.txt

Failure Signature
-----------------
dynamoDB result backend incorrect exception handling when table exists
* handle that exception differently

Error Message

Signature-only (no traceback captured)

error.txt

Error Message
-------------
dynamoDB result backend incorrect exception handling when table exists
* handle that exception differently

Minimal Reproduction

repro.py

"dynamodb:DescribeTable",
   "dynamodb:PutItem",
   "dynamodb:UpdateItem",
   "dynamodb:DeleteItem",
   "dynamodb:BatchWriteItem",
   "dynamodb:GetItem",
   "dynamodb:BatchGetItem",
   "dynamodb:Scan",
   "dynamodb:Query",
   "dynamodb:ConditionCheckItem"

What Broke

Users experience failures when the table exists but the role lacks CreateTable permission, causing exceptions.

Why It Broke

The code attempts to create a DynamoDB table without the necessary permissions, leading to unexpected exceptions

Fix Options (Details)

Option A — Apply the official fix

Fixes incorrect exception handling when checking for the existence of a DynamoDB table by using `describe_table` instead of `create_table`.

When NOT to use: Do not use this fix if the role has CreateTable permission, as it may lead to unnecessary complexity.

Fix reference: https://github.com/celery/celery/pull/7734

Last verified: 2026-02-11. Validate in your environment.

When NOT to Use This Fix

Do not use this fix if the role has CreateTable permission, as it may lead to unnecessary complexity.

Verify Fix

verify

Re-run the minimal reproduction on your broken version, then apply the fix and re-run.

Did This Fix Work in Your Case?

Quick signal helps us prioritize which fixes to verify and improve.

Prevention

Capture the exact failing error string in logs and tests so you can reproduce via a minimal script.
Pin production dependencies and upgrade only with a reproducible test that hits the failing path.

Related Issues

No related fixes found.

Cluster: celery:configuration-error Celery hub Celery best practices All hubs All clusters

Related clusters: Data consistency Duplicates Duplicate processing

Sources

We don’t republish the full GitHub discussion text. Use the links above for context.

Celery dynamoDB result backend incorrect exception handling when table exists (Fix)