Celery Corrupted scheduler database (Fix)

The Fix

pip install celery==5.5.0

Based on closed celery/celery issue #4777 · PR/commit linked

Jump to Verify Open PR/Commit

@@ -1,6 +1,7 @@
 
 import copy
+import dbm
 import errno
 import heapq

repro.py

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/celery/apps/beat.py", line 113, in start_scheduler
    service.start()
  File "/usr/local/lib/python3.10/dist-packages/celery/beat.py", line 634, in start
    humanize_seconds(self.scheduler.max_interval))
  File "/usr/local/lib/python3.10/dist-packages/kombu/utils/objects.py", line 40, in __get__
    return super().__get__(instance, owner)
  File "/usr/lib/python3.10/functools.py", line 981, in __get__
    val = self.func(instance)
  File "/usr/local/lib/python3.10/dist-packages/celery/beat.py", line 677, in scheduler
    return self.get_scheduler()
  File "/usr/local/lib/python3.10/dist-packages/celery/beat.py", line 668, in get_scheduler
    return symbol_by_name(self.scheduler_cls, aliases=aliases)(
  File "/usr/local/lib/python3.10/dist-packages/celery/beat.py", line 513, in __init__
    super().__init__(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/celery/beat.py", line 264, in __init__
    self.setup_schedule()
  File "/usr/local/lib/python3.10/dist-packages/celery/beat.py", line 541, in setup_schedule
    self._create_schedule()
  File "/usr/local/lib/python3.10/dist-packages/celery/beat.py", line 574, in _create_schedule
    self._store['entries'] = {}
  File "/usr/lib/python3.10/shelve.py", line 125, in __setitem__
    self.dict[key.encode(self.keyencoding)] = f.getvalue()
_dbm.error: cannot add item to database

verify

Re-run the minimal reproduction on your broken version, then apply the fix and re-run.

fix.md

Option A — Upgrade to fixed release\npip install celery==5.5.0\nWhen NOT to use: This fix should not be applied if using a non-dbm based storage for the scheduler.\n\n

Why This Fix Works in Production

Trigger: Corrupted scheduler database
Mechanism: Updates the code to handle dbm.error as a corrupted schedule file, preventing crashes when attempting to clear 'entries'.
Why the fix works: Updates the code to handle dbm.error as a corrupted schedule file, preventing crashes when attempting to clear 'entries'. (first fixed release: 5.5.0).

Production impact:

If left unfixed, this can cause silent data inconsistencies that propagate (bad cache entries, incorrect downstream decisions).

Why This Breaks in Prod

Shows up under Python 3.6 in real deployments (not just unit tests).
Surfaces as: Traceback (most recent call last):\n File "/usr/lib/sillonlorrain-media/lib/python3.6/site-packages/kombu/utils/objects.py", line 42, in __get__\n return…

Proof / Evidence

GitHub issue: #4777
Fix PR: https://github.com/celery/celery/pull/9331
First fixed release: 5.5.0
Reproduced locally: No (not executed)
Last verified: 2026-02-09
Confidence: 0.85
Did this fix it?: Yes (upstream fix exists)
Own content ratio: 0.34

Discussion

High-signal excerpts from the issue thread (symptoms, repros, edge-cases).

Jump to Sources Open on GitHub

“This still seems like an issue with 5.4.0, based on https://github.com/paperless-ngx/paperless-ngx/discussions/7440, which reports: It seems like this line should include dbm.error: https://github.com/celery/celery/blob/40dafda3ff49ea082613d975a850a374a6ac161e”

@stumpylog · 2024-08-11 · source

“Experiencing the same problem: CPython 3.11.2 Celery 5.3.6 <details> <summary>Traceback (click to expand)</summary> </details>”

@KaczuH · 2024-09-23 · source

“@auvipy could we get this reopened? We are on celery 5.2.7 and experiencing the same issue”

@calebsyring · 2023-11-03 · source

Failure Signature (Search String)

Corrupted scheduler database

Error Message

Stack trace

error.txt

Error Message
-------------
Traceback (most recent call last):\n  File "/usr/lib/sillonlorrain-media/lib/python3.6/site-packages/kombu/utils/objects.py", line 42, in __get__\n    return obj.__dict__[self.__name__]

Stack trace

error.txt

Error Message
-------------
Traceback (most recent call last):\n  File "/usr/lib/python3.6/shelve.py", line 111, in __getitem__\n    value = self.cache[key]

Stack trace

error.txt

Error Message
-------------
Traceback (most recent call last):\n  File "/usr/lib/sillonlorrain-media/lib/python3.6/site-packages/celery/beat.py", line 481, in _create_schedule\n    self._store[str('entries')]\n  File "/usr/lib/python3.6/shelve.py", line 113, in __getitem__\n    f = BytesIO(self.dict[key.encode(self.keyencoding)])

Minimal Reproduction

repro.py

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/celery/apps/beat.py", line 113, in start_scheduler
    service.start()
  File "/usr/local/lib/python3.10/dist-packages/celery/beat.py", line 634, in start
    humanize_seconds(self.scheduler.max_interval))
  File "/usr/local/lib/python3.10/dist-packages/kombu/utils/objects.py", line 40, in __get__
    return super().__get__(instance, owner)
  File "/usr/lib/python3.10/functools.py", line 981, in __get__
    val = self.func(instance)
  File "/usr/local/lib/python3.10/dist-packages/celery/beat.py", line 677, in scheduler
    return self.get_scheduler()
  File "/usr/local/lib/python3.10/dist-packages/celery/beat.py", line 668, in get_scheduler
    return symbol_by_name(self.scheduler_cls, aliases=aliases)(
  File "/usr/local/lib/python3.10/dist-packages/celery/beat.py", line 513, in __init__
    super().__init__(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/celery/beat.py", line 264, in __init__
    self.setup_schedule()
  File "/usr/local/lib/python3.10/dist-packages/celery/beat.py", line 541, in setup_schedule
    self._create_schedule()
  File "/usr/local/lib/python3.10/dist-packages/celery/beat.py", line 574, in _create_schedule
    self._store['entries'] = {}
  File "/usr/lib/python3.10/shelve.py", line 125, in __setitem__
    self.dict[key.encode(self.keyencoding)] = f.getvalue()
_dbm.error: cannot add item to database

Environment

Python: 3.6

What Broke

The scheduler fails to start, causing task scheduling to halt and impacting production workflows.

Fix Options (Details)

Option A — Upgrade to fixed release Safe default (recommended)

pip install celery==5.5.0

When NOT to use: This fix should not be applied if using a non-dbm based storage for the scheduler.

Use when you can deploy the upstream fix. It is usually lower-risk than long-lived workarounds.

Fix reference: https://github.com/celery/celery/pull/9331

First fixed release: 5.5.0

Last verified: 2026-02-09. Validate in your environment.

When NOT to Use This Fix

This fix should not be applied if using a non-dbm based storage for the scheduler.

Verify Fix

verify

Re-run the minimal reproduction on your broken version, then apply the fix and re-run.

Did This Fix Work in Your Case?

Quick signal helps us prioritize which fixes to verify and improve.

Prevention

Add a TLS smoke test that performs a real handshake in CI (include CA bundle validation and hostname checks).
Alert on handshake failures by error string and endpoint to catch cert/CA changes quickly.

Version Compatibility Table

Version	Status
5.5.0	Fixed

Related Issues

No related fixes found.

Cluster: celery:data-consistency Celery hub Celery best practices All hubs All clusters

Related clusters: Configuration error Duplicates Duplicate processing

Sources

We don’t republish the full GitHub discussion text. Use the links above for context.

Celery Corrupted scheduler database (Fix)

The Fix

Why This Fix Works in Production

Why This Breaks in Prod

Proof / Evidence

Discussion

Failure Signature (Search String)

Error Message

Minimal Reproduction

Environment

What Broke

Fix Options (Details)

Option A — Upgrade to fixed release Safe default (recommended)

Get updates

When NOT to Use This Fix

Verify Fix

Did This Fix Work in Your Case?

Prevention

Version Compatibility Table

Related Issues

Sources