The Fix
pip install celery==5.6.0
Based on closed celery/celery issue #9951 · PR/commit linked
Production note: This usually shows up under retries/timeouts. Treat it as a side-effect risk until you can verify behavior with a canary + real traffic.
@@ -12,6 +12,7 @@
from collections import namedtuple
from functools import total_ordering
+from pickle import UnpicklingError
from threading import Event, Thread
repro.py
Traceback (most recent call last):
File "shelve.py", line 111, in __getitem__
value = self.cache[key]
KeyError: 'entries'
Traceback (most recent call last):
File "celery/apps/beat.py", line 105, in start_scheduler
service.start()
File "celery/beat.py", line 636, in start
humanize_seconds(self.scheduler.max_interval))
File "kombu/utils/objects.py", line 30, in __get__
return super().__get__(instance, owner)
File "functools.py", line 993, in __get__
val = self.func(instance)
File "celery/beat.py", line 680, in scheduler
return self.get_scheduler()
File "celery/beat.py", line 671, in get_scheduler
return symbol_by_name(self.scheduler_cls, aliases=aliases)(
File "celery/beat.py", line 515, in __init__
super().__init__(*args, **kwargs)
File "celery/beat.py", line 271, in __init__
self.setup_schedule()
File "celery/beat.py", line 543, in setup_schedule
self._create_schedule()
File "celery/beat.py", line 572, in _create_schedule
self._store['entries']
File "shelve.py", line 114, in __getitem__
value = Unpickler(f).load()
UnpicklingError: invalid load key, '\xf2'.
verify
Re-run the minimal reproduction on your broken version, then apply the fix and re-run.
fix.md
Option A — Upgrade to fixed release\npip install celery==5.6.0\nWhen NOT to use: Do not use this fix if the scheduler's persistence mechanism is not being utilized.\n\n
Why This Fix Works in Production
- Trigger: value = self.cache[key]
- Mechanism: Celery beat crashes due to UnpicklingError when accessing the persistent scheduler
- Why the fix works: Handles UnpicklingError in the persistent scheduler initialization, preventing crashes in celery beat. (first fixed release: 5.6.0).
Production impact:
- If left unfixed, the same config can fail only in production (env differences), causing startup failures or partial feature outages.
Why This Breaks in Prod
- Celery beat crashes due to UnpicklingError when accessing the persistent scheduler
- Surfaces as: Traceback (most recent call last):
Proof / Evidence
- GitHub issue: #9951
- Fix PR: https://github.com/celery/celery/pull/9952
- First fixed release: 5.6.0
- Reproduced locally: No (not executed)
- Last verified: 2026-02-08
- Confidence: 0.85
- Did this fix it?: Yes (upstream fix exists)
- Own content ratio: 0.32
Discussion
High-signal excerpts from the issue thread (symptoms, repros, edge-cases).
“Sometimes celery beat crashes with the following error: UnpicklingError - invalid load key. I will work on a fix soon. Here is the relevant stack trace: # Checklist <!-- To check an item on the list replace [ ] with [x]. --> - [X] I have ve”
Failure Signature (Search String)
- value = self.cache[key]
Error Message
Stack trace
error.txt
Error Message
-------------
Traceback (most recent call last):
File "shelve.py", line 111, in __getitem__
value = self.cache[key]
KeyError: 'entries'
Traceback (most recent call last):
File "celery/apps/beat.py", line 105, in start_scheduler
service.start()
File "celery/beat.py", line 636, in start
humanize_seconds(self.scheduler.max_interval))
File "kombu/utils/objects.py", line 30, in __get__
return super().__get__(instance, owner)
File "functools.py", line 993, in __get__
val = self.func(instance)
File "celery/beat.py", line 680, in scheduler
return self.get_scheduler()
File "celery/beat.py", line 671, in get_scheduler
return symbol_by_name(self.scheduler_cls, aliases=aliases)(
File "celery/beat.py", line 515, in __init__
super().__init__(*args, **kwargs)
File "celery/beat.py", line 271, in __init__
self.setup_schedule()
File "celery/beat.py", line 543, in setup_schedule
self._create_schedule()
File "celery/beat.py", line 572, in _create_schedule
self._store['entries']
File "shelve.py", line 114, in __getitem__
value = Unpickler(f).load()
UnpicklingError: invalid load key, '\xf2'.
Minimal Reproduction
repro.py
Traceback (most recent call last):
File "shelve.py", line 111, in __getitem__
value = self.cache[key]
KeyError: 'entries'
Traceback (most recent call last):
File "celery/apps/beat.py", line 105, in start_scheduler
service.start()
File "celery/beat.py", line 636, in start
humanize_seconds(self.scheduler.max_interval))
File "kombu/utils/objects.py", line 30, in __get__
return super().__get__(instance, owner)
File "functools.py", line 993, in __get__
val = self.func(instance)
File "celery/beat.py", line 680, in scheduler
return self.get_scheduler()
File "celery/beat.py", line 671, in get_scheduler
return symbol_by_name(self.scheduler_cls, aliases=aliases)(
File "celery/beat.py", line 515, in __init__
super().__init__(*args, **kwargs)
File "celery/beat.py", line 271, in __init__
self.setup_schedule()
File "celery/beat.py", line 543, in setup_schedule
self._create_schedule()
File "celery/beat.py", line 572, in _create_schedule
self._store['entries']
File "shelve.py", line 114, in __getitem__
value = Unpickler(f).load()
UnpicklingError: invalid load key, '\xf2'.
What Broke
Celery beat becomes unresponsive and crashes, leading to missed scheduled tasks.
Why It Broke
Celery beat crashes due to UnpicklingError when accessing the persistent scheduler
Fix Options (Details)
Option A — Upgrade to fixed release Safe default (recommended)
pip install celery==5.6.0
When NOT to use: Do not use this fix if the scheduler's persistence mechanism is not being utilized.
Use when you can deploy the upstream fix. It is usually lower-risk than long-lived workarounds.
Option D — Guard side-effects with OnceOnly Guardrail for side-effects
Mitigate duplicate external side-effects under retries/timeouts/agent loops by gating the operation before calling external systems.
- Place OnceOnly between your code/agent and real side-effects (Stripe, emails, CRM, APIs).
- Use a stable key per side-effect (e.g., customer_id + action + idempotency_key).
- Fail-safe: configure fail-open vs fail-closed based on blast radius and spend risk.
Show example snippet (optional)
onceonly.py
from onceonly import OnceOnly
import os
once = OnceOnly(api_key=os.environ["ONCEONLY_API_KEY"], fail_open=True)
# Stable idempotency key per real side-effect.
# Use a request id / job id / webhook delivery id / Stripe event id, etc.
event_id = "evt_..." # replace
key = f"stripe:webhook:{event_id}"
res = once.check_lock(key=key, ttl=3600)
if res.duplicate:
return {"status": "already_processed"}
# Safe to execute the side-effect exactly once.
handle_event(event_id)
When NOT to use: Do not use this to hide logic bugs or data corruption. Use it to block duplicate external side-effects and enforce tool permissions/spend caps.
Fix reference: https://github.com/celery/celery/pull/9952
First fixed release: 5.6.0
Last verified: 2026-02-08. Validate in your environment.
When NOT to Use This Fix
- Do not use this fix if the scheduler's persistence mechanism is not being utilized.
- Do not use this to hide logic bugs or data corruption. Use it to block duplicate external side-effects and enforce tool permissions/spend caps.
Verify Fix
verify
Re-run the minimal reproduction on your broken version, then apply the fix and re-run.
Did This Fix Work in Your Case?
Quick signal helps us prioritize which fixes to verify and improve.
Prevention
- Capture the exact failing error string in logs and tests so you can reproduce via a minimal script.
- Pin production dependencies and upgrade only with a reproducible test that hits the failing path.
Version Compatibility Table
| Version | Status |
|---|---|
| 5.6.0 | Fixed |
Related Issues
No related fixes found.
Sources
We don’t republish the full GitHub discussion text. Use the links above for context.