Celery ETA field on task is not correctly parsed in

The Fix

pip install celery==5.3.5

Based on closed celery/celery issue #8499 · PR/commit linked

Production note: This usually shows up under retries/timeouts. Treat it as a side-effect risk until you can verify behavior with a canary + real traffic.

Jump to Verify Open PR/Commit

@@ -36,7 +36,7 @@ jobs:
             - python-version: 'pypy-3.10'
               os: "windows-latest"
-            
+
     steps:

repro.py

compose-worker_fast-5  | [2023-09-12 15:39:16,095: ERROR/MainProcess] Received invalid task message: invalid ETA value '
2023-09-12T15:39:16Z': Invalid isoformat string: '2023-09-12T15:39:16Z'
compose-worker_fast-5  | The message has been ignored and discarded.
compose-worker_fast-5  |
compose-worker_fast-5  | Please ensure your message conforms to the task
compose-worker_fast-5  | message protocol as described here:
compose-worker_fast-5  | https://docs.celeryq.dev/en/latest/internals/protocol.html
compose-worker_fast-5  |
compose-worker_fast-5  | The full contents of the message body was:
compose-worker_fast-5  | {'id': '', 'task': 'tasks.notification.alert.send_notification_alert_mail', 'args': [3022248],
'kwargs': {}, 'retries': 0, 'eta': '2023-09-12T15:39:16Z'} (142b)
compose-worker_fast-5  | Traceback (most recent call last):
compose-worker_fast-5  |   File "/home/celery/venv/lib/python3.10/site-packages/celery/worker/request.py", line 135, in
__init__
compose-worker_fast-5  |     eta = maybe_iso8601(eta)
compose-worker_fast-5  |   File "/home/celery/venv/lib/python3.10/site-packages/celery/utils/time.py", line 291, in mayb
e_iso8601
compose-worker_fast-5  |     return datetime.fromisoformat(dt)
compose-worker_fast-5  | ValueError: Invalid isoformat string: '2023-09-12T15:39:16Z'
compose-worker_fast-5  |
compose-worker_fast-5  | During handling of the above exception, another exception occurred:
compose-worker_fast-5  |
compose-worker_fast-5  | Traceback (most recent call last):
compose-worker_fast-5  |   File "/home/celery/venv/lib/python3.10/site-packages/celery/worker/consumer/consumer.py", lin
e 668, in on_task_received
compose-worker_fast-5  |     strategy(
compose-worker_fast-5  |   File "/home/celery/venv/lib/python3.10/site-packages/celery/worker/strategy.py", line 146, in
 task_message_handler
compose-worker_fast-5  |     req = Req(
compose-worker_fast-5  |   File "/home/celery/venv/lib/python3.10/site-packages/celery/worker/request.py", line 137, in
__init__
compose-worker_fast-5  |     raise InvalidTaskError(
compose-worker_fast-5  | celery.exceptions.InvalidTaskError: invalid ETA value '2023-09-12T15:39:16Z': Invalid isoformat
 string: '2023-09-12T15:39:16Z'

verify

Re-run the minimal reproduction on your broken version, then apply the fix and re-run.

fix.md

Option A — Upgrade to fixed release\npip install celery==5.3.5\nWhen NOT to use: Do not use if it changes public behavior or if the failure cannot be reproduced.\n\n

Why This Fix Works in Production

Trigger: - [x] I have included all related issues and possible duplicate issues
Mechanism: Replaces datetime.fromisoformat with the more lenient dateutil parser to correctly handle the 'Z' timezone in ISO8601 dates, fixing the parsing issue in Celery for Python versions below 3.11.
Why the fix works: Replaces datetime.fromisoformat with the more lenient dateutil parser to correctly handle the 'Z' timezone in ISO8601 dates, fixing the parsing issue in Celery for Python versions below 3.11. (first fixed release: 5.3.5).

Production impact:

If left unfixed, the same config can fail only in production (env differences), causing startup failures or partial feature outages.

Why This Breaks in Prod

Shows up under Python 3.10 in real deployments (not just unit tests).
Production symptom (often without a traceback): - [x] I have included all related issues and possible duplicate issues

Proof / Evidence

GitHub issue: #8499
Fix PR: https://github.com/celery/celery/pull/8507
First fixed release: 5.3.5
Reproduced locally: No (not executed)
Last verified: 2026-02-09
Confidence: 0.75
Did this fix it?: Yes (upstream fix exists)
Own content ratio: 0.43

Discussion

High-signal excerpts from the issue thread (symptoms, repros, edge-cases).

Jump to Sources Open on GitHub

“It's a valid iso8601 date, which according to the celery docs is allowed”

@jaapz · 2023-09-13 · confirmation · source

“Temporary fix is rolling back to the celery version before 5.3.0”

@jaapz · 2023-09-13 · source

“It's the Z timezone portion, that's what older Python doesn't handle. Replacing Z with +00:00 produces an equivalent aware datetime. Probably better to handle that…”

@stumpylog · 2023-09-13 · source

“> Probably better to handle that internally As I said...”

@stumpylog · 2023-09-13 · source

Failure Signature (Search String)

- [x] I have included all related issues and possible duplicate issues
or possible duplicates to this issue as requested by the checklist above.

Copy-friendly signature

signature.txt

Failure Signature
-----------------
- [x] I have included all related issues and possible duplicate issues
or possible duplicates to this issue as requested by the checklist above.

Error Message

Signature-only (no traceback captured)

error.txt

Error Message
-------------
- [x] I have included all related issues and possible duplicate issues
or possible duplicates to this issue as requested by the checklist above.

Minimal Reproduction

repro.py

compose-worker_fast-5  | [2023-09-12 15:39:16,095: ERROR/MainProcess] Received invalid task message: invalid ETA value '
2023-09-12T15:39:16Z': Invalid isoformat string: '2023-09-12T15:39:16Z'
compose-worker_fast-5  | The message has been ignored and discarded.
compose-worker_fast-5  |
compose-worker_fast-5  | Please ensure your message conforms to the task
compose-worker_fast-5  | message protocol as described here:
compose-worker_fast-5  | https://docs.celeryq.dev/en/latest/internals/protocol.html
compose-worker_fast-5  |
compose-worker_fast-5  | The full contents of the message body was:
compose-worker_fast-5  | {'id': '', 'task': 'tasks.notification.alert.send_notification_alert_mail', 'args': [3022248],
'kwargs': {}, 'retries': 0, 'eta': '2023-09-12T15:39:16Z'} (142b)
compose-worker_fast-5  | Traceback (most recent call last):
compose-worker_fast-5  |   File "/home/celery/venv/lib/python3.10/site-packages/celery/worker/request.py", line 135, in
__init__
compose-worker_fast-5  |     eta = maybe_iso8601(eta)
compose-worker_fast-5  |   File "/home/celery/venv/lib/python3.10/site-packages/celery/utils/time.py", line 291, in mayb
e_iso8601
compose-worker_fast-5  |     return datetime.fromisoformat(dt)
compose-worker_fast-5  | ValueError: Invalid isoformat string: '2023-09-12T15:39:16Z'
compose-worker_fast-5  |
compose-worker_fast-5  | During handling of the above exception, another exception occurred:
compose-worker_fast-5  |
compose-worker_fast-5  | Traceback (most recent call last):
compose-worker_fast-5  |   File "/home/celery/venv/lib/python3.10/site-packages/celery/worker/consumer/consumer.py", lin
e 668, in on_task_received
compose-worker_fast-5  |     strategy(
compose-worker_fast-5  |   File "/home/celery/venv/lib/python3.10/site-packages/celery/worker/strategy.py", line 146, in
 task_message_handler
compose-worker_fast-5  |     req = Req(
compose-worker_fast-5  |   File "/home/celery/venv/lib/python3.10/site-packages/celery/worker/request.py", line 137, in
__init__
compose-worker_fast-5  |     raise InvalidTaskError(
compose-worker_fast-5  | celery.exceptions.InvalidTaskError: invalid ETA value '2023-09-12T15:39:16Z': Invalid isoformat
 string: '2023-09-12T15:39:16Z'

Environment

Python: 3.10

What Broke

Tasks with valid ETA fields fail to parse, causing scheduling issues.

Fix Options (Details)

Option A — Upgrade to fixed release Safe default (recommended)

pip install celery==5.3.5

When NOT to use: Do not use if it changes public behavior or if the failure cannot be reproduced.

Use when you can deploy the upstream fix. It is usually lower-risk than long-lived workarounds.

Option D — Guard side-effects with OnceOnly Guardrail for side-effects

Mitigate duplicate external side-effects under retries/timeouts/agent loops by gating the operation before calling external systems.

Place OnceOnly between your code/agent and real side-effects (Stripe, emails, CRM, APIs).
Use a stable key per side-effect (e.g., customer_id + action + idempotency_key).
Fail-safe: configure fail-open vs fail-closed based on blast radius and spend risk.

Show example snippet (optional)

onceonly.py

from onceonly import OnceOnly
import os

once = OnceOnly(api_key=os.environ["ONCEONLY_API_KEY"], fail_open=True)

# Stable idempotency key per real side-effect.
# Use a request id / job id / webhook delivery id / Stripe event id, etc.
event_id = "evt_..."  # replace
key = f"stripe:webhook:{event_id}"

res = once.check_lock(key=key, ttl=3600)
if res.duplicate:
    return {"status": "already_processed"}

# Safe to execute the side-effect exactly once.
handle_event(event_id)

See OnceOnly SDK

When NOT to use: Do not use this to hide logic bugs or data corruption. Use it to block duplicate external side-effects and enforce tool permissions/spend caps.

Fix reference: https://github.com/celery/celery/pull/8507

First fixed release: 5.3.5

Last verified: 2026-02-09. Validate in your environment.

When NOT to Use This Fix

Do not use if it changes public behavior or if the failure cannot be reproduced.
Do not use this to hide logic bugs or data corruption. Use it to block duplicate external side-effects and enforce tool permissions/spend caps.

Verify Fix

verify

Re-run the minimal reproduction on your broken version, then apply the fix and re-run.

Did This Fix Work in Your Case?

Quick signal helps us prioritize which fixes to verify and improve.

Prevention

Capture the exact failing error string in logs and tests so you can reproduce via a minimal script.
Pin production dependencies and upgrade only with a reproducible test that hits the failing path.

Version Compatibility Table

Version	Status
5.3.5	Fixed

Related Issues

No related fixes found.

Cluster: celery:configuration-error Celery hub Celery best practices All hubs All clusters

Related clusters: Data consistency Duplicates Duplicate processing

Sources

We don’t republish the full GitHub discussion text. Use the links above for context.

Celery ETA field on task is not correctly parsed in celery>=5.3.0 and python<3.11 (Fix)