Celery [CassandraBackend.process_cleanup] is unnecessary

The Fix

pip install celery==4.4.6

Based on closed celery/celery issue #6143 · PR/commit linked

Production note: This usually shows up under retries/timeouts. Treat it as a side-effect risk until you can verify behavior with a canary + real traffic.

Jump to Verify Open PR/Commit

@@ -3,6 +3,7 @@
 
 import sys
+import threading
 
 from celery import states

repro.py

software -> celery:4.4.3 (cliffs) kombu:4.6.9 py:3.7.6
            billiard:3.6.3.0 py-amqp:2.6.0
platform -> system:Linux arch:64bit, ELF
            kernel version:4.15.0-101-generic imp:CPython
loader   -> celery.loaders.app.AppLoader
settings -> transport:amqp results:cassandra

Queue: <class 'kombu.entity.Queue'>
RoundRobinPolicy: <class 'cassandra.policies.RoundRobinPolicy'>
broker_url: 'amqp://guest:********@localhost:5672//'
cassandra_auth_kwargs: {
 'password': '********', 'username': 'cassandra'}
cassandra_auth_provider: 'PlainTextAuthProvider'
cassandra_entry_ttl: 86400
cassandra_keyspace: '********'
cassandra_options: {
    'load_balancing_policy': <cassandra.policies.RoundRobinPolicy object at 0x7f09de9a34d0>}
cassandra_servers: ['127.0.0.1']
cassandra_table: 'celery_jobs_result'
ignore_result: False
result_backend: 'cassandra'
worker_concurrency: 4

verify

Re-run the minimal reproduction on your broken version, then apply the fix and re-run.

fix.md

Option A — Upgrade to fixed release\npip install celery==4.4.6\nWhen NOT to use: Do not use this fix if persistent session behavior is required.\n\n

Why This Fix Works in Production

Trigger: [2020-06-02 21:52:19,592: ERROR/MainProcess] Task add[4bd528a0-c1ba-456c-b74e-7dee054616bb] raised unexpected: AttributeError("'NoneType' object has no…
Mechanism: process_cleanup closes the Cassandra session, causing errors in thread pools and gevent
Why the fix works: Removes the unnecessary process_cleanup method from CassandraBackend, which was causing issues in thread pools and gevent. (first fixed release: 4.4.6).

Production impact:

If left unfixed, the same config can fail only in production (env differences), causing startup failures or partial feature outages.

Why This Breaks in Prod

process_cleanup closes the Cassandra session, causing errors in thread pools and gevent
Surfaces as: [2020-06-02 21:52:19,592: ERROR/MainProcess] Task add[4bd528a0-c1ba-456c-b74e-7dee054616bb] raised unexpected: AttributeError("'NoneType' object has no attribute 'execute'")

Proof / Evidence

GitHub issue: #6143
Fix PR: https://github.com/celery/celery/pull/6147
First fixed release: 4.4.6
Reproduced locally: No (not executed)
Last verified: 2026-02-09
Confidence: 0.85
Did this fix it?: Yes (upstream fix exists)
Own content ratio: 0.28

Discussion

High-signal excerpts from the issue thread (symptoms, repros, edge-cases).

Jump to Sources Open on GitHub

“Hi @baixuexue123, thanks for this contribution. I don't understand why process_cleanup is unnecessary. And it seems nothing is closing the session. It may explain why…”

@mchataigner · 2020-06-04 · confirmation · source

“> Hi @baixuexue123, thanks for this contribution”

@baixuexue123 · 2020-06-04 · confirmation · source

“@auvipy I am improving it , and up a PR later”

@baixuexue123 · 2020-06-03 · source

Failure Signature (Search String)

[2020-06-02 21:52:19,592: ERROR/MainProcess] Task add[4bd528a0-c1ba-456c-b74e-7dee054616bb] raised unexpected: AttributeError("'NoneType' object has no attribute 'execute'")

Error Message

Stack trace

error.txt

Error Message
-------------
[2020-06-02 21:52:19,592: ERROR/MainProcess] Task add[4bd528a0-c1ba-456c-b74e-7dee054616bb] raised unexpected: AttributeError("'NoneType' object has no attribute 'execute'")
Traceback (most recent call last):
  File "/home/bx/project/celery/celery/app/trace.py", line 480, in trace_task
    uuid, retval, task_request, publish_result,
  File "/home/bx/project/celery/celery/backends/base.py", line 158, in mark_as_done
    self.store_result(task_id, result, state, request=request)
  File "/home/bx/project/celery/celery/backends/base.py", line 443, in store_result
    request=request, **kwargs)
  File "/home/bx/project/celery/celery/backends/cassandra.py", line 201, in _store_result
    self._session.execute(self._write_stmt, (
AttributeError: 'NoneType' object has no attribute 'execute'

Stack trace

error.txt

Error Message
-------------
[2020-06-02 21:52:29,773: ERROR/MainProcess] Task add[27c980d3-126a-430d-a8e0-92d749ceb735] raised unexpected: OperationTimedOut("errors=errors={'Connection defunct by heartbeat': 'Client request timeout. See Session.execute[_async](timeout)'}, last_host=127.0.0.1:9042, last_host=None")
Traceback (most recent call last):
  File "/home/bx/project/celery/celery/app/trace.py", line 480, in trace_task
    uuid, retval, task_request, publish_result,
  File "/home/bx/project/celery/celery/backends/base.py", line 158, in mark_as_done
    self.store_result(task_id, result, state, request=request)
  File "/home/bx/project/celery/celery/backends/base.py", line 443, in store_result
    request=request, **kwargs)
  File "/home/bx/project/celery/celery/backends/cassandra.py", line 207, in _store_result
    buf_t(self.encode(self.current_task_children(request)))
  File "cassandra/cluster.py", line 2345, in cassandra.cluster.Session.execute
  File "cassandra/cluster.py", line 4304, in cassandra.cluster.ResponseFuture.result
cassandra.OperationTimedOut: errors=errors={'Connection defunct by heartbeat': 'Client request timeout. See Session.execute[_async](timeout)'}, last_host=127.0.0.1:9042, last_host=None

Minimal Reproduction

repro.py

software -> celery:4.4.3 (cliffs) kombu:4.6.9 py:3.7.6
            billiard:3.6.3.0 py-amqp:2.6.0
platform -> system:Linux arch:64bit, ELF
            kernel version:4.15.0-101-generic imp:CPython
loader   -> celery.loaders.app.AppLoader
settings -> transport:amqp results:cassandra

Queue: <class 'kombu.entity.Queue'>
RoundRobinPolicy: <class 'cassandra.policies.RoundRobinPolicy'>
broker_url: 'amqp://guest:********@localhost:5672//'
cassandra_auth_kwargs: {
 'password': '********', 'username': 'cassandra'}
cassandra_auth_provider: 'PlainTextAuthProvider'
cassandra_entry_ttl: 86400
cassandra_keyspace: '********'
cassandra_options: {
    'load_balancing_policy': <cassandra.policies.RoundRobinPolicy object at 0x7f09de9a34d0>}
cassandra_servers: ['127.0.0.1']
cassandra_table: 'celery_jobs_result'
ignore_result: False
result_backend: 'cassandra'
worker_concurrency: 4

What Broke

Integration tests hang due to closed sessions in concurrent tasks.

Why It Broke

process_cleanup closes the Cassandra session, causing errors in thread pools and gevent

Fix Options (Details)

Option A — Upgrade to fixed release Safe default (recommended)

pip install celery==4.4.6

When NOT to use: Do not use this fix if persistent session behavior is required.

Use when you can deploy the upstream fix. It is usually lower-risk than long-lived workarounds.

Fix reference: https://github.com/celery/celery/pull/6147

First fixed release: 4.4.6

Last verified: 2026-02-09. Validate in your environment.

When NOT to Use This Fix

Do not use this fix if persistent session behavior is required.

Verify Fix

verify

Re-run the minimal reproduction on your broken version, then apply the fix and re-run.

Did This Fix Work in Your Case?

Quick signal helps us prioritize which fixes to verify and improve.

Prevention

Capture the exact failing error string in logs and tests so you can reproduce via a minimal script.
Pin production dependencies and upgrade only with a reproducible test that hits the failing path.

Version Compatibility Table

Version	Status
4.4.6	Fixed

Related Issues

No related fixes found.

Cluster: celery:configuration-error Celery hub Celery best practices All hubs All clusters

Related clusters: Data consistency Duplicates Duplicate processing

Sources

We don’t republish the full GitHub discussion text. Use the links above for context.

Celery [CassandraBackend.process_cleanup] is unnecessary (Fix)