Silent total event loss under gunicorn gevent workers: flush crashes on gevent Queue (all_tasks_done), and 7.38.4 still emits only a boot-time burst with --preload

[ThaddKara]

Bug description

Recent posthog-python releases silently stop delivering all events when running inside gunicorn workers with --worker-class gevent (and --preload). capture() succeeds from the application's point of view, but the consumer/flush path is broken, so events buffer in memory and are never sent. For us this produced a multi-day, alert-free blackout of all server-side events from our web tier, while celery workers (no gevent) on the same installed version kept emitting normally.

Environment

• Flask app on gunicorn: gunicorn --workers 12 --worker-class gevent --preload wsgi:app (gevent monkey-patching active in workers)
• posthog installed unpinned, so deploys pick up the latest release
• Python 3.x on Amazon Linux (Elastic Beanstalk)

Phase 1 — loud failure (release current as of 2026-08-04)

A deploy on 2026-08-04 picked up the then-latest release. From that moment every flush attempt in the gevent workers crashed:

[ERROR] posthog: error flushing queue: 'gevent._gevent_cqueue.Queue' object has no attribute 'all_tasks_done'

gevent's monkey-patched Queue replacement does not implement all_tasks_done / join() semantics the SDK's flush path relies on. The consumer effectively dies; events are enqueued forever and 100% dropped in every gevent worker.

Phase 2 — silent failure (7.38.4)

After upgrading to 7.38.4 (2026-08-10) the crash log disappeared, but emission from gunicorn workers still did not recover. The observable pattern: each worker emits a brief burst of events right at process boot, then goes permanently silent. Because gunicorn recycles workers (--max-requests), this shows up as a tiny trickle (~0.1% of normal volume) rather than zero — which is arguably worse, since it defeats "did events stop entirely?" alerting.

This smells like an interaction with --preload similar to #290: if consumer threads are now started eagerly (pre-fork, in the master), forked workers inherit a dead consumer thread and their queues are never drained. Under the older release (lazy consumer start on first capture in the worker) the same gunicorn config worked fine for months.

Repro sketch

1. gunicorn --worker-class gevent --preload app:app
2. posthog.capture(...) inside request handlers
3. Observe: no events delivered (phase 1 versions crash in flush; 7.38.4 delivers only a boot-time burst per worker)

Impact

• 6+ days of missing server-side analytics across two incident investigations; nothing user-visible failed, so nothing alerted
• The per-provider/per-endpoint gaps looked like real traffic drops, which sent us chasing phantom outages in downstream vendors

Suggested fixes

• Feature-detect gevent monkey-patching / missing all_tasks_done instead of assuming stdlib queue.Queue behavior in the consumer/flush path
• Start (or restart) consumer threads post-fork, or document a post_fork posthog.setup() requirement loudly for --preload users (extends Using posthog-python with Gunicorn and preload_app = True #290)
• Emit a loud warning when events are being enqueued while no live consumer thread exists — silent buffering-forever is the worst failure mode for an analytics SDK
• CI regression test: capture + flush under (a) gevent monkey-patching, (b) gunicorn preload-fork

Happy to provide log excerpts or exact version pins from our deploy history if useful.