[Bug]: Centralized Event Service for Multi-Worker Environments for all services #1506
Description
🐞 Bug Summary
The current event system uses in-process queues for publish_events and subscribe_events for all the services. This approach works only in a single-worker environment. In multi-worker deployments, in-memory queues are not shared, causing events published in one worker to be missed by subscribers in other workers.
A centralized event mechanism is required. It should support:
- Redis Pub/Sub as the primary transport for multi-worker environments
- In-process queue fallback when Redis is unavailable
- A unified interface usable across all services
🧩 Affected Component
Select the area of the project impacted:
-
mcpgateway- API -
mcpgateway- UI (admin panel) -
mcpgateway.wrapper- stdio wrapper - Federation or Transports
- CLI, Makefiles, or shell scripts
- Container setup (Docker/Podman/Compose)
- Other (explain below)
🔁 Steps to Reproduce
- Start the application with multiple workers (e.g., Gunicorn with more than one worker).
- Publish an event using publish_events.
- Observe that subscribers in other workers do not receive the event due to isolated in-memory queues.
🤔 Expected Behavior
Events published by any worker should be delivered to all subscribed workers or services. The event system should automatically use Redis when available and fall back to an in-process queue when it is not.
🧠 Environment Info
You can retrieve most of this from the /version endpoint.
| Key | Value |
|---|---|
| Version or commit | e.g. v0.9.0 or main@a1b2c3d |
| Runtime | e.g. Python 3.11, Gunicorn |
| Platform / OS | e.g. Ubuntu 22.04, macOS |
| Container | e.g. Docker, Podman, none |
🧩 Additional Context (optional)
Add any configuration details, flags, or related issues.