Handling high traffic with the current 5-second polling mechanism?

[Rayafarezi]

Hi Bita,
Great job on this delivery tracking system! I really liked how you integrated the real-time polling on the frontend.

My question is regarding scalability: As the number of active drivers and customers grows, frequent 5-second HTTP polling requests might heavily strain the Django server and database. Have you considered moving towards WebSockets (via Django Channels) or Server-Sent Events (SSE) for a more event-driven real-time update in the future? Or do you have specific optimization/caching strategies in mind for the current polling mechanism?

Thanks!

[Bitafarezi]

Hi! Thanks for the great question. You've pointed out a very critical architectural challenge.

You are completely right. As the system scales, having hundreds or thousands of active users hitting the Django server every 5 seconds via standard HTTP polling will inevitably lead to database bottlenecks and high CPU usage, even with optimized querysets.

Here is my approach and how I see this evolving:

1. Current State (Why Polling?)

For the initial version of this project, HTTP Live-Polling was chosen for a few reasons:

• Simplicity & Speed of Development: It allowed quick implementation of a real-time feel on the frontend without adding infrastructure complexity (like ASGI, Redis, or protocol routers).
• Low Initial Load: For a smaller or medium-sized courier network, this works fine as a starting point.

2. Next Steps for Optimization (Before WebSockets)

Before completely rewriting the communication layer, we can optimize the current polling mechanism by:

• Database Caching: Implementing Django’s cache framework (e.g., using Redis) on the get_queryset level so the server doesn't hit the SQLite/PostgreSQL database every 5 seconds for unchanged order statuses.
• Conditional Requests: Utilizing HTTP headers like If-Modified-Since or ETag so the server returns a lightweight 304 Not Modified if no status updates occurred.

3. The Future: Moving to WebSockets (Django Channels)

To make this system truly production-grade and highly scalable, the ultimate roadmap definitely includes migrating to WebSockets via Django Channels:

• Event-Driven Architecture: Instead of clients asking "Any updates?" every 5 seconds, the server will instantly push data only when an order status actually changes (e.g., when a driver hits /accept/ or /deliver/).
• Pub/Sub with Redis: We can use Redis as a channel layer to group users (e.g., a specific customer and their assigned driver) into a dedicated room, minimizing unnecessary data broadcast.

I'd love to hear your thoughts on this! If you've implemented Django Channels for similar location/order tracking systems before, feel free to share any tips or pitfalls to avoid.