Polling publishers

[Tijani-Dia]

This PR adds a PollingPublisherMixin for publishers using the polling technique.

It also adds an IntervalPollingPublisher and a LongPollingPublisher.

I haven't added tests yet, I want to have a feedback on the design first.
To try it out with the testapp, checkout this branch (polling-publisher).

Here are the steps to take if you want to test the interval polling publisher:

1. Add this to your settings.py file:
   WAGTAIL_LIVE_PUBLISHER = "wagtail_live.publishers.IntervalPollingPublisher"

2. Add wagtail_live urls in tests.urls like this:

from wagtail_live import urls as live_urls

urlpatterns += [
    path("wagtail_live/", include(live_urls)),
]

3. Create a template for BlogPage model and add the following:

{% include "wagtail_live/live_posts.html" %}

{% include "wagtail_live/polling/interval_polling.html" %}

4. Create a live page in the admin interface with channel_id=test_channel for example

5. Head to http://127.0.0.1:8000/wagtail_live_interface/channels and create a channel with the same name as the BlogPage created, in this example test_channel

6. (Optional) You can also add a WAGTAIL_LIVE_POLLING_INTERVAL value in milliseconds in your settings.

We need to set DEBUG to True in order to get static files served.
And it should be fine.

The steps to take for the long polling publisher are similar with these differences:

1. WAGTAIL_LIVE_PUBLISHER = "wagtail_live.publishers.LongPollingPublisher"

2. Create a template for BlogPage model and add the following:

{% include "wagtail_live/live_posts.html" %}

{% include "wagtail_live/polling/long_polling.html" %}

6. (Optional) You can add a WAGTAIL_LIVE_POLLING_TIMEOUT value in seconds in your settings.

Additional remarks

From django docs, long polling is a good candidate for async views:

Under a WSGI server, async views will run in their own, one-off event loop. This means you can use async features, like concurrent async HTTP requests, without any issues, but you will not get the benefits of an async stack.

The main benefits are the ability to service hundreds of connections without using Python threads. This allows you to use slow streaming, long-polling, and other exciting response types.

I tried to make an AsyncLongPollingPublisher but I don't think it changes a lot under WSGI but maybe we can add it for apps running on ASGI?

[lucasmoeskops]

Nice work!

[lucasmoeskops]

At some point this might be optimized since when many people watch the live-stream they will essentially all be doing many identical queries to the database all the time. But database caching like django-cachalot will already help out in cases like this I think (only when you have considerably many users).

[Tijani-Dia]

Alright, do we support it out of the box and define a boolean setting like WAGTAIL_LIVE_USE_CACHALOT or shall we just mention in the docs that this optimisation is possible?

[lucasmoeskops]

I think we should just mention this possibility. If people use e.g. cachalot it will be automatically fixed, so no setting is required.

[Tijani-Dia]

Good👍

[Stormheg]

Suggestion: what if the request doesn't have a last_update_ts parameter? Should we return something else instead? Maybe None but code that calls this method would have to handle that as well.

[Tijani-Dia]

I think that this can happen only if page.latest_revision_created_at is None. And if that happens, we will be in trouble sooner since we call the timestamp method on None object here. So, we need to handle it from there I think. How do you suggest to handle it?

[Stormheg]

Hmm, ok. Maybe we shouldn't worry too much and just return a 400 bad request response to the client if it doesn't include a last_update_ts parameter.