Streamed event feed does not send the latest events

[ningit]

Description of the problem

Contest event feeds are not sending the latest events when stream=true, but they do when stream=false. Moreover, the output stops in the middle of the JSON object. This looks like a flushing issue on the DOMjudge side.

Your environment

• DOMjudge version: 8.3.0.
• Operating system: Debian 12 (bookworm)
• Web server: Apache 2.4.62

Steps to reproduce

The issue could be reproduced (with some instance of DOMjudge 8.3.0, I have also tried with demoweb, but the event-feed method always produces {"code":500,"message":"Internal Server Error"}) with the following commands:

curl -v --no-buffer --user 'user:passwd' \
    "https://<server name>/domjudge/api/v4/contests/<contest>/event-feed?stream=false&since_token=33874"
... request headers ...
< HTTP/1.1 200 OK
< Server: Apache/2.4.62 (Debian)
< Cache-Control: no-cache, private
< X-Accel-Buffering: no
< X-Current-Contest:
< Transfer-Encoding: chunked
< Content-Type: application/x-ndjson
{"token":"33875","id":"4301", ...}
... some more messages ...
{"token":"34174","id":"4323", ...}

The previous request sets the stream parameter to false, so the command finishes and outputs all the events. However, if we set stream to true, the command keeps waiting as expected, but it stops in the middle of an older event.

curl -v --no-buffer --user 'user:passwd' \
    "https://<server name>/domjudge/api/v4/contests/<contest>/event-feed?stream=true&since_token=33874"
... same headers and events ...
{"token":"34101","id

The stream does not progress unless there is more activity in the contest. Hence, events will arrive with some (unbounded) delay or will not arrive at all if there is no more interaction.

The problem is not likely on the client side because curl is run with the --no-buffer flag. Moreover, I have checked with Wireshark that the last chunk stops at "id and then only keep-alive messages are received.

Expected behaviour

The stream stops after the last event (in real time).

Actual behaviour

The stream stops in the middle of an older event.

[meisterT]

We have not seen this before, I wonder if this is an artifact of Apache or general webserver setup.

Just to double check - you see the same behavior as with curl also with httpie?

[ningit]

It must be related to the webserver: changing from Apache to nginx seems to solve the problem. Now the events are arriving in real time.

However, I followed DOMjudge's installation instructions to configure Apache (and now nginx), so this may also happen to other people using the default settings.

Just for completeness, I have tried with httpie when still using Apache with the same result. First, I ran

$ http GET "https://<server name>/domjudge/api/v4/contests/<contest name>/event-feed?stream=true&since_token=33874" \
            --auth 'user:passwd' --stream

and the output stopped at the event with token 44257 (which had happened more than 12 hours before). While this command was still waiting, in another terminal window, I ran the non-streamed version without --stream and with ?stream=false. The last event had token 44543 here (just 5 minutes before). In the other window, the --stream version was still blocked in 44257. I have waited for half an hour and the situation is the same (even after some new activity in the contest).

The stream has gone further in a previous run. It was blocked in 44257 or some event before that, but then it advanced some more events when there was new activity (still not until the last event).

[meisterT]

I am using Apache locally and am not seeing this problem.

The issue could be reproduced (with some instance of DOMjudge 8.3.0, I have also tried with demoweb, but the event-feed method always produces {"code":500,"message":"Internal Server Error"}) with the following commands:

This is a separate issue, most likely caused by a bug here: ef8c832#diff-efa9e2dd8b5fccf408219fb062d6e943b7c49fa715163a4a600fbb23c941bae1R659

[Kevinjil]

Could you try to modify you Apache configuration to include a ProxySet flushpackets=auto and ProxySet flushwait=500?
These should be used by the FastCGI module to flush if no new content is received from PHP in 500ms.

[ningit]

It works, I have added the following to the Apache configuration file

<Proxy "fcgi://localhost">
    ProxySet flushpackets=auto
    ProxySet flushwait=500
</Proxy>

and now the events are received in real time with Apache too.

Thank you!

[Kevinjil]

I'll open a PR to include it in the example configuration.