XP becomes unresponsive after use of management endpoint

[rymsha]

Thread dump shows that resteasy locks all threads

It happens when SSE is used extensively

[gbbirkisson]

This was observed in the operator. Here is a typical communication between the operator and XP:

GET /app/events HTTP/1.1
Accept: text/event-stream
Authorization: Basic c3U6c2Nvb2JlZG9v
Host: all.my-namespace.svc.cluster.local:4848
Connection: Keep-Alive
Accept-Encoding: gzip
User-Agent: okhttp/4.9.1

HTTP/1.1 200 OK
Date: Thu, 03 Jun 2021 07:38:16 GMT
Set-Cookie: JSESSIONID=1ezktbxlreaid1wltymdsamgk0; Path=/; HttpOnly; SameSite=Lax
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Content-Type: text/event-stream
Transfer-Encoding: chunked

....

Replicating with curl:

$ curl -N \
     -H 'Accept: text/event-stream' \
     -H 'Authorization: Basic c3U6c2Nvb2JlZG9v' \
     -H 'Accept-Encoding: gzip' \
     -H 'Connection: Keep-Alive' \
     http://all.my-namespace.svc.cluster.local:4848/app/events

GET /app/events HTTP/1.1
Host: all.my-namespace.svc.cluster.local:4848
User-Agent: curl/7.76.1
Accept: text/event-stream
Authorization: Basic c3U6c2Nvb2JlZG9v
Accept-Encoding: gzip
Connection: Keep-Alive

HTTP/1.1 200 OK
Date: Thu, 03 Jun 2021 07:50:43 GMT
Set-Cookie: JSESSIONID=rrl02pxhpjda1cnjbx9yfq8rl5; Path=/; HttpOnly; SameSite=Lax
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Content-Type: text/event-stream
Transfer-Encoding: chunked

...

[rymsha]

It appears that JaxRs wrapper uses old Servlet 2.5 "async simulator" that keeps holding a Jetty thread even when client disconnects from events stream. A attempt to send a single event to all those disconnected clients would free all stuck threads, but if events are rare, potentially all jetty threads get occupied.

Threads are no longer occupied by SSE requests when native "Servlet 3.0 Async" is in use.