Load test call fails - Calls are processed too slow on the headless Firefox side

[atsakiridis]

I reproduced this when load testing 50 concurrent calls (i.e. having 50 browser tabs registered with Restcomm and receiving calls). With 20 it is not reproducible.

Analysis:

Sipp starts sending SIP requests
The browser webrtc clients start answering, but it seems that overtime 200 OK sent by the clients are delayed reaching more thatn 30 secs at which point Restcomm times out and CANCELs the calls towards the clients, which in turn terminate the call prematurely
I don't see too much load on the browser (no swapping, no excess waiting on I/O, memory and CPU consumption seem normal) that would explain it. And some webrtc client side errors seem irrelevant to that behaviour

Notice that I tried originally hosting the headless browsers in EC2 with magnetic HD and the issue occurred as follows. The delay from receiving INVITE until sending back 200 OK starts at ~5 seconds (delay to answer included) in the first call, 12 seconds in the 5th call, 19 seconds in the 10th call and so on until Restcomm times out.

I then tried general purpose SSD and seems that it occurs less. delay from receiving INVITE until sending back 200 OK starts at ~5 seconds (delay to answer included) in the first call, 7 seconds in the 5th call, 9 seconds in the 10th call and so on until Restcomm times out. But still there's an issue.