Inactive users thumbnails in the filmstrip with more than 5 participants

[crprata]

Description:

When I have more than 5 participants in the room and use filmstrip mode, the participants appear as inactive. When I change the title view to grid, it returns to normal. I checked the information on the js console and I got this:

2020-07-03T15:21:53.366Z [modules/RTC/BridgeChannel.js] <WebSocket.e.onmessage>: Channel new last-n event: (5) ["c655d6f0", "f6f05d5c", "20aa1132", "d53ddd54", "b814f683"] {colibriClass: "LastNEndpointsChangeEvent", endpointsEnteringLastN: Array(4), conferenceEndpoints: Array(5), lastNEndpoints: Array(5)}
Logger.js:154
2020-07-03T15:21:53.760Z [modules/RTC/BridgeChannel.js] <WebSocket.e.onmessage>: Channel new last-n event: ["c655d6f0"] {colibriClass: "LastNEndpointsChangeEvent", endpointsEnteringLastN: Array(0), conferenceEndpoints: Array(5), lastNEndpoints: Array(1)}

For some strange reason, the endpointsEnteringLastN: Array(0) goes to zero, even though I've set my LastN to 20 on config.js. I'm using the same version of jicofo, prosody and videobridge of meet.jit.si. I suppose that it's a bug. I think it's related to:
https://community.jitsi.org/t/inactive-users-in-with-black-screen/48489
https://community.jitsi.org/t/videostream-stopps-and-goes-to-connection-status-inactive/36981

The same behavior happens in meet.jit.si (https://prnt.sc/tb96jn).

Steps to reproduce:

1. Access https://meet.jit.si
2. Stay in a room with more than 5 participants using filmstrip view.
3. After a few minutes, all participants change to inactive status.
4. If you change to the grid view, it returns to normal.

Server information:

• Jitsi Meet version: meetjitsi_4203
• Operating System: Ubuntu Bionic

Client information:

• Browser / app version: Chrome Version 83.0.4103.116
• Operating System: MacOS Catalina

[crprata]

Can anyone help me? The same issue is happening on the latest stable Jibri with youtube.
https://community.jitsi.org/t/videostream-stopps-and-goes-to-connection-status-inactive/36981/24

[arktronic]

Another thread with the same issue: https://community.jitsi.org/t/inactive-users-thumbnails-in-the-filmstrip-with-more-than-5-participants/68962/8

I'm experiencing this as well, using docker-jitsi-meet. Everything was working correctly in stable-4857, but broke in subsequent releases.

[rasos]

I can confirm that since an update from 1.0.4127 to 1.0.4428 we frequently see inactive users in the filmstrip, without having bandwidth or network issues, even when there are only 5 users in a call.

We have set as limits:

 channelLastN: 8,    
 lastNLimits: {
     5: 12,
     20: 10,
     30: 8,
     50: 5,
     60: 2
 },

[saghul]

2020-07-03T15:21:53.760Z [modules/RTC/BridgeChannel.js] <WebSocket.e.onmessage>: Channel new last-n event: ["c655d6f0"] {colibriClass: "LastNEndpointsChangeEvent", endpointsEnteringLastN: Array(0), conferenceEndpoints: Array(5), lastNEndpoints: Array(1)}

For some strange reason, the endpointsEnteringLastN: Array(0) goes to zero, even though I've set my LastN to 20 on config.js.

Just to be clear, that is not the indication of a problem. endpointsEnteringLastN means the endpoints that just entered the last N set now.

As for the general problem, we have been a bit more aggressive at suspending participants due to network conditions. Note that available bandwidth is not the only metric, packet loss has significant impact.

[0xnor0]

I can confirm this behaviour with latest stable release.

[rasos]

@saghul is there a config parameter to set the packet loss threshold for making a user inactive, so that we can experiment with the best settings. Not everybody has a backbone like 8x8 :-)

[saghul]

@rasos There is no such setting. It's possible there is a bug in our bridge too. @jallamsetty1 do you know?

[0xnor0]

Any news on this?

[rasos]

Discussing this issue at the bi-weekly community call: switching users to video inactive happens, when packets stop arriving, or if there is a lot of loss, so when the video tag stops. This could be analyzed further by analysing SSRC of that person with wireshark.

[arktronic]

I'm sure that is a possible scenario for switching users to inactive, but it seems unlikely to be the only one, especially since the fact remains: older verions of Jitsi Meet simply do not exhibit this behavior (at least not nearly as frequently as newer ones do).

[petearvanitis]

I'm also seeing this increase more frequently as well (currently running 1.0.4466). At first I thought it was bandwidth (I'm hosting this on a Digital Ocean server in NYC1 during peak hours) but I'm also seeing this during non-peak hours with 3 users (vp8 @ 360p in tile mode, simulcast disabled), all using chrome browsers (no mobile) on fast connections, and all on ethernet. We ran speedtests to the region from each location and up/down bandwidth were well above where we needed to be during this timeframe. channelLastN is set to -1 (unlimited), but I changed to 20 just in case and that made no difference. It's almost random at which videos are hidden as INACTIVE. The only consistent thing is that the video always appears again when put back into full screen (vs tile) mode, or if the INACTIVE user speaks for over a second or so. So, if I had to guess, there is either a false positive putting the video into the INACTIVE state, or it was legitimately INACTIVE, but the check to restore it to ACTIVE isn't triggering fast enough. I'm wondering if some hysteresis on the state change would mitigate the issue, or if there is an issue w/ the state change in redux where the ACTIVE state isn't propagating to the component for some reason.

I have put a running build of the latest master commit of jitsi-meet and lib-jisti-meet on my server to try for my next call. Can you point me to to the files/methods that perform the check mentioned above (specifically where the connectionState is set to INACTIVE based on bandwidth and packet loss) so I can see how the server performs with that feature disabled?

I think I have the an (untested) workaround that should prevent the window from being hidden when it's connectionStatus is INACTIVE (we would prefer to see a freeze of last frame than the avatar), but I'd rather address the issue at the source of the status calculation vs altering the display code. I've spent the last few hours searching through the code of both jitsi-meet and lib-jitsi-meet, and didn't see anything that looked like it was checking frame or packet loss and setting video to the inactive state.

I'm also making the assumption that this hasn't already been addressed between 1.0.4466 and the master branch.

[damencho]

I've spent the last few hours searching through the code of both jitsi-meet and lib-jitsi-meet, and didn't see anything that looked like it was checking frame or packet loss and setting video to the inactive state.

The inactive status is coming from the bridge, the bridge decides there is not enough bandwidth and sets a video to inactive, and the client reacts to that change.

[rasos]

We have de-activated the threshold array and the inactive streams are gone.

    // Default value for the channel "last N" attribute. -1 for unlimited.
    channelLastN: 16,

   // deactivated below to check if array causes inactive video streams
    //    lastNLimits: {
    //         5: 12,
    //         20: 10,
    //         30: 8,
    //         50: 5,
    //         60: 2
    //    },

So maybe it calculates wrongly at some point. Should we see any message in the log when it is touching a threshold?

[jallamsetty1]

So maybe it calculates wrongly at some point. Should we see any message in the log when it is touching a threshold?

Thats an interesting observation. You should see the following info log in the browser console - "Setting last N to:" whenever the client tries to set the lastN value on the conference. Can you please grep for this log whenever you are able to reproduce this issue ?

[petearvanitis]

An update on this. Tried it again last night during peak hours w/ the following changes:

• disabled bwe via the videobridge conf.

On meet's conf file:

• simulcast on

• channelLastN 20

• Modified jitsi-meet code where I removed the functionality where the avatar was shown instead of the video when the connectionStatus is "INACTIVE".

What I saw was a massive improvement. As expected, I never saw the "Inactive" screen, I also didn't see any video freezes or dropouts. We did get choppy/low framerate video from time to time, but never lost video.

So this leads me back to either the videobridge marking video as inactive when it isn't, or the a issue w/ the redux state where the user connectionStatus is being left as INACTIVE state when it's not.

[jallamsetty1]

What I saw was a massive improvement. As expected, I never saw the "Inactive" screen, I also didn't see any video freezes or dropouts. We did get choppy/low framerate video from time to time, but never lost video.

@petearvanitis , the fact that you were getting choppy/low fps video indicates that the bwe values calculated by the bridge are coming in low and the bridge was struggling to send the full 30 fps video. I assume that in this the case, bridge would mark the stream as inactive and the UI would display an avatar (w/o your modifications). If you are pretty sure you had enough downlink b/w on those endpoints, someone from the bridge team should take a look. cc: @bgrozev @JonathanLennox

[petearvanitis]

The speed test from my house to my NYC1 DigitalOcean server is usually 200MB+ down and 20MB+ up during non-primetime hours. During last Saturday night (primetime hours), the DO speeds dropped down to 40MB down and 6-8MB up. That is still more than what was required but given the overall network saturation, I could see reduced framerates being a result. As a side note, I also verified (through other speed tests) that the congestion was to DO as I got normal speeds from other test sites like fast.com, etc. I mention this as the type of network congestion we were experiencing (national vs local ISP) in case that helps w/ diagnosis of the issue.

That being said, if the server is using 30fps to determine active/inactive, then that is waaaay too stringent of a test for any real world usage. I suggest making that an editable value via the server conf files, or dropping that significantly. I'd rather see a 5fps video feed than no feed at all. I would say the average framerate we were seeing were probably closer to 15-20fps consistently.

From a use-case perspective, the majority of my calls are ~6 user tile-view conferences. I could see the benefit of 'inactive' in a presenter-type use case, but for tile view it makes the app almost unusable for video.

[bgrozev]

There are two very differrent conditions that can result is something similar to what is described here.

The bridge marks an endpoint as INACTIVE if it receives no packets from it for 3 seconds (this includes responses to periodic ICE consent checks), it has nothing to do with the FPS. When an endpoint becomes inactive, the bridge broadcasts this info to the other endpoints in the conference. I'm not quite sure how the UI handles this, I think it greys out the endpoints viewport and shows an "inactive" message. Since this is between a sending endpoint and the bridge, BWE configuration has no effect on it.

The second condition is estimated bandwidth towards a specific receiver being too low. In this case the bridge may decide to not forward video for some of the endpoints. When this happens, the bridge sends a LastNEndpointsChangeEvent message to the receiver, with lastNEndpoints set to the endpoints for which it is forwarding video. When a client sees that an endpoint has video enabled (via signaling), but it does not appear in the lastNEndpoints, it assumes that it has been suspended due to bandwidth constraints and shows the "ninja" icon (as seen in the original post). Disabling BWE (videobridge.cc.trust-bwe=false) makes the bridge ignore the bandwidth estimation, and forward the highest quality streams at all times.

[jallamsetty1]

There are two very differrent conditions that can result is something similar to what is described here.

The bridge marks an endpoint as INACTIVE if it receives no packets from it for 3 seconds (this includes responses to periodic ICE consent checks), it has nothing to do with the FPS. When an endpoint becomes inactive, the bridge broadcasts this info to the other endpoints in the conference. I'm not quite sure how the UI handles this, I think it greys out the endpoints viewport and shows an "inactive" message. Since this is between a sending endpoint and the bridge, BWE configuration has no effect on it.

The second condition is estimated bandwidth towards a specific receiver being too low. In this case the bridge may decide to not forward video for some of the endpoints. When this happens, the bridge sends a LastNEndpointsChangeEvent message to the receiver, with lastNEndpoints set to the endpoints for which it is forwarding video. When a client sees that an endpoint has video enabled (via signaling), but it does not appear in the lastNEndpoints, it assumes that it has been suspended due to bandwidth constraints and shows the "ninja" icon (as seen in the original post). Disabling BWE (videobridge.cc.trust-bwe=false) makes the bridge ignore the bandwidth estimation, and forward the highest quality streams at all times.

Thanks for the explanation @bgrozev. In that case, maybe we can narrow down the issue by disabling BWE on bridge via the config and not make the INACTIVE state modification in the client. @petearvanitis, if you see the same behavior as your last test, then it could be BWE related. If you still see the ninjas, you can then apply your client side modification for INACTIVE state and see if the ninjas disappear. If that is the case, there could be an issue in the clients handling of connectionStatus updates. Does that sound reasonable @bgrozev ?

[arktronic]

I can confirm that, after setting trust-bwe=false, I'm no longer seeing inactive ninja-participants.

[pdarcos]

I'm having the same issues here. Where exactly does one set trust-bwe=false?

Thanks

[pdarcos]

I tried adding

BWE {
videobridge.cc.trust-bwe=false
}

}

to jvb.conf but that didn't work

[arktronic]

@pdarcos Inside of the existing videobridge { ... } structure in jvb.conf, add:

    cc {
        trust-bwe=false
    }

[pdarcos]

Thank you @arktronic

[rjacquemin]

@arktronic @bgrozev

I am curious by setting

 cc {
        trust-bwe=false
    }

you are basically telling the JVB to not trust bandwidth estimation sent by clients ?

How jvb will handling the streams ?
keep forwarding video stream even if client says he has crappy network ?

Does that have an impact on simulcast ?

let me know if you know some other side effects

[Tiennchen]

Yesterday I have had two different Jitsi-Sessions (Browser: Edge, Windows 10,public server meet.jit.si) each of 1 hour duration , both with more than 4 participants.
I was teaching dancing the others and it was a problem, that I wasn't able to see them in tile mode. Audio was working perfectly.
During both sessions I have had the inactive sessions of the other participants and their thumbnails were black. Some screens of the participants flipped on for a short time and went black again, some were always off. My Bandwith: 50 Mbit down, 11 Mbit up.
All other members of both meetings haven't had any problem. Only my sessions have had these issues.
Unfortunately I have not tried to click on individuals (not use the tile overview mode) to see if that is working.
The other participants usually view only my video (not the tile view), perhaps was that the reason why it worked for them.

I am always using the same equipiment and have done these kind of sessions weekly already for more than 7 times since February 2021 without issues.

In parallel I logged in to these meetings with my smartphone --> no issue with thumbnails everything was fine

How can I investigate the root cause?
What can cause these kind of issues with seeing the other participants of the meeting?
Is there a workaround?

Thank you very much for you help.

[rabesocke]

We have the same use case as @Tiennchen and see also such problems at several trainer since around 4 weeks. Before that, there were no problems at all with the public server meet.jit.si.

In most cases the trainer has the problem, that he/she doesn't see the participants, but the participants see each other.

Is there anything we can do in order to get this working again smoothly?

[damencho]

We had identified a problem and will be updating meet.jit.si very soon.

[damencho]

jitsi/jitsi-videobridge#1643

[rabesocke]

@damencho: Is the fix already online on meet.jit.si?

I'm asking, because we will have tomorrow a new training session and want to inform our trainers about the situation.

[damencho]

Yes, it is.

[Tiennchen]

Thanks for the information. I will check on my next Jitsi-Meetings on the Public Server.

[rabesocke]

Here a short feedback: Unfortunately the problem still exists and was presents last Tuesday. The interesting thing: With another computer over LTE everything was working fine. But before the problem occurs, the meetings went well with the completely same setup.

[Tiennchen]

Yesterday I was part of two Jitsi-Meetings on meet.jit.si. There was again the issue with "inactive" Video-Thumbnails in Tile Mode with more than 4 participants.
The Videos of the others kept most of the time black, sometimes switched shortly on/off. At 21:15 (CET) pm suddenly all 7 videos were visible for 1 minute, then partly went off again. As all Videos were visible the Down-Bitrate was about 400 Kbps.
LAN-Internet-Connection Speedtest showed me Up 11 MB, Down 51 MB.
I attached the Connection-Information Screenshot of my Jitsi-Connection. It's only a Snapshot for a time when it was not working (Down-Bitrate 8 Kbps).

[initpwn]

An update on this. Tried it again last night during peak hours w/ the following changes:

• disabled bwe via the videobridge conf.

On meet's conf file:

• simulcast on
• channelLastN 20
• Modified jitsi-meet code where I removed the functionality where the avatar was shown instead of the video when the connectionStatus is "INACTIVE".

What I saw was a massive improvement. As expected, I never saw the "Inactive" screen, I also didn't see any video freezes or dropouts. We did get choppy/low framerate video from time to time, but never lost video.

So this leads me back to either the videobridge marking video as inactive when it isn't, or the a issue w/ the redux state where the user connectionStatus is being left as INACTIVE state when it's not.

Hi where you've modified the code for Jitsi meet inactive status?
I'm also having blankscreen problem.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[0xnor0]

No bot pls the issue still exists

[saghul]

Have you tested the latest stable? Significant changes have been made in this are in the last few releases.

[rasos]

I can confirm that we do not have any further issues with inactive users / dropping videos with the current release. Thank you Jitsi team for taking care on this.