[bug] publishing glitches on certain Android devices

[davidzhao]

Describe the bug

When publishing on certain devices, their video appears to have an artifact. Sometimes it's a green border, other times it appears distorted. This appears hardware specific.

The user also reported that it could be related to their internet connection.

To Reproduce

User reports reproduction on

• HUAWEI P30 Lite
• HUAWEI Y9 Prime 2019

[furkanKotic]

flutter client sdk : v1.1.11
livekit server : v1.3.2

[furkanKotic]

New user phone models

Black/white video issue phone models
Xiaomi Redmi Note 8 Pro
Oppo Reno2 Z

Green border issue phone models
Samsung Galaxy J6+
Huawei P40 Pro

[davidliu]

@furkanKotic would you happen to know if it happens only some times or all of the times for the affected devices?

[davidliu]

@furkanKotic Also, who's reporting the visual artifacts? Are the streamers seeing the issue on their streaming phone, or is it viewers seeing issues on other video streamers? Curious to see if it's potentially a decoding issue vs. encoding issue.

[furkanKotic]

@furkanKotic would you happen to know if it happens only some times or all of the times for the affected devices?

It happens only some times. The broadcaster's video freezes and then continues, but a green stripe appears at the bottom of the video or its video is black and white.

[davidliu]

Is it possible to get logs for when this happens? Wondering if this is caused by a bad reconnect.

[furkanKotic]

The system runs on kubernetes. It is necessary to examine the pod logs at the time of the problem. We do not have such a record yet.

[furkanKotic]

@furkanKotic Also, who's reporting the visual artifacts? Are the streamers seeing the issue on their streaming phone, or is it viewers seeing issues on other video streamers? Curious to see if it's potentially a decoding issue vs. encoding issue.

We're pretty sure the viewers has this problem. We are examining whether the streamers themselves see these problems.

[furkanKotic]

For the black and white problem: The broadcaster sees herself as normal, while the audience sees her in black and white.
For the green line problem : We are not sure, but the publisher seems to be seeing the green line in himself.

[furkanKotic]

@cloudwebrtc Could the PR below be a solution to this problem?
flutter-webrtc/flutter-webrtc#1258

[cloudwebrtc]

it seems that the reason is the same, but we may need to replace the frame from here https://github.com/webrtc-sdk/webrtc/blob/m104_release/sdk/android/src/java/org/webrtc/CameraCapturer.java#L155-L167. I can modify a version, but I don't have an android device to reproduce this issue. maybe you can try to verify

[furkanKotic]

I do not have a device with exactly this problem. I can try to send a test application to my users who are having the problem.

[furkanKotic]

@cloudwebrtc @davidzhao @davidliu

One minute before and after is as follows:

Before	After

Some places are specifically censored.

[tomdk333]

My answer is semi related as this is the main Google entry for the "glitching" we've been running into.

Devices:
We've observed a different kind of glitching with Livekit in Angular, only on some Android devices running Chrome, mainly:

• Samsung Tab Active 3
• Google Pixel 7 pro.
  No windows device exhibited this effect. A now quite old Galaxy Tab S2 would also render the videos fine.
  The affected devices would also render fine when hardware acceleration for WebRTC video was disabled (Firefox also worked fine).

Symptoms
This would occur whenever 2+ feeds rendered on screen (we have one large feed and small feeds over it).
In our case, it would seem that a sections of the main feed would "freeze" and flash in on every second video frame, overlapping with the current frame from that camera.
The attached mock up is meant to illustrate that ("glitch" below black arrows).

After quite the amount of experimentation, we noticed that the artefacts occurred at the relative to other UI components.

Solution?
We did notice that the rendering issues would go start going away when the UI component relative to that glitch were touched, causing them to render an update.

Our solution was to render a UI component above that affected area (on the sides of the screen).
We simply added a div that shows the current timestamp, every 100ms..

// Sample code in Angular component
<div id="anti-glitch" style="width: 100%;
  position: fixed;
  top: 0;
  height: 10px;
  z-index: -1; 
  display: flex; 
   justify-content: space-between;">

  <span>{{ time.on100ms }}</span>
  <span>{{ time.on100ms }}</span>
</div>

Mock up:

Then we simply added that component behind the UI elements and made, removed the red background and gave the font the colour rgba(0, 0, 0, 0.01) to make it invisible, while still causing the browser to render something.

I hope this helps at least someone as that annoying glitching behaviour has cost a lot of time in trouble-shooting.