[Bug]: rendering broken in 94.1.2

[shiroyagi]

Steps to reproduce

1. go to a website eg. en.wikipedia.org
2. scroll down the page

Expected behaviour

Text and images should be legible

Actual behaviour

Some images are corrupted or not drawn. Some text is replaced with incorrect characters (some not ASCII), sometimes overlapping. Scrolling backwards and forwards will change which text and images are rendered correctly, so after a while it may be possible to read the whole page.

Device name

Motorola Moto X

Android version

Android 5.1

Firefox release type

Firefox

Firefox version

94.1.2

Device logs

No response

Additional information

The same behaviour applies to Fennec 94.1.1. The previous release of Firefox and Fennec 93.1.0 work correctly.

┆Issue is synchronized with this Jira Task

[shiroyagi]

See below:

[Mugurell]

This seems to be device specific.
Can you please try newer versions of Firefox, like the current Nightly or Beta and test if those are still affected?

cc @jamienicol

[shiroyagi]

Yes, same problem with nightly 96.0a1. Note you may have to scroll back and forth a few times to replicate the problem. Sometimes I see it almost immediately and nearly everywhere, other times not so much.

[Mugurell]

Moving to Bugzilla for more investigation from the graphics team.
Please follow the progress there.

[Mugurell]

Moved to bugzilla: https://bugzilla.mozilla.org/show_bug.cgi?id=1741174

Change performed by the Move to Bugzilla add-on.

[jamienicol]

@shiroyagi thank you for the bug report. Could you please go to about:support, click "copy text to clipboard", and attach the text to this bug?

Since you say this is a regression in version 94, it would be incredibly helpful if you could run the program mozregression. You can either install the GUI version from the link there, or if you are running Linux or Macos I find it's easier to use the command line:

pip install --user mozregression

Then plug your phone in to the computer and make sure USB debugging is enabled. Open the mozregression application and click the scissor icon to start a new bisection. On the first screen, select gve from the "application" drop down menu then click next. Then click "next" again to skip the profile selection screen, then on the "build selection" screen change the "date" dropdown menus to say "release", and and select 93 as "good" and 94 as "bad". Then click "finish".

Alternatively, if using the command line just run

mozregression --app gve --good 93 --bad 94

This will download and run a series of versions of firefox. For each one, see if you are affected by the bug, then enter either "good" or "bad". Eventually this will tell you what change to the code caused the bug.

I know that's a bit complicated, sorry. But it will be incredibly helpful to us! Please let me know if you need any help with that!

[shiroyagi]

This is from my usual install. A fresh install of nightly (no extensions etc) gave the same result:
support.txt
I will try the bisection later. Should I comment here or #1741174 in future?

[jamienicol]

On bugzilla would be preferable, but here is fine too if that's easier for you

[shiroyagi]

Here is the end of the bisection, is this enough?

36:54.89 INFO: Narrowed integration regression window from [ebefb02b, 234f83cf] (4 builds) to [ebefb02b, d31a1c19] (3 builds) (~1 steps left)
36:54.89 INFO: Pushlog:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=ebefb02bb4e60bda8dff9397417fdad802e96645&tochange=d31a1c199922d7b4e83edb297ea63975a264c487

36:54.89 INFO: Using local file: /tmp/tmp9prats72/d2720681264c-shippable--autoland--arm--geckoview_example.apk (downloaded in background)
36:54.89 INFO: Running autoland build built on 2021-10-06 16:08:21.837000, revision d2720681
36:54.92 WARNING: Unable to find application.ini
36:54.92 WARNING: Unable to find platform.ini
36:57.29 INFO: Resetting test root from /data/data/org.mozilla.geckoview_example/test_root to None
37:44.73 INFO: Setting run_as_package to org.mozilla.geckoview_example
37:46.28 INFO: Setting test_root to /data/data/org.mozilla.geckoview_example/test_root
37:46.98 INFO: launch_application: am start -W -n org.mozilla.geckoview_example/org.mozilla.geckoview_example.GeckoViewActivity -a android.intent.action.MAIN --es args '-profile /data/data/org.mozilla.geckoview_example/test_root/tmpuwf_6a9p.mozrunner' --ez use_multiprocess True
Was this integration build good, bad, or broken? (type 'good', 'bad', 'skip', 'retry', 'back' or 'exit' and press Enter): bad
38:23.99 INFO: Narrowed integration regression window from [ebefb02b, d31a1c19] (3 builds) to [ebefb02b, d2720681] (2 builds) (~1 steps left)
38:24.00 INFO: No more integration revisions, bisection finished.
38:24.00 INFO: Last good revision: ebefb02bb4e60bda8dff9397417fdad802e96645
38:24.00 INFO: First bad revision: d2720681264c9712c04583e9d0670adb51a4f861
38:24.00 INFO: Pushlog:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=ebefb02bb4e60bda8dff9397417fdad802e96645&tochange=d2720681264c9712c04583e9d0670adb51a4f861

[jamienicol]

Yes that's correct. But the result is a little bit surprising. Would you be able to try again just to confirm you get the same result? Thanks!

[shiroyagi]

It took quite a long time as the number of builds jumped back up:
17:05.00 INFO: ************* Switching to autoland by process of elimination (no branch detected in commit message)

The most likely mistake would be a false negative (good), if the problem wasn't evident. Does the algorithm detect inconsistent responses?

Can I start from closer commits to just confirm the result?

[jamienicol]

Unfortunately no it can't detect that. However, if you notice that after a certain point all of your answers are the same (all good or all bad) then it is probable that you've gone wrong somewhere.

Do you still have the full log available or have you closed the window? If you still have the log then please post it

[shiroyagi]

Yes
mozregression.txt

[jamienicol]

Thank you! It's my evening here now, but tomorrow I will create some builds for you to test to confirm.

[jamienicol]

@shiroyagi could you please try these two builds and see if you can reproduce the bug in them:

1. https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/Dt29Ax8cRMWqe7uibTA0PA/runs/0/artifacts/public/build/geckoview_example.apk
2. https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/XkYHya1yR_uuqVcGIBmL8Q/runs/0/artifacts/public/build/geckoview_example.apk

[shiroyagi]

1. Dt29 does have the bug
2. XkYH does not have the bug

[jamienicol]

Thanks for testing. That does confirm that your bisection was correct.

Would you be able to test whether setting widget.non-native-theme.scrollbar.size.override to -1 in about:config fixed the issue?

[shiroyagi]

Using the app above 1. Dt29 if I set widget.non-native-theme.scrollbar.size to -1 it seems to partly fix it. Scrolling quickly I can see some of the text is sometimes corrupt as before, but as soon as scrolling stops it is quickly drawn correctly.

There is no widget.non-native-theme.scrollbar.size.override, adding it seems to have the same effect.

2. XkYH appears to behave better than this: I can't see any corruption even with fast scrolling, although it does occasionally stutter.

I assume there is no way to use about:config in Firefox, although I notice it is still available in Fennec.

[jamienicol]

@shiroyagi oh. I've just realized that widget.non-native-theme.scrollbar.size.override was only very recently added (and widget.non-native-theme.scrollbar.size was removed). In those builds above the override version doesn't exist yet so should have no effect.

Dt29 is the first build with the new scrollbar rendering code. Setting the size to -1 helps, but does not fix it entirely.
XkYH is the last build with the old scrollbar rendering code. So is unaffected.

So it seems like a driver bug which is triggered by the new scrollbar rendering code.

Unfortunately there is no about:config on Firefox stable, but you can install Beta or Nightly from the play store and use it in those.

[shiroyagi]

Thanks, probably my mistake was not to refresh the page after testing widget.non-native-theme.scrollbar.size=-1 and replacing it with widget.non-native-theme.scrollbar.size.override.

So if this is a driver bug, what will the resolution be? Will there be a code change to, eg, detect this hardware/version or will it rely on users setting about:config?

FYI I have never seen similar behaviour in any other app.

[jamienicol]

I'm trying to get my hands on a device which can reproduce the issue, and then I will find a workaround for it. Apologies for any inconvenience this has caused in the meantime.