Possible restart behavior experienced by some users

[wingman-jr-addon]

I've gotten feedback from the exit polls as well as on the ratings that for some folks Firefox seems to do weird things, almost like it is constantly restarting the plugin. Unfortunately I don't see the issue myself and I don't have a particularly easy way to get logs from users so it is a little difficult to figure out what's going on.

I think the first thing I'd like to do is collect raw user feedback so that patterns can emerge.

From the user review kitsunesan:

Amazing extension
WTF is up with firefox though? Since the past few days, it warns me about this extension literally EVERY minute.
It would be really nice if the developer has a solution to this. I'm okay with going into advanced settings and disabling stuff

What I've observed behind the scenes is that - at least at one point - Tensorflow.js seemed to have a bug where predictions would start becoming incorrect. To counter that, I have Firefox get the initial prediction for a known image, and then test it again on a recurring basis to make sure it's still getting the expected value. If it doesn't, it will restart the plugin eventually. That could be what's causing this.

[Umbrella1234]

Hey, thanks for the extension! This is what I'm seeing. Even if I click the "Keep tabs hidden" button, the pop-up keeps showing up every 30 seconds or so, it looks like there is no way to stop the pop-up from showing up, which is super-annoying. I'm using Firefox 111.0.1 on ubuntu, let me know if you need any other details that can help.

[wingman-jr-addon]

Hey thanks for the update @Umbrella1234 - I'm not sure how long you've been using the addon, but is this new behavior with e.g. a Firefox update or has it always been like this? Also, are you running Ubuntu natively or in a virtual machine? Thanks for any info you can share.
(Update: was just trying with Ubuntu 22.04 HyperV VM and latest Firefox - could not get issue to happen, so maybe something hardware dependent)

[Umbrella1234]

I'm running ubuntu natively (22.04 version), installed it 2 days ago. I hadn't used the addon on ubuntu before, I used to use it on windows and it worked fine. I started seeing the popup right after the installation of the addon

[Umbrella1234]

Made a screen recording of the bug, maybe it can help a little bit https://app.birdeatsbug.com/workspaces/453b3ab0-f26d-4e63-8461-80ba12ef7028/collections/c7ca043f-253b-4c72-af9d-ac30d75fc6ba/sessions/T-PdlK-sdPPedeC-EL6SFcfewtQe1dQ54cz9gy1QHtSs

[wingman-jr-addon]

Thanks, that is useful. It definitely looks like the addon is reloading, first at about 10-11 seconds after start, and then about 20-22 seconds after start.

I'm not sure how technical you are, but I could make a branch of some test code to try. If you've never loaded an addon in Firefox temporarily, it's quite simple: clone the code and switch to the right branch, go to about:debugging in Firefox, then "Load Temporary Addon..." and pick any file in the plugin folder. I don't have any build process so that's all there is to it.

What I'd do as a first step would be to disable the crash detection restarts. I'd leave the logic so we see errors, but I'd disable it from restarting so we could diagnose why it's cranky. It might "work", but there's a good chance that if the crash detection is having a problem, something is actually wrong.

Let me know if you have interest in pursuing this route.

[Umbrella1234]

Yea, I can try to help, let me know what branch to download the test code from. My knowledge about browser extensions is quite limited since it's not my area of expertise, but I can try following your instructions.

[wingman-jr-addon]

All right then @Umbrella1234 let's give it a try!

1. Checkout this branch: https://github.com/wingman-jr-addon/wingman_jr/tree/check-restart-behavior
2. Type "about:debugging"
3. Hit "Load Temporary Addon..."
4. Browse the git repository folder and select any file in the root, like manifest.json
5. Hit "Inspect" for Wingman Jr. on the about:debugging page once it is loaded.
6. Go to the Console tab.
7. Type CRASH in the text filter box.
8. Ensure that all the log levels are enabled for viewing (Error, Warning etc.)
9. Wait and/or browse to reproduce the issue you were seeing. Make sure to let it run for at least 60 seconds so we can see what's going on.
10. Select/copy/paste logs to a file and post it here.
11. Bug is found and fixed 🤣

[Umbrella1234]

log.txt
Here you go.

[wingman-jr-addon]

Thanks @Umbrella1234 ! So what we have here is quite interesting - the second prediction is not the same as the first on your platform, but every prediction from the second one on seem to be the same. On my system I see [{"0":0.13779525458812714},{"0":0.48976776003837585,"1":0.4457016587257385,"2":0.057600148022174835,"3":0.006930441129952669}], but it is stable.

I've pushed a potential fix where I let the prediction pipeline warmup a few times before I capture the expected result. In theory if you're seeing the same one after the first time, this should fix it.

Can you please update your branch and try once more? This has the proposed fix and will restart if it has problems so we're looking for no restarts.

[Umbrella1234]

log.txt

Here is my console log from the updated branch. It looks like the bug got fixed, the popup stopped reappearing and all the nudes get blocked yay. However, there are still errors in the console.

[wingman-jr-addon]

Thanks @Umbrella1234 for the followup on the logs. It's interesting to see it's still getting destabilized for a while, but the eventually clears up - I'm sure that's still annoying. Thinking about options here I think we've got a few different options to explore:

1. Increase the warmup period. I would have thought that three times would have been enough based on the first set of logs, but maybe it's non-deterministic. If so, this approach will always break down eventually.
2. Implement an "approximately equal" check instead. It looks like that might pass for your current set of logs too.
3. Try the WASM backend and see if you get consistent results. Might result in a slowdown though.
4. Upgrade the Tensorflow.js library version. I'm on 3.x and 4.x is now available. Maybe it would fix something, maybe not. When I upgraded from 2.x to 3.x initially I saw performance degradation so I had to hold off a while, but 3.x got there eventually.

Maybe I'll try adding approach 2 in addition to the warmup measures already added and it'll stabilized. Gonna think on this for a bit.

[Umbrella1234]

Thanks for trying to fix this :) Let me know if you need my help with testing things out.

[wingman-jr-addon]

All right @Umbrella1234 - I implemented a basic form of approach 2 from above. Please give it a try and let's see if your logs show that the crash detection passes consistently...

[Umbrella1234]

@wingman-jr-addon
log.txt
Looks like something is wrong. I can't access any websites while the add-on is on because websites don't load.

[wingman-jr-addon]

Huh, that's interesting @Umbrella1234 - it doesn't even seem to be reaching the new code on this run, almost like it's failing for some other reason entirely right when loading. Maybe try removing the addon and installing again? Otherwise the full logs might help but they might get a little too verbose.

[Umbrella1234]

log.txt
Here is a full log, maybe that helps.

I tried reinstalling it as a temporary addon on the about:debugging page, it didn't help. And it's not installed as a browser extension.

[Umbrella1234]

It looks like the problem was caused by webGL not loading. I fixed it by setting webgl.out-of-process to true and webgl.force-enabled to true in about:config. Now there are no errors in the console and the extension seems to work properly. I wonder if there is a way to make the extension work without those settings.

[wingman-jr-addon]

Good find @Umbrella1234 - given that the extension had worked previously but then stopped, it is rather curious and makes me wonder if there was some issue with another part of running state. For example, I'm guessing you had restarted Firefox itself but had you restarted the OS? My gut says it might be something like some sort of bad recovery from sleep mode etc. - but on the other hand it could be Ubuntu + Firefox + WebGL as I see lots of issues on the web about those as well. :(

[Umbrella1234]

I hadn't restarted the OS, just changed my browser settings. I tried restarting the OS, it doesn't fix the issue.

[wingman-jr-addon]

That's too bad @Umbrella1234 , I'm not quite sure what I can do to help on that part as I think it is a Firefox issue.
However, back to this particular fix - after the settings changes you made, does it run without restarting the hidden tab? If so, I'll get it merged in for whenever the next release goes out.

[Umbrella1234]

Yes, your fixes resolved the issue for me, I think it's worth merging.

What's weird though is that I didn't change any settings or whatnot prior to seeing the webgl error, it started happening out of the blue. Hopefully, no one else is going to run into this.

[wingman-jr-addon]

Indeed I hope we don't see more of that new issue...
Thanks for your help and I'll merge this one.

[wingman-jr-addon]

See #189