STG takes forever to start up, waiting for all tabs to load

[jlokier]

Describe the bug
I have about 5000 tabs in about 40 STG groups. Switching between groups is fine. And restarting the browser to reload the session is quick too, when STG is disabled. But restarting the browser with STG enabled, or enabling STG after disabling it, are consistently either extremely slow, or never finish. Most tabs are in the "discarded" state, I don't know if this is relevant but it might be, if STG thinks these are "not yet loaded" forever. In addition, while STG is in this state, opening a new tab and typing a URL to visit a site often gets stuck loading the site with nothing appearing and the "loading" spinner carrying on. This is a little odd because the existing tabs that are reloaded on Firefox startup do load just fine. This issue makes STG practically unusable, even though it works quite well if it does manage to start up.

To Reproduce
Steps to reproduce the behavior:

1. Have only two extensions enabled, to rule out other things like uBlock Origin being a problem: STG and Auto Tab Discard.
2. Have about 5000 tabs in about 40 groups, most of them in the "discarded" state put there by the Auto Tab Discard extension.
3. Most of the tabs are hidden, by STG, because they are in groups which aren't current in any window. There are only a few windows (4 in my case).
4. Disable the Auto Tab Discard extension, so it should just be the Firefox "discarded tab" state that is the problem, not that extension.
5. Quit Firefox, wait for it to finish quitting (this is quick).
6. Start Firefox.
7. Wait forever for STG to stop spinning, or on some occasions, wait just a few minutes.

Expected behavior
Firefox loads in about 10-20 seconds with all 5000 tabs just fine, without STG enabled. This is whether most tabs are hidden or not.

Screenshots
No picture, but the STG icon is a spinner which spins, and the tooltip says it is waiting for all tabs to load. The STG Group Manager window also says this, if it was open when Firefox was quit (because this tab is recreated on STG startup). If other add-ons are enabled, then other add-on icons don't work properly either while STG is in this stage: Their popups tend to be empty small boxes, instead of normal menus.

Desktop (please complete the following information):

• OS is Mac OSX 10.9 (Mavericks)
• Firefox is 67.0b14 (beta channel at the time of writing)
• STG version is 3.4.4, updated 26 April 2019

Additional context
Some ideas:

• In normal use I have a lot of unloaded / discarded tabs. This is normal after Firefox restarts, as it only reloads tabs on demand as they are selected. This is great for Firefox restart time, (or session restore time after a FF forced quit): FF starts up in 10-20 seconds even with 5000 tabs. The STG message about waiting for all tabs to load makes me wonder if it's waiting for tabs to load that will never load.
• It's impossible to go through and look for all tabs to load. Most of them are hidden, and there are thousands. I have tried making sure all non-hidden tabs are loaded, and that didn't help.
• I have tried making sure no other extensions are enabled, and that didn't fix the problem.
• I have tried restarting Firefox with STG disabled, and then enabling STG. That didn't fix the problem.
• I have seen STG abort bulk operations when one of the tabs has "about:" in the URL, such as "about:performance", and sometimes given a notification with an error containing the word "undefined" and a couple of numbers (and mentioning the URL). I wonder if this is causing some internal STG process on startup to fail to complete, because when testing I sometimes have "about:addons" or "about:debug" open.
• There are some other STG operations which are very slow. Maybe quadratic time algorithms are being used for some tab operations and the same at startup? A few times I have been shown a long list of unsynchronised hidden tabs. The menu options to add them all to the current group or to a new group didn't do anything. So I selected all items in the list at once and added them to a group. This took maybe 10 minutes. Strangely, at other times, STG has managed to create a group and add >1000 tabs to it in just a few seconds.
• Every so often I end up with a few hundred "about:blank" or "New Tab" tabs in a group. It worries me that STG is causing saved tab state to be lost, but if it's just creating extra empty tabs I'm not so worried. It would be good to know.

[jlokier]

A few other things I'd like to add:

1. Thanks for developing STG. The UX is really nicely put together and it clearly has quite a bit of time and care put into it. My bug reports may come across as all about problems, so I want to include a note of appreciation as well.

2. While the STG icon is spinning waiting for all tabs to load, for a few minutes or seemingly forever (it varies which), the Firefox CPU load is around 200-300% on my Mac (that's 2-3 cores). All active tabs in all windows are loaded quickly, and Firefox doesn't load other tabs (they are loaded on demand as they are selected), so I'm wondering if the 200-300% CPU is STG doing something intensive. I was unable to open the add-on debug window to try the Performance tab on it, because clicking "debug" in about:debug doesn't do anything while the STG icon is spinning - it waits until the icon stops spinning and only after that opens the debug window.

3. I've said that in the starting-takes-forever state, opening new tabs and entering URLs leads to the new tabs staying blank and not loading. This was certainly the case, but I've tried it today and the problem didn't happen. So this behaviour varies. Today also STG finished spinning after a few minutes. Perhaps there's a difference cause between spinning-a-few-minutes behaviour and spinning-seemingly-forever behaviour.

Item 2 makes me think the slow or unterminating startups are not due to STG treating some tab states as unloaded and waiting on change events of some kind, because I'd expect low CPU in that case.

Thanks again!

[jlokier]

A bit more on debugging startup and CPU usage. While trying to debug the add-on while enabling, I enabled the add-on (necessary to be able to open the debug window unfortunately), then opened the debug window, then disabled the add-on, then enabled it again. At every attempt to re-enable, I consistently got this error notification:

Error: TransactionInactiveError: A request was placed against a transaction which is currently n...

Afterwards, the CPU is running at about 80%, about:performance shows the only significant energy user is the STG add-on ("Medium" energy), and "Error: wait loading addon" is reported to the console about once per second. I assume that message is from STG itself? Expanding those messages shows a stacktrace which looks like this:

Error: wait loading addon options.js:7:128902
moz-extension://3ef78140-02da-be40-b8cd-9f2625dcd770/options/options.js:7
n moz-extension://3ef78140-02da-be40-b8cd-9f2625dcd770/options/options.js:1
moz-extension://3ef78140-02da-be40-b8cd-9f2625dcd770/options/options.js:1
moz-extension://3ef78140-02da-be40-b8cd-9f2625dcd770/options/options.js:1
enter resource://devtools/server/actors/utils/event-loop.js:118
enter self-hosted:983
_pushThreadPause resource://devtools/server/actors/thread.js:183
onAttach resource://devtools/server/actors/thread.js:310
onAttach self-hosted:987
onPacket resource://devtools/server/main.js:1291
receiveMessage resource://devtools/shared/transport/child-transport.js:66
enter resource://devtools/server/actors/utils/event-loop.js:118
enter self-hosted:983

I don't know if this is helpful, but it shows a state in which STG can use CPU while not doing anything obviously useful, and it also suggests a callback sometimes tries to use IndexedDB incorrectly, causing a transaction error.

[jlokier]

I've just had to Force Quit in Firefox and restart This meant STG was still enabled when I restarted Firefox, which is different from most of my recent tested. After restarting, I clicked the session restore button.

The STG icon has been spinning now for about 16 minutes, still going, and it says "Waiting for session restore...", not the message about waiting for all tabs to load. The session is completely restored though. All visible tabs have finished loading, and other tabs should not be loading.

In other recent tests I've been starting Firefox with STG disabled (because it gets stuck on startup), then enabling it, and in those tests today it spins for a few minutes then stops.

I haven't kept very careful track of which thing I was doing, because I assumed session restore and add-on enable were behaving similarly, but perhaps that's a misake?

As a speculation, I wonder if the behaviour on session restore is different from when the add-on changes from disabled to enabled, and the session restore is getting stuck forever? (I'm going to speculate further that this might be due to the existence of "about:" tabs like "about:addons" in the session that is restored).

Here is a console log from STG debugging:

20:47:26.200 uncaught exception: undefined
20:47:46.085 Error: An unexpected error occurred undefined
20:54:17.510 Promise resolved after context unloaded hotkeys.js:1
s moz-extension://3ef78140-02da-be40-b8cd-9f2625dcd770/web/hotkeys.js:1

Note, the time 20:47:26 is about when Firefox was started or when I clicked restore session (both very close). The time 20:54:17 is when I opened the debug window. And I'm posting this as 21:17, when the STG icon is still spinning with "Waiting for session restore...", Firefox CPU is >100%, and there are no more console messages from STG.

[e-motiv]

Today, STG took 10 minutes!!! to get out "Waiting for all tabs to load"! No visible tabs were loading and I only have about 40 tabs in total (hidden or not).
And of course this is pretty annoying when in those 10 minutes you started using firefox with many tabs, because they are all closed at the end.
Why does it take 10 minutes?! This is not something rare. It ALWAYS takes minutes...

[e-motiv]

Duplicate of #329

[dpalic]

having exactly the same issue. will report it in #329

[Drive4ik]

@jlokier Please retest this issue on latest addon version. This bug should be fixed.

[e-motiv]

It still takes time for me. Maybe not 10 minutes like before, but still beyond comfortable levels.

Also, I'm not sure to open a new issue, but even if this is fixed, STG keeps putting tabs in "Hidden" tabs (probably when opening tabs before it's ready). Now, Firefox doesn't allow to close them all at once, so can you offer that option? On Mozilla, they say you might want to do this.
--> https://support.mozilla.org/en-US/questions/1279465

[Drive4ik]

@e-motiv Please, check (disable and retest) other addons. Some addons has conflict with STG.
In future I make a page that tell user about this conflicted addons

small parts of this addons:
Mate Translate - a long while change group, #533
SmartAdBlock - a long while change group, #393

[grahamperrin]

Yesterday I discovered the Archive feature. This allowed me to reduce my everyday session:

• from more than 1,400 tabs
• to fewer than 600

🎉

[nekohayo]

I don't know if it's necessarily STG being the bottleneck, or if it's Firefox itself? Because Firefox tends to choke if you try to (re)load too many tabs at once, though I guess that might only be the case when "importing" STG tabs (restoring from backup) rather than simply browser startup or tab group switching.

See https://www.reddit.com/r/firefox/comments/jgphir/why_is_firefox_blazing_fast_at_loading_individual/ for the general "multiple tabs loading at once" performance problem I'm referring to...

[grahamperrin]

… Firefox itself? …

Recently I used a copy of a ~1,400 tab session for extensive testing with minimal sets of extensions in connection with Mozilla bug 1694699. Not a performance-related bug, but it was a rare opportunity for me to tell the degree to which Firefox 86.0 startup times, with so large a session, are affected by Simple Tab Groups 4.7.

Startup times are slower with the extension, however the slowdowns were neither unexpected nor troublesome. Given the feature set of Simple Tab Groups, I was amazed at how little impact there was.

… when "importing" STG tabs (restoring from backup) …

Depending on the size of a session: a 'full' restoration can take longer than a startup because there's time taken to delete stuff before the restoration phase begins.

---

The post in Reddit:

Why is Firefox blazing fast at loading individual tabs, but slows to a crawl when loading many tabs quickly one after another? (with a fast PC and network)

I don't know, why does it crawl quickly? ;-)

Joking aside, yesterday at and under https://github.com/Drive4ik/simple-tab-groups/issues/671#issuecomment-787420339 I found myself actively loading broad arrays of tabs – for test purposes (I very rarely do this during everyday browsing).

In my tests, Firefox 86.0 default behaviour appeared to be: load one at a time. Maybe with a little overlap (I wasn't looking in that much detail). Like, walking each array. Many previous releases of Firefox might behave similarly, I can't recall this walk-like behaviour was a default when Firefox Quantum was released.

YMMV, depending on hardware etc; there's the automation of recommended performance settings in modern versions of Firefox.

Two extensions that may be of interest:

• Load Background Tabs Lazily
• tabPreLoader

[grahamperrin]

#379 (comment)

Please, check (disable and retest) other addons. …

@e-motiv please, will you find an opportunity to feed back?

[grahamperrin]

@jlokier in addition to improvements to Simple Tab Groups, there have been performance-related (and other) fixes to Firefox in the ~22 months since this issue was opened.

Do you still use the same Mac?

Please, do you still find startup times unreasonably slow? If so please let us have a system profile (use Apple's utility) or equivalent information …

… you might boot in live mode from release 0.4.0 or greater of helloSystem and use its Hardware Probe utility to upload a probe to https://bsd-hardware.info/

Thanks

[Sohimaster]

Very nice extension, but I am going to remove it.
It is completely impossible to wait 10 minutes every time firefox starts up.
(The waiting time does not depend on amount of opened tabs, you can wait 10 minutes if you have only 5)

firefox 87.0
uBlock origin, tree style tab and other extensions installed

I don't know what is the problem, but the fact it has not been solved so many years makes me very sad.

Anyway, thank you

[Drive4ik]

@Sohimaster It is very strange that STG takes 5 minutes or more to load. This should not be the case. If you want to help me - turn on DEBUG option in addon settings, restart browser, wait for STG to load, go to settings and turn off DEBUG. The addon will offer to save the log file. Here is this file, send me an email with a link to your comment on github, so that I don't have to look too much and immediately understand what it's all about.
Also send me a list of all installed addons.
Thank you.

[Sohimaster]

Ok, thanks
I'll try to do it and provide necessary info

[Sohimaster]

@Drive4ik sent you 2 letters with the corresponding logs

[Drive4ik]

@Sohimaster Please update the addon to the latest version, and if the problem repeats - send me the log file. Thanks

[Drive4ik]

Guys, if someone is still a long load addon, and he has the latest version of the addon, please send logs addon when you reload your browser.
Turn it on, reload your browser, wait for the addon to load, turn it off, save the log file and send it to me by e-mail.

[ReaLadge]

Found an issue which i've tried to find a solution for hours (for some reason it was just constantly loading for VERY long periods), and finally fixed it. If you have a high refresh rate monitor (above 144hz), it can cause some extensions to not function well, in this case at all. Here's a link on how to set firefox's frame rate to your desired monitor refresh rate and it should just work.

https://ubuntuhandbook.org/index.php/2021/01/change-firefox-frame-rate-high-refresh-rate-monitor/

[taoeffect]

@ReaLadge just wanted to say thanks so much for your comment!!!

That turned out to be my issue as well!

@Drive4ik - you might want to mention in the extension readme that users with high monitor refresh rates need to adjust the layout.frame_rate setting if they experience the extension getting stuck loading.

[nekohayo]

For others without the framerate cause, it may have something to do with this bug I freshly reported in Firefox: https://bugzilla.mozilla.org/show_bug.cgi?id=1756368 (not sure it affects STG on startup, but it absolutely does when you are restoring from backup)

[ReaLadge]

After a new update of firefox 99 they broke something with the layout.frame_rate with locked rates causing this extension and other stuff to not work on browser startup even if it's set to your monitor refresh rate. It does work temporarily if you go into it and change it to something else and revert it back to your desired refresh rate but the issue still occurs after restarting the browser.

Fix: Set the layout.frame_rate to 0 to make it an unlocked variable and it should work with browser restarts.

[taoeffect]

Thanks for letting us know @ReaLadge! It seems to be working still in 99.0 with your fix.

[ReaLadge]

I did have a little issue with it not working with it being 0 at one stage, but if you set it to -1 and switch it back to 0 it works again. Idk what they are doing lmao.

[shuhradk]

@nekohayo - I upgraded to STG 4.7.2.1 today on OSX 12.3.1 (my STG was at 3.2.1 prior to that). Firefox is version 99.0. Then I started to get the high CPU issue all of sudden/at the next restart. I have around 1500-2000 tabs in my STG.

After reading through the bugzilla notes by Giijs (https://bugzilla.mozilla.org/show_bug.cgi?id=1756368) I disabled the ublock origin extension and the CPU became normal again within 4-5 seconds. Since then I do see the Firefox CPU hit 100% at load up but for about 20-30 seconds as opposed to staying there endlessly and causing my fans to hit peak RPM. Prior to disabling ublock STG just kept spinning.

@Drive4ik - I was unable to get to the Manage extension page for STG as the CPU was pegged at 100% for Firefox while ublock was also enabled. Edit: I have shared debug logs over email after disabling ublock origin (1.42.4) as STG is still taking up to 30 seconds at 100% CPU.

[taoeffect]

OK, so, on my end with Firefox v100 I had to actually set layout.frame_rate back to -1 (the default) for it to work. I had it set to 0 per @ReaLadge's comment but it would prevent loading, oddly enough.

[ReaLadge]

OK, so, on my end with Firefox v100 I had to actually set layout.frame_rate back to -1 (the default) for it to work. I had it set to 0 per @ReaLadge's comment but it would prevent loading, oddly enough.

For me, it works most of the time these days but random days it fails to load. I just switch between -1 & 360 (my refresh rate) whenever it does. It's not as bad as it was a month or so when I had to change it every time I boot.

[okt-ivanm]

I've experienced the same issue recently.
Turned out that after a Firefox update I had browser.sessionstore.restore_on_demand option turned on in about:config.
Turing it to false solved the problem