Off-by-one bug in tab sidebar

[ssokolow]

Short description

I initially didn't notice when it happened, so I don't know how to reproduce it, but my tree has been mangled as follows:

1. Each group parent got stuck onto the end of the previous group.
2. Each first child became the parent for its group.
3. Nothing I've tried, including hitting F1 twice, disabling and re-enabling the extension, and disabling and re-enabling the cache restored the proper tree layout.

So this:

A
B (collapsed)
- B1
- B2
- B3
C (collapsed)
- C1
- C2
- C3

...became this:

A
B
B1 (collapsed)
- B2
- B3
- C
C1 (collapsed)
- C2
- C3

It seems to be specific to just one of the four Firefox windows I have open across my three monitors, but it's still 159 tabs I have to un-discard and reorganize, which makes this extremely frustrating.

Environment

• Platform (OS): Lubuntu Linux 14.04.5 LTS
• Version of Firefox: 63.0b2 (64-bit)
• Version (or revision) of Tree Style Tab: 2.5.3

[ssokolow]

Gah! Half-way through trying to put my tabs back together, it happened again and threw me back to square one!

I'm starting to think that, in the post-XUL era, it may simply be untenable to have large numbers of tabs open.

[ssokolow]

...and apparently more than an off-by-one. While I'm attempting to put things back together, I have top-level tabs vanishing from the sidebar entirely, requiring me to force a refresh to get them back.

I'm just going to move as much as possible into a manually-maintained HTML file until I have time to either rip out and rewrite TST's sidebar code to my own standards or write my own alternative to TST. (Probably the latter. I have plans for a Personal Information Manager with browser integration and it'd probably be most comfortable to just let tabs migrate to its unified datastore once they've been idle for long enough.)

[SXZ1]

I currently have 4 windows open with 1080, 1861, 2700 and 4365 tabs and TST has been rock solid, I haven't encountered any bugs for a long time. I think a key to success is to make sure TST doesn't interfere with anything related to tab opening or closing. In order to do that, just go to TST settings and configure them in such a way that the browser gets full control of new tabs behavior.

[ssokolow]

@SXZ1 Your advice is helpful in theory, but not in practice. There are a lot of options in there and only a tiny number of them have "Firefox's default" annotations.

[ssokolow]

Also, it still seems to be working fine in my other three Firefox windows and was working fine on this window until this problem spontaneously cropped up.

[ssokolow]

...and given the behaviour of the group tabs, it appears that the off-by-one error is some kind of mismatch in the sidebar's model which doesn't get resolved by disabling and re-enabling the cache.

(They still list their children correctly and they unexpectedly delete them if they think they're collapsed when the children are clearly visible as children of another node.)

[ssokolow]

Update: With this layout...

BParent
BChild1
- BChild2
- CParent

Closing BParent will remove BChild1 and BChild2 regardless of whether the expander on BChild1 is open or closed.

[Lej77]

I have observed off-by-one issues many times with my Firefox session in many previous version of Tree Style Tab. I usually reverted my session to a backup when that happened. If you haven't restarted Firefox after the issue appeared you might be able to use the backup Firefox made from when it was last started. It should be stored at FirefoxProfileFolder\sessionstore-backups\previous.jsonlz4. To use it simply close Firefox and copy it to FirefoxProfileFolder\sessionstore.jsonlz4 and then start Firefox again.

Profile folder - Firefox - MozillaZine Knowledge Base

[ssokolow]

By the time I noticed, I'd already done too much opening, closing, and organizing of tabs to be willing to revert, so I took a different tack.

I used Multiple Tab Handler's Markdown export to dump the tree of tabs, used pandoc to convert it to reStructuredText, fixed up TST's glitches by hand in gVim, and have just finished reproducing everything relevant to a read-only view except the favicons and expand/collapse in a stylesheet I've been testing using restview and Open in Sidebar.

Now that the immediate threat of losing or corrupting important groupings is gone, I'll go back to migrating my TiddlyWiki Classic setup and Greasemonkey 3.x userscripts in preparation for 52 ESR being sunsetted in a few days. (The pile of tabs which caused the problem were the tabs from the 52 ESR profile that I'd only just finished transferring over to my Quantum profile a couple of days before.)

Then I'll work on implementing the expand-collapse and some quick scripting to scrape favicons, then work on moving it all into a Django+SQLite-based backend that can serve as a dogfoodable seed that I can expedite the building of my PIM tool around. (I've already searched up a set of candidates for abstracting away the implementation of manually reorderable rows in Django ORM and I remember having a similar set of links for tree-structured data bookmarked so, hopefully, it shouldn't be too difficult to integrate the two in some form or other.)

(I still have three other browser windows, comprising a total of 273 tabs, which I don't want to revert if I can avoid it. They still appear to be working fine and they rely far less on complex groupings if they do start to misbehave.)

[irvinm]

@ssokolow can you comment if you are still having a problem or not? There have been a ton of changes with Firefox and TST since August 2018. If the issue is resolved, can you could this issue?

[ssokolow]

Unfortunately, I can't give any input. I haven't used TST in months because it was bogging down my browser too much.

(Most visibly, rebuilding the tab sidebar whenever an update spontaneously got downloaded and installed could render the browser unresponsive for so long that it was at least half a minute. I never timed it because I'd taken to going to the kitchen for a snack whenever it happened.)

[irvinm]

Ok, thanks for the feedback @ssokolow. There have been a ton of improvements to TST since last August and I would guess the performance issues you were seeing might be resolved or at least improved. If you wanted to try again we could revisit this, but if you don't want to I understand.

If you don't want to try, can you close this item?

[ssokolow]

I'll try to find some time within the next few days to test the performance with the current version and decide what to do then.

EDIT: Sorry that it's been almost two weeks without a response. Things got unpredictable here. I haven't forgotten this and I'll get to it as soon as possible.

[irvinm]

@ssokolow any new updates? Maybe let's close this out and re-open when you get a chance to retry?

[ssokolow]

I'll try to fit it in today or tomorrow and we can close if I don't get to it by then.

Over the last week, a lot of plans got messed up and this was one of them.

[ssokolow]

Bah. What's an alternative source for the latest AMO-signed release of TST? There's apparently some kind of bug or bad AMO-extension interaction that's causing AMO to claim that my Firefox is Firefox 60.0 and lock out the installation button.

[Fanolian]

Bah. What's an alternative source for the latest AMO-signed release of TST? There's apparently some kind of bug or bad AMO-extension interaction that's causing AMO to claim that my Firefox is Firefox 60.0 and lock out the installation button.

privacy.resistFingerprinting changes your useragent. Toggling it back to false and install extensions on AMO.

[ssokolow]

Yep. I'd forgotten about that.

It's now installed again, but it'll be a little while before I've manipulated the tree enough to see if the problem recurs. I never did manage to figure out exactly what sequence of steps triggers it.

(Though TST does still have that annoying problem where it makes the browser temporarily unresponsive on opening the sidebar. However, it seems to have dropped to about 10 seconds now. Maybe I should see if there's a way to force Firefox to defer updates to TST until I'm asleep.)

[ssokolow]

I'm not yet sure if this particular bug is still present, but data model corruption still occurs.

1. For reasons unknown to me, opening a bunch of tabs from an RSS feed in Thunderbird caused some to be children or even grandchildren of others.
2. When I dragged one in the tab bar, it and its grandchild came along (which I don't remember happening before).
3. When I dragged it back by grabbing it in the sidebar, its child and grandchild stayed at their old position in the tree, with their old indentation level.
4. Closing the top-level one then didn't close the child and grandchild that it had been moved away from.
5. I then somehow managed to wind up with said tabs being children of the tab below them in the sidebar.

While it would be rude and arrogant of me to assume that all this buggy complexity in TST's sidebar implementation is unnecessary, I do question whether whatever functionality that complexity enables is worth the pain.

If I can get my more pressing problems out of the way, I've been considering writing a cut-down competitor to TST which uses the sidebar's DOM as the authoritative model, relies on things like HTML5 drag-and-drop of subtrees for robust definition of what should be moved in what way when dragging and dropping, and resets the backend copy to match the DOM whenever it suspects something has gone awry.

(I've actually already prototyped out a partial implementation of the sidebar frontend itself using nested <ul> elements and <details>/<summary> for expand/collapse without JavaScript. The main hold-up has been my procrastinating making a full break from using a ReStructuredText dump of my tabs and restview as a rapid-prototyping backend.)

[piroor]

For reasons unknown to me, opening a bunch of tabs from an RSS feed in Thunderbird caused some to be children or even grandchildren of others.

TST tries to attach a newly opened tab to the previous active tab if they have same origin. This behavior is aimed to attach new tabs to the old active tab, when they are opened from the location bar. You can deactivate that on TST's options page: "New Tabs Behavior" => "When new tab with a website same to the current tab is opened from the location bar (or others)" => "Open as" => "an independent tab".

When I dragged one in the tab bar, it and its grandchild came along (which I don't remember happening before).

Did you write about the option?: "Tree Behavior" => "When a parent tab is moved or closed by someone from outside of the tree sidebar" => "Treat as a parent always (move or close all descendant tabs also together), whether the tree sidebar is visible or not". Then, the option "Treat as a parent by operations inside the tree sidebar. / Treat just as a solo tab, by operations outside the tree sidebar." will work as you expected.

When I dragged it back by grabbing it in the sidebar, its child and grandchild stayed at their old position in the tree, with their old indentation level.

I've confirmed a problem similar to this. After tabs are moved from a window to another, TST looks fails to render them in the sidebar of the destination window. (And after reopening of the sidebar, tree structure looks fixed, because they are re-fetched from the master data stored in the background page.) So I've filed it as #2316.

Closing the top-level one then didn't close the child and grandchild that it had been moved away from.

Sorry I couldn't reproduce this. There maybe more other conditions.

Anyway, if TST doesn't fit to your case anymore, there are some competitor addons: https://github.com/piroor/treestyletab/blob/master/README.md#similar-projects I hope that you can find out the best solution matching to your case.

[irvinm]

@ssokolow can you review and comment?

[ssokolow]

I'm still getting bugs like this, but I'm having trouble pinning down a reliable way to reproduce them.

[irvinm]

Ok, thanks for the update.

[irvinm]

@ssokolow any new updates? Or can you close this item?

[ssokolow]

I still get the occasional tree structure corruption, but I haven't been able to consistently replicate them and I don't think they're the same "off by one" bug that I opened this for.

Given that, should I keep this open or close it?

[irvinm]

@ssokolow my vote would be to close it. Even once it is closed, you can reopen or even just add comments to it if more information around how @piroor might be able to reproduce himself.

[ssokolow]

Understood. Closing.