Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
What is the problem?
Sometimes syncing 1 new bookmark kills 80% of my bookmarks on 3 perfectly synced machines!
How can the issue be reproduced?
I use xBrowserSync on 3 machines (see specs below) with Chrome (3x Version 78.0.3904.97) and Firefox, but 99% with Chrome.
I use the native Chrome Sync Profiles, but on each of the 3 machines I have disabled just the native bookmark sync part of Google Chrome for my primary profile.
I do not use Chrome Sync on mobile phones, I use Brave there.
I set up the bookmarks, create a new sync ID, and populate each of these machines (let's call them A, B and C) with approx. 2272 bookmarks.
I'm using "simple bookmark count" to keep track of the number/irregularities of my bookmarks.
Everything is going well for some weeks, my bookmarks have been expanded only very cautiously up to 2279 bookmarks on each of my machines - A, B and C - for several days.
I am using Chrome on these 3 devices on a daily basis and everything is in sync. :)
But then I add just 1 bookmark to Chrome on machine A - now I have 2280 bookmarks there.
Then I start Chrome on machine C and probably later on B. Suddenly my CPU usage raises up to 99% for 3 minutes on machine B while syncing. After that the count for B is still 2280.
But on machine C my number of bookmarks suddenly drops to 185.
The structure of folders and the sorting order of bookmarks is intact, but some folders are empty, some bookmarks are missing, completely random!
Then I clear everything, de-install the extension, populate the browser with new bookmarks, re-install the extension, create a new sync ID and start from scratch. This whole process I do repeat on each of my 3 machines.
Now I am in "phase VI" - I have already done this "complete cleanup" 5 times...
Sometimes I even don't dare to touch or alter my bookmarks just in fear of screwing everything up.
But then I could also replicate a bookmarks.html file from time to time and it would be safer.
The idea of a sync tool should be to sync.
I love xBrowserSync and backed it from the early days on Patreon after looking for an alternative to (fo)Xmarks.
I am also willing to help providing more insights getting this resolved.
Nevertheless: Keep up the good work, guys!
Hi @sky76, really sorry to hear you're having these issues, and many thanks for providing such a detailed issue report. From the log you posted everything looks fine until...
The log shows that the post-startup update check executes twice, which it shouldn't do. That leads to the event handlers being re-enabled after the first update sync (
I'm currently trying to finish v1.5.1 which is mainly an Android update, but I'll do some investigation and try to find the reason why the post-startup update check is sometimes executing twice.
Hi @nero120 - thank you for your reply.
Here I have added the sync logs from all 3 machines:
Machine3 is my MacBookPro, the other two machines are iMac 27" Retina models.
All Chrome's are identical with the exception of the MacBookPro, where I have a local extension installed called "Google PageSpeed Insights" which is now deprecated (the MacBookPro is from late 2012). Other than that these should be nearly identical.
The Chrome profile I am using is the main (first) profile (I have approx. 5 to 25 profiles on each Chrome browser for client stuff) and the native Chrome bookmark sync is explicitly disabled on these profiles.
Anyhow, I don't think the sync problem has to do with any of the extensions.
The bookmark count extension I have installed recently is also very silent - it only updates the number of bookmarks if you click on it.
I hope it won't be too much of a pain to find what's causing the problem.
For now I'll try not to touch anything regarding bookmarks in case I need to provide additional information.
On the MacBookPro I have disabled the sync of the xBrowserSync extension - there I have my 2000+ bookmarks (and of course I have a bookmark html-export at hand, just in case).
All my bookmarks are regular http(s)-only bookmarks, no bookmarklets or similar stuff.
I think I am having a similar issue. I sometimes lose big chunks of bookmarks for seemingly no reason. It happens every couple of weeks without any warning (no errors, no sign of issues). It actually just happened today where every other bookmark disappeared in one section of my bookmark list. I can't figure it out but I have a feeling that firefox is to blame somehow. I say this because I use chrome 99% of the time and it doesn't seem like I ever have this problem. I only open firefox if I need to use a different browser for whatever reason. Maybe its a coincidence but I feel like whenever I have this problem I have used firefox that day. I think the extension has trouble with bookmark folders or nested folders in firefox? I noticed after the first 200 bookmars in the JSON file the "id" field for each has the number "id": "84911111111111111111111111" with a varying amount of "1"s that can get very long. Is there a way to clean or rebuild the bookmark database without losing any of the tags, descriptions or titles I added?
My setup has the bookmark server self-hosted on my ubuntu-server 19.04, I only access it from my internal network with my 2 computers (a desktop and laptop running Windows 10), I use chrome 99% of the time and only use firefox once in a while. No sync profiles on any browser.
For now, what I do is backup my whole bookmark database via the chrome extension which outputs a JSON file. I do it at least every day and sometimes more frequently if I have added a lot of bookmarks. I then compare the file size of this backup to a prior saved backup that I know is good.
Please don't feel like I'm complaining because I absolutely love the xBrowserSync! I just felt like mentioning it on this thread because it sounded familiar. Thank you!
Just a quick note - the size of the JSON file might vary if your favicon data is saved there as well. When I export my bookmarks manually, the favicon data (BASE64 encoded or something like that) makes up for more than half of the file size. Don't know whether this gets exportet by your extension.
Anyhow - when I clear my bookmarks entirely(!) and import the backup HTML file, no "Bookmarks backup folder" is created. So I have instant restor of bookmarks. After 1-2 minutes also the favicons are "back" (this takes some time) - at least the ones that are still in the cache.
And no, my extensions does not solve the problem, but I have an eye on bookmark changes because I remember the last "highest" number of bookmarks.
Thanks @sky76 for the "Über Simple Bookmark Count" Chrome extension idea! This will definitely make it much easier for me to track the number of bookmarks so maybe I will be able to pinpoint the conditions that causes a loss of bookmarks for me.
The JSON backup doesn't have the favicon images, only the bookmark data so that's why I can use the file size as a good indicator of lost bookmarks. As a test, I restored the backup from yesterday (when I noticed this issue occurred again because the JSON backup file size was smaller) and used the bookmark counter extension that you recommended...it showed 503 bookmarks. I then restored the last known good backup and it showed 807 bookmarks. So 300 went missing at some point yesterday lol. It's nice to know the exact number that disappeared now using the counter extension. That's why I constantly backup my bookmarks using the extension's JSON file backup feature so I don't lose any.
Since I self-host the server, I decided what I am going to do is create a cron script to automatically backup the database on the server like every hour and keep a few days worth of the backups in case. I may also include a mechanism to check the size of the current to the last backup (or actual number of bookmarks if possible) and if it's smaller by a certain amount, save the good backup permanently with a timestamp so I can go back and fix it manually.
Unfortunately I've not managed to find a reproducable instance of the issue which makes it very hard to debug. It's definitely there as users are experiencing it, I can't say whether it's Firefox only where the issues occurs. Can you tell me if you had made any bookmark changes when the issues have occurred or when you've noticed the missing bookmarks? I'm wondering if it happens when checking for updates on browser startup or whether it's triggered by a bookmark change on the part of the user.
Ok that's definitely a bug so I'll look into that. The ids should all be unique and incremental, note that as xBrowserSync does not store added dates it uses the id to determine the order the bookmarks were added. There is no automatic way to reset the bookmark ids at the moment, so unfortunately it would be a manual clean up job, sorry. I'll look into adding an id clean up process when backing up for a future release.
Could you test something for me please? Could you disable your current sync, then back up your bookmarks and see if the new back up file still contains the broken ids?
Not at all, I'm just very sorry when users lose data on account of a bug and I really appreciate your input and help in getting to the bottom of this!
Very useful suggestion, thanks!!
Just to confirm, no the favicons are not exported/stored with the xBrowserSync back up file.
As bookmark data is encrypted client side, you'd need to write the decryption process into your script in order to get the raw JSON, then parse the JSON and count the number of bookmarks from there. If you'd like to attempt this then this issue might assist you.
@nero120 Thanks for the replies! I did the above test and yes it fixed the broken id's perfectly!! I wish I could just use that as my main bookmark sync but it doesn't keep the descriptions or tags.
I looked at my regular backup closer and found where the ids stop counting correctly, it stops at 213 (out of a total of 807)...And now I see what is happening, the next id is "8491" including the parentheses so its being used as a string instead of a number. That is why I see tons of 1's because the string "8491" is being incremented by concatenating a "1" as it goes down the list of the remaining 600 bookmarks lol. I have no idea why that's happening. I attached a screenshot of the JSON, showing when it changes over to "8491".
I can use the regular expression "8491.*" to find all the bad id numbers but is there something that I can replace it with that will fix my database? If I at least replace all 398 bad id numbers with an integer (instead of a string being there), even if its the same integer, will the program be able to fix the ids?
When creating a new sync it gets the bookmark data from the Chrome API, since Chrome does not support descriptions or tags it cannot keep them, so yes not the best solution to your issue.
If you can manually replace the broken string ids with unique integers that increment from the highest existing integer id then it will fix the issue once you restore the edited backup file.
Yeah, I figured a new sync gets its data directly from chrome which only has a url and title. To fix my bookmark sync I am going to write a python script that recursively traverses the new syncs JSON that I know has correct structure and id numbers, and then add the missing data like description and tags from the messed up sync. The script will match on the url field and output a new JSON file that correctly includes everything and lastly restore that JSON into the chrome extension. This way I have a sync where I know the id's and structure are correct but still have my descriptions and tags data.