Can this be used with Waterfox, Librewolf, Pale Moon etc?

[eater]

Ideally, can it merge histories from all of them? But at least, can I point ffexport toward those profiles to save separately in some way?

[seanbreckenridge]

Huh -- since those are all based on firefox, I assume they use the same sqlite format?

I would have to look into it/install them on my system to see if thats possible

[seanbreckenridge]

Most of what would have to be changed would be in this function, adding new cases for each browser and locating the database

[eater]

I am no coder but I've got all those browsers installed and I use them all intermittently, so let me know if I can help with testing etc.

[seanbreckenridge]

Hmm -- do you happen to know which ones out of these are the most popular?

Else looking up a list of firefox derivatives the list is quite endless...

Im thinking of adding a --path flag, to let you point it at a database in particular, so that thats always possible to back it up, even if I havent added a case for it, but would be nice to support the major ones by defualt

[eater]

Pale Moon is the most popular of those, last time I checked desktop browser market share. If you want to put in a --path flag, I can try that out and see if it works on the browsers I have before you code in default paths.

[seanbreckenridge]

Alright, seems feasible

have backups working for palemoon/waterfox

Was able to add waterfox and that matches the firefox database schema exactly
palemoon seems to have a different sqlite database structure, so will have to write an additional query

will try to work on this this weekend

[seanbreckenridge]

not done, but; as I've recently trying to find a way to merge my random bits of chrome history, thought I might as well take the oppurtunity to make this more extendable

As its such a radical change will probably just rename the repo/push a new name to pypi, as any library usage anyone was depending on would definitely break

[seanbreckenridge]

May not be perfect, but would appreciate a test to see if anything goes wrong on your end:

git clone 'https://github.com/seanbreckenridge/ffexport'
cd ffexport
git checkout -b v0.2 origin/v0.2
python3 -m pip install .
browserexport save --help
# probably run a couple save commands to see if those work
export BROWSEREXPORT_LOGS=5
browserexport save -b firefox -t .
browserexport save -b chrome -t .
browserexport save -b palemoon -t .
# run browserexport save --help for others, if you have those installed to test 

then those should all be able to merge into the common schema:

browserexport merge *.sqlite
# or to JSON
browserexport merge ./*sqlite --json | jq | less

(jq incase you dont have it installed, is not required, just pretty prints the JSON)

@eater

[seanbreckenridge]

Currently (at least partially) supports:

Options:
  -b, --browser [chrome|firefox|brave|waterfox|palemoon|chromium]

[seanbreckenridge]

Oh, and for librewolf, don't know where the file is, but gives you an oppurtunity to try the --path flag

I assume it'd be something like:

browserexport save -b firefox --path ./.librewolf/profiles/...default/places.sqlite

I assume its very close to the firefox schema, so it should detect/parse it

If its not able to, let me know

[eater]

WOW, it seems to work great. I am impressed, thanks!

I exported from Pale Moon, Firefox, Waterfox, Librewolf, and Vivaldi (using -b chromium --path ~/.config/vivaldi/Default/History). I tested the --profile flag and exported multiple Pale Moon profiles. Everything extremely smooth and fast.

Summary: removed 169662 duplicates...
Summary: returning 265393 visit entries...

Can I write the merged data to a new sqlite file that a browser can read, do you suppose?

[seanbreckenridge]

Cool -- I'll probably add vivaldi as that seems to be similar.

Can I write the merged data to a new sqlite file that a browser can read, do you suppose?

That is a relatively more complicated problem to solve, not something I'm sure I could tackle for every browser here, as they each have their own complicated structure

Seems that a lot of the chrome/firefox derivates end up following a similar enough structure for the visits, so adding something like palemoon is just a quick look at the database and then a couple dozen lines of code.

Converting that back to a database would probably double the amount of code here. May be a feature I could add in the future, but isn't a priority for me -- since the purpose here is to just make sure I don't lose any of my browser history, and I can programatically feed that to other sources.

There is some talk to integrate this with promnesia, so all the saved databases could be indexed by that, and then you get a search interface with all the dumped data like this...

I'll create an issue to track doing that, perhaps specically for Firefox/Chrome as those are the most popular...

[seanbreckenridge]

Also created #17, maybe that sort of solves this problem a bit, as mentioned in #16, there are some additional problems with polluting the schema if we try and fit the merged data back into the database...

[seanbreckenridge]

Changes have been merged into master, you can install this with

python3 -m pip install git+https://github.com/seanbreckenridge/browserexport

while I havent pushed an updated version to pypi, then you'll just be able to do

python3 -m pip install browserexport

Going to close this issue, feel free to comment on the other issues I've opened