Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert merged data back into sqlite databases browsers can read #16

Open
seanbreckenridge opened this issue Apr 18, 2021 · 5 comments
Open
Labels
enhancement New feature or request low priority low priority issue

Comments

@seanbreckenridge
Copy link
Owner

seanbreckenridge commented Apr 18, 2021

As discussed here, is possible to take the merged data and convert it back into a new database to use, but supporting every browser would probably be a lot of code

Maybe support Chrome/Firefox, so at least you could merge data from browsers you use less into your 'main' (but this also promotes a silo, and pushes you towards firefox/chrome)

However,

  • I expect this would become a lot of code
  • you'd lose some data since this doesn't extract all attributes from every table, and we don't have some data, so we'd probably end up writing dummy values? Which then pollute the database even more for future exports...
  • re-normalizing all the data is an annoying task for every different schema.
  • the browser databases remove history over time anyways, so theres a limit, so you cant just merge millions of entries into your database to have history going back for years -- this is the whole reason browserexport exists, to save your old data.

Relatively low priority as I can interact with this programmatically by writing my own promnesia sources, but I can see this feature being useful for others...

Feels like a pretty difficult problem to solve, adds much more complexity to browserexport, when otherwise this is more functional -- taking databases, merging them and extracting visits

@seanbreckenridge seanbreckenridge added enhancement New feature or request help wanted Extra attention is needed low priority low priority issue labels Apr 18, 2021
@seanbreckenridge seanbreckenridge changed the title Convert merged data back into databases Convert merged data back into sqlite databases browsers can read Apr 18, 2021
@seanbreckenridge
Copy link
Owner Author

Perhaps an alternative is to have a nicer way to view the data? Currently you can dump it to JSON, and thats machine consumable, but end users of this don't have a great way to search this... could create a basic HTML file which searches the JSON using some javascript?

@seanbreckenridge
Copy link
Owner Author

seanbreckenridge commented Apr 18, 2021

considering the complications, tentatively consdering this wontfix unless someone wants to contribute changes

would probably go in another namespace package, at browserexport/reconstruct or something

feel free to comment here if this is something you're interested in doing

@eater
Copy link

eater commented Apr 19, 2021

For what it's worth, all the Mozilla family browsers have the ability to import a Chrome-format history file. So it would be useful to simply export in one format.

@seanbreckenridge
Copy link
Owner Author

seanbreckenridge commented Apr 19, 2021

Hm alright

I guess if this was to be implemented we would probably 'force' firefox (or chome?) as the base, and then once you have the merged database you can convert/import it into chrome...

I wonder if merged databases can somehow be marked so that the additional dummy data we're writing back isn't merged into 'real data' again by browserexport merge...

Not against this feature in general, it just adds more complexity and personally isn't a feature I need. I could understand how it would be useful for others though, just not something I'm going to invest days of time in currently

@eater
Copy link

eater commented Apr 20, 2021

As far as I can tell, not all Firefox-based browsers support importing Firefox format, but they all support Chrome format. (Don't ask me why.) That's why I suggested Chrome as the most acceptable export format.

Marking dummy data I'm not sure about. I imagine the browser's import tool does its own cleanup, if that matters.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request low priority low priority issue
Projects
None yet
Development

No branches or pull requests

2 participants