[Bug]: History import ends with error messages, successful but misses ~2 years

[TheDromundKaas]

Guidelines

• I have encountered this bug in the latest release of FreeTube.
• I have encountered this bug in the official downloads of FreeTube.
• I have searched the issue tracker for open and closed issues that are similar to the bug report I want to file, without success.
• I have searched the documentation for information that matches the description of the bug I want to file, without success.
• This issue contains only one bug.

Describe the bug

I have moved PCs, from one EndeavourOS linux PC to the next one. I have backed all Freetube settings (history, playlists, subscriptions), and while subscriptions and playlists could be imported successfully, during the history import the following messages occur:

Unknown data key: lastViewedPlaylistItemId
Unknown data key: lastViewedPlaylistItemId
Unknown data key: lastViewedPlaylistType
Unknown data key: lastViewedPlaylistItemId
All watched history has been successfully imported

I have exported twice, to make sure that there was no error there, and looking at the history file it looks ok. I see that the videos I watched yesterday are at the top of the history export file.

The import worked up to a point in 2022 specifically, all watched items after that are nonexistent.

I can supply the history file if needed, but optimally not with an open link in Github.

Expected Behavior

Go to Settings -> Data -> Import History, choose the freetube-history-2024-05-26.db file, import finishes successfully, videos watched up to yesterday at export time appear.

Issue Labels

data loss

FreeTube Version

v0.20.0 Beta

Operating System Version

EndeavourOS rolling (Linux pure2 6.9.1-arch1-2 #1 SMP PREEMPT_DYNAMIC Wed, 22 May 2024 13:47:07 +0000 x86_64 GNU/Linux)

Installation Method

Flathub

Primary API used

Local API

Last Known Working FreeTube Version (If Any)

v0.20.0 Beta

Additional Information

I've tested the import for the versions freetube (from the AUR), freetube-bin (from the AUR) and io.freetubeapp.FreeTube (from Flathub), all show the same import issue.

Nightly Build

• I have encountered this bug in the latest nightly build.

[TheDromundKaas]

I must apologize. The way I tested this problem was:

• Install Freetube
• Import all imports (subscriptions, playlists, history)
• Check history

I could see there that the last video was from about 2 years ago, and none of any current video are displayed. However, now, after watching a few videos and checking the history again, all imported history videos are displayed.

I am not sure what changed. Should I keep this report open, or do you want to close it?

[absidue]

Please keep it open.

It's a known issue that the history import shows the message about it being done way before it is actually finished and that inside the app it displays them in the wrong order right after the import (it's correct on disk so restarting the app fixes that order problem).

I'll assign this to myself, as I actually solved those issues months ago when I was improving the performance of the history import, I just haven't got round to opening a pull request for it yet (need to create custom builds so that the other maintainers can properly test before and after, need to take all the performance measurements and document them, I don't use the watch history myself so I've only be able to test it on small test cases I created, so I have no idea how well it actually performs with an actually large history).

[github-actions]

This issue is stale because it has been open 28 days with no activity. Remove stale label or comment or this will be closed in 7 days.