Thumbnail Improvements

[cal2195]

👍 Ready to go!

This PR will add many improvements to the thumbnail system:

• Change thumbnail id's to be hashes of the original files.
• Use a single thumbnail sheet rather than 10 individual thumbnail files.
• Allow resumable thumbnail generation. ( Depends on File Hashes #5 )
• Don't generate thumbnails if they already exist.
• Add backwards compatibility to previous versions. (?) See Implement Backwards Compatibility #10.
• Fix the demo content.

Some notes about this PR:

• I've removed (hopefully) all references to the jpg number system.
• It currently generates 10 thumbnails as before, and then combines them - this could possible be improved to reduce the number of read and writes.
• I'm unsure whether a compatibility script is worth it, as it will require reading and writing all the thumbnails again anyway - it would be slightly quicker, but not sure by how much and adds complexity. If not, backing up the old files by renaming the folder and library to .bak would be a good way to go.
• As predicted, the single thumbnail sheet works perfectly on the thumbnail view, but has awful performance on the filmstrip view! 😢 It appears to be related to the animations, so I've removed them from the filmstrip view - not as pretty, but at least it performs well!
• I'm going to test the performance on a larger number of files when I'm home.

Any and all comments accepted! 😄

[whyboris]

Fantastic!

Thank you for your work -- I'll try to review everything by Sunday (max 3 days).

[cal2195]

How does the demo content work/is it relevant? 😄

[whyboris]

Demo content was meant to be an online version of the app -- as a demo of how users can interact with it. This example from February 2018: http://dev.yboris.com/vh/

I am thinking showing an online demo to users might cause more confusion, so I'm not sharing it publicly (and the demo is of the app version 1.0.0). Feel free to ignore these files without any updating.

Ps - to get the demo to work, just run index.html once you have the built app in the dist folder 😉

[whyboris]

I'm loving so much about this!

One minor note - the new screenshots are much larger than the sum of the previous files. In one randomly-chosen example the previous 10 screenshots weighed in at 214kb, while the new one was 1,150kb (I checked the file, the JPEG quality was set to 100). I suspect we just need to pass in a lower jpg compression setting somehow 👍

I looked at merge-img but it doesn't seem to have any such options. It uses jimp which seems to have something related to quality but I don't know how to access it.

Perhaps, we'll have to import jimp as a dependency too, and then before you do img.write we could use something like this to set the quality.

[whyboris]

I love that both thumbnails and filmstrip works so well with the new system!

Could you please add outline: 3px solid black; to the .filmstrip-container? This will bring back the "filmstrip" feel to the preview (dark bars above and below)

[whyboris]

Compared to the master and angular7 branch, for some reason the App uses a lot more RAM (500mb+ compared to ~130mb) on startup.

[whyboris]

While some people have requested automatic updating of screenshots on start, I would prefer it would not be the default, and only available (if you think it's worth it) with a setting in the menu.

In part it's a personal preference, but in part it could cause some trouble: making the app unresponsive/slow while a user just wants to use it. There are likely other scenarios where auto-update is not preferred.

Do you think it would be possible to have this behavior disabled until a user enables it in the settings?

[whyboris]

I suspect it's possible to bypass writing the 10 images to disk, and then reading the 10 images from disk to create the single image.

merge-img interface seems to allow a buffer -- so perhaps we could keep the images in RAM and write once? That should make things faster (though unsure how much).

You said you've found a way to not use the fluent-ffmpeg which might mean we'll have a way to save the 10 screenshots to buffer rather than disk?

What are your thoughts on this?

[whyboris]

This PR is so good! You accomplished many of the things that were on my todo list and improved some things I didn't know I could improve! Thank you very much 👍

I'm eager to get angular7 merged in -- I tried your branch on top of angular7 and it ran great 🙌 Could you review #1 if you have time, and I'll merge it in once we're both happy with it. Then we could merge this PR on top (after a few minor changes).

I suspect there's some tasks the app running your branch performs on start-up that cause the RAM spike (over 1GB and CPU as high as 100%). Perhaps those can be commented out at least for now, or placed behind a setting that's off by default unless a user changes?

At work we sometimes add a feature toggle across code we disable, e.g.
// FEATURE_TOGGLE - DISABLE_AUTO_UPDATE
So that later when we're ready to enable it, we can find all instances to toggle quickly and easily.

Please let me know 🥇

[cal2195]

I'm loving so much about this!

One minor note - the new screenshots are much larger than the sum of the previous files. In one randomly-chosen example the previous 10 screenshots weighed in at 214kb, while the new one was 1,150kb (I checked the file, the JPEG quality was set to 100). I suspect we just need to pass in a lower jpg compression setting somehow 👍

I looked at merge-img but it doesn't seem to have any such options. It uses jimp which seems to have something related to quality but I don't know how to access it.

Perhaps, we'll have to import jimp as a dependency too, and then before you do img.write we could use something like this to set the quality.

This should be fixed now with the new ffmpeg - I'm getting 61kb files here, can you confirm?

[cal2195]

Compared to the master and angular7 branch, for some reason the App uses a lot more RAM (500mb+ compared to ~130mb) on startup.

Can you confirm if this is still an issue?

[cal2195]

While some people have requested automatic updating of screenshots on start, I would prefer it would not be the default, and only available (if you think it's worth it) with a setting in the menu.

In part it's a personal preference, but in part it could cause some trouble: making the app unresponsive/slow while a user just wants to use it. There are likely other scenarios where auto-update is not preferred.

Do you think it would be possible to have this behavior disabled until a user enables it in the settings?

I've opened #15 to track this! 👍

[cal2195]

I suspect it's possible to bypass writing the 10 images to disk, and then reading the 10 images from disk to create the single image.

merge-img interface seems to allow a buffer -- so perhaps we could keep the images in RAM and write once? That should make things faster (though unsure how much).

You said you've found a way to not use the fluent-ffmpeg which might mean we'll have a way to save the 10 screenshots to buffer rather than disk?

What are your thoughts on this?

Haha, thanks for this! 👍 This prompted me to go and do some more research into ffmpeg, and I now have both image and video thumbnails being extracted with a single command into a single file! 🎉

The image command has been added to this PR! Thanks again! 😄

[whyboris]

Amazing! I will have time after work on Friday (tomorrow) to try this out.

[cal2195]

Don't merge yet, optimising thumbnails!

[cal2195]

That last commit majorly improves thumbnail extraction speed! 👍
I am noticing an issue with mkv files though, where the same first frame is used for all the thumbnails... more investigation is needed!

[whyboris]

Confirming that your branch at this moment works on top of angular7 and is fast at extracting screenshots.

Confirming that mkv files don't get extracted properly: usually the entire file is a solid black fill. In one case, the file was 22MB (1min long) and the extractor picked up the same screenshot for every single one of the 10 preview screens.

I wonder if this could be solved with a newer version of ffmpeg?
kribblo/node-ffmpeg-installer#15 (comment)

[whyboris]

I wonder if subtitles have something to do with the mkv problems you're having.

The 1min-long video extracted the first screenshot, just before any subtitles kicked in. I haven't confirmed, but I think all the mkv files I tried this on (just a few in my case) - they all had subtitles!

maybe the -sn flag can help?

[cal2195]

Can you confirm this works for you too? 😄

[whyboris]

Fantastic! mkv works perfectly now!

Details: 43 videos imported in 1min 23sec! Average jpg weight is 420kb (at 300px height per screenshot).

App uses around 780mb of RAM instead of the usual ~150mb. I suspect it has to do with multiple loading of the images. I have ideas about getting that number down - can be done after this PR merges.

[whyboris]

If you have the time, could you spot-check the angular7 #1 PR - once you approve it I'll merge it in, and then we can merge this one on top. I expect only about 2 lines of merge conflicts then 👌

[cal2195]

In the middle of it right now! 😉

[cal2195]

App uses around 780mb of RAM instead of the usual ~150mb. I suspect it has to do with multiple loading of the images. I have ideas about getting that number down - can be done after this PR merges.

I'd assume this has to do with:

• on the thumbs view, the images are 10 times larger (each image loads all 10, rather than just loading when you hover over)
• on the film strip view, each strip is 10 images, each of which has all 10 screenshots - not sure if chrome reuses images here.

I'm sure there's some optimisation we could do, but merging first would be nice! 👍

[whyboris]

It looks like when running the app in development mode the RAM usage is higher. I built the app and installed it with your branch, and it looks to be around ~165MB of RAM -- sorry for the false alarm 😓

[cal2195]

Oh nice! 👍 Should be good to go then! 😄

[whyboris]

I'm happy to merge this PR once the merge conflicts have been resolved 😁

Once merged in, would you mind if I fix up some minor things (e.g. var -> let; let -> const; " -> '; etc -- in a few places) - it's a bit easier than going back and forth with requests 😅

[cal2195]

Rebased onto master! 👍

Feel free to fix those minor things before merging! I think you can push to this branch?

[whyboris]

Looks great!

I'll merge in and create a small PR for format changes - just things like var -> let, etc 👌