cre cache: allow for disabling compression

[poire-z]

With ["cre_compress_cached_data"] = false, in settings.reader.lua.
Not using compression for cache files does indeed take more space, but it does speed up opening, rendering, page turns, and closing, with big documents.
Details in koreader/crengine#97 and originally in #3261 (comment).

[poire-z]

Mentionning that this setting cre_compress_cached_data does not bring much anymore. See koreader/crengine#97 (comment)

[suddenlyfleck]

I don't understand why ["cre_compress_cached_data"] = false, is not the default setting for - at least - the PocketBook devices:

I was regularly waiting for minutes for certain books to open. Now, any book I have opened at least once opens in a second or two. The trade-off? A little less free memory (I can still have reading material for the next decade or more with me).

When people take their e-readers into their hands, they want to read, not wait.

Please consider making this the default behavior!

[Frenzie]

If you have such a file, could you attach it after an ebookscrambler treatment?

[poire-z]

When people take their e-readers into their hands, they want to read, not wait.

They may also want their memory/SDCard to not die because of too many read/write :)

Last time I checked (5 years ago, above), there was no sensible difference in loading time.
We since switched from using zip or gzip to using zstd I think.
I also just checked now (on the emulator, would have to be checked on a device too):

Initial loading with no cache:
12/29/23-17:29:26 DEBUG   loading took 17.139 seconds
12/29/23-17:30:46 DEBUG   rendering took 80.110 seconds

Saving, with compression, cache created, 66 Mb:
66 102 272 Dec 29 16:45 test-issue11-Opening.epub.ff154f31.1.cr3

Reopening it:
12/29/23-17:27:53 DEBUG   loading took 2.089 seconds
12/29/23-17:27:53 DEBUG   rendering took 0.153 seconds

Cleaned up, re initial loading.

Saving, without compression, cache created, 187 Mb:
187 891 712 Dec 29 17:31 test-issue11-Opening.epub.ff154f31.1.cr3

Reopening it:
12/29/23-17:32:02 DEBUG   loading took 2.230 seconds
12/29/23-17:32:03 DEBUG   rendering took 0.153 seconds

I was regularly waiting for minutes for certain books to open

I find this hard to believe that you'd have to wait minutes with a compressed cache.
It would be nice if you could find one book that you could share with us, and re-do your own timing of opening (and look at the logs that you don't get for some reasons messages like "CRE... cache invalid"), so we get something tangible to look at, and not just feelings.

[poire-z]

Tested on my 8 years old Kobo Glo HD, with my reference big book (a Bible):

Initial loading with no cache:
12/29/23-20:50:42 INFO    loading took 21.826 seconds
12/29/23-20:50:55 INFO    rendering took 12.894 seconds
12/29/23-20:50:55 INFO    opening took 35.493 seconds

Saving, with compression, cache created, 16.4 Mb.

Reopening it:
12/29/23-20:51:19 INFO    loading took 2.723 seconds
12/29/23-20:51:19 INFO    rendering took 0.139 seconds
12/29/23-20:51:20 INFO    opening took 3.158 seconds

Cleaned up, re initial loading.

Saving, without compression, cache created, 64 Mb.

Reopening it:
12/29/23-20:53:34 INFO    loading took 2.709 seconds
12/29/23-20:53:35 INFO    rendering took 0.146 seconds
12/29/23-20:53:35 INFO    opening took 3.202 seconds

So, there again, no noticable difference - except for the file size.

[Frenzie]

I didn't do any tests right now but that maps to my prior experience on my H2O.

[suddenlyfleck]

I tried with multiple epub files: slow loads only happen the first time a book is opened. The difference between ["cre_compress_cached_data"] set to false or true is negligible.

Now I am not sure, why I remembered so distinctly, that the same book would take very long times to open. Maybe I was selecting certain memories after changing my e-reader?
Very recently, I updated KOReader from a 2021 version to the latest release - maybe the caching was done differently then?
Maybe I just did not understand that loading the book for the very first time is a different event then loading the book again after that - always forming and re-enforcing the memory how slow opening a book is, when doing so the very first time?

Anyway, I agree: the default settings are good as they are.

Let me leave with some more heartfelt praise for KOReader:
After my second last update of KOReader, the text-highlighting feature behaved so much better than what the stock-software was doing. I still remember my amazement, how KOReader continues to improve the UX with every release and that now it finally got better in all points than the stock reader software.

Now again after doing this upgrade, a pleasant surprise with this loading time issue (maybe unrelated to the update).

An awesome project! Thank you

[poire-z]

Good, thanks :)
You'll probably appreciate another recent feature then, "Partial rendering support", described under https://koreader.rocks/user_guide/#uitips.