Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initialize the libzim worker #934

Merged
merged 6 commits into from
Nov 27, 2022
Merged

Initialize the libzim worker #934

merged 6 commits into from
Nov 27, 2022

Conversation

Jaifroid
Copy link
Member

Fixes #929.

Currently there is no code to prevent initialization if the archive is split arbitrarily. I would prefer to be able to test what happens in this scenario.

@Jaifroid Jaifroid added this to the v3.7 milestone Nov 27, 2022
@Jaifroid Jaifroid self-assigned this Nov 27, 2022
@Jaifroid
Copy link
Member Author

Seems to initialize properly:

ZIM has fullTextIndex with listed size: 930615296
Instantiating libzim Web Worker...
Calling libzim WebWorker with parameters {action: 'init', files: Array(1)}action: "init"files: [File][[Prototype]]: Object
WebWorker called with action=init
baseZimFileName = wikipedia_es_all_maxi_2022-09.zim
Module["arguments"] = /work/wikipedia_es_all_maxi_2022-09.zim
runtime initialized
archive loaded libzim-wasm.js:2497
wasm initialized libzim-wasm.js:2497

@Jaifroid
Copy link
Member Author

Codefactor warning is on the library file. I've set it to ignore.

@Jaifroid
Copy link
Member Author

When loading a split ZIM with a full text index, we get:

WebWorker called with action=init
baseZimFileName = wikivoyage_en_all_maxi_2022-10.zim
Module["arguments"] = /work/wikivoyage_en_all_maxi_2022-10.zimaa,/work/wikivoyage_en_all_maxi_2022-10.zimab,/work/wikivoyage_en_all_maxi_2022-10.zimac,/work/wikivoyage_en_all_maxi_2022-10.zimad,/work/wikivoyage_en_all_maxi_2022-10.zimae,/work/wikivoyage_en_all_maxi_2022-10.zimaf,/work/wikivoyage_en_all_maxi_2022-10.zimag,/work/wikivoyage_en_all_maxi_2022-10.zimah,/work/wikivoyage_en_all_maxi_2022-10.zimai,/work/wikivoyage_en_all_maxi_2022-10.zimaj,/work/wikivoyage_en_all_maxi_2022-10.zimak,/work/wikivoyage_en_all_maxi_2022-10.zimal,/work/wikivoyage_en_all_maxi_2022-10.zimam,/work/wikivoyage_en_all_maxi_2022-10.ziman,/work/wikivoyage_en_all_maxi_2022-10.zimao
runtime initialized
***Uncaught (in promise) 5539920***

Clearly an exception is being thrown. It doesn't prevent the archive from loading and displaying, but if we can't fix this now, we should work around it by adding split ZIM detection.

@Jaifroid
Copy link
Member Author

We're getting a File System error with split ZIMs:

image

@Jaifroid
Copy link
Member Author

This is an error quite deep in the virtual file system code. I don't think it is in the scope of this PR. I'll add an issue for it on javascript-libzim, and we need to prevent libzim loading if we have a split archive here for now.

@Jaifroid Jaifroid merged commit dc450d7 into master Nov 27, 2022
@Jaifroid Jaifroid deleted the Initialize-the-libzim-worker branch November 27, 2022 14:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Initialize the libzim worker if specific criteria are met
1 participant