Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use webworker for stream copy #364

Closed
peaceps opened this issue Sep 7, 2022 · 5 comments
Closed

Use webworker for stream copy #364

peaceps opened this issue Sep 7, 2022 · 5 comments

Comments

@peaceps
Copy link

peaceps commented Sep 7, 2022

In zipjs I find that for not compressed entry it will never use webworker.
While in my case I also want to use webworker to do the copy.
Is it possible to add an option to support this? Thanks.

@gildas-lormeau
Copy link
Owner

Why do you want to use a web worker in such a case?

@peaceps
Copy link
Author

peaceps commented Sep 8, 2022

In my zip file there're lots of zip/gz/tar files inside, they will be copied in chrome main thread, which will cost much resource of main thread and will also be affected by user action(minimize chrome/switch to other tab)

@gildas-lormeau
Copy link
Owner

gildas-lormeau commented Sep 8, 2022

The problem is that web workers would do nothing. It would just help to slow down the extraction (thanks to message passing).

In zip.js, the main thread is always responsible for concatenating chunks. A way to solve this issue would be to pass the ReadableStream and the WritableStream to the web worker when extracting an entry from a zip file. Thus the data would be concatenated in the worker instead of the main thread. This requires some refactoring but it should be doable.

@gildas-lormeau
Copy link
Owner

gildas-lormeau commented Sep 8, 2022

I just pushed the version 2.6.24 which allows you to pass a new option when creating a new ZipReader instance or when calling ZipReader#getData(). This new option is named transferStreams and is set to true by default. When set to false, zip.js should work as before. When set to true, zip.js will try to transfer the ReadableStream and the WritableStream instances to the web workers. Thus, the decompression is entirely done in the web workers. Note that currently this feature is not supported in Deno.

@peaceps
Copy link
Author

peaceps commented Sep 9, 2022

Thanks so much!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants