Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extremely slow performance of generateAsync with 3.2.X JSZip versions #617

Closed
TestPolygon opened this issue Sep 5, 2019 · 9 comments · Fixed by #829
Closed

Extremely slow performance of generateAsync with 3.2.X JSZip versions #617

TestPolygon opened this issue Sep 5, 2019 · 9 comments · Fixed by #829

Comments

@TestPolygon
Copy link

TestPolygon commented Sep 5, 2019

generateAsync works ~50 times slower in JSZip 3.2.X than in JSZip 3.1.5!
I use JSZip in a Chromium 76 based browser.

Here is a code to test it:

function addBlobToZipMultipleTimes(blob, times) {
    
    console.log("Input blob.size: ", formatBytes(blob.size));

    const zip = new JSZip();
    
    for(let i = 0; i < times; i++) {
        zip.file("file_" + i, blob);
    }
    
    console.log("generateAsync...");
    console.time("generateAsync");
    zip.generateAsync({type:"blob"})
        .then(function(blob) {
            console.timeEnd("generateAsync");
            console.log("Result zip.size: ", formatBytes(blob.size));
            
            // downloadBlob(blob); // some work with the result zip
        });
}

// https://stackoverflow.com/a/18650828/11468937
function formatBytes(bytes, decimals = 2) {
    if (bytes === 0) return '0 Bytes';

    const k = 1024;
    const dm = decimals < 0 ? 0 : decimals;
    const sizes = ['Bytes', 'KB', 'MB', 'GB', 'TB', 'PB', 'EB', 'ZB', 'YB'];

    const i = Math.floor(Math.log(bytes) / Math.log(k));

    return parseFloat((bytes / Math.pow(k, i)).toFixed(dm)) + ' ' + sizes[i];
}

The example

For addBlobToZipMultipleTimes(blob, 1):
JSZIP 3.1.5:

Input blob.size: 2.75 MB
generateAsync...
generateAsync: 22.086181640625ms
Result zip.size: 2.75 MB

JSZIP 3.2.1:

Input blob.size: 2.75 MB
generateAsync...
generateAsync: 926.528076171875ms
Result zip.size: 2.75 MB

JSZIP 3.2.0, 3.2.2 have the same speed.

22 ms vs 926 ms!


Some more examples

JSZIP 3.1.5:

generateAsync: 97.862060546875ms
Result zip.size: 13.76 MB

JSZIP 3.2.1:

generateAsync: 4944.34423828125ms
Result zip.size: 13.76 M

97 ms vs 4944 ms


For 41.29 MB:

JSZIP 3.1.5:

287.92626953125ms

JSZIP 3.2.1:

14723.44287109375ms

287 ms vs 14723 ms


For 137.65 MB:

JSZIP 3.1.5:

858.16796875ms

JSZIP 3.2.1:

49275.451904296875ms

0.8 sec vs 49 sec!

@maximerety
Copy link

maximerety commented Sep 10, 2019

Hi @TestPolygon,

Based on your code, I added a live demo here: https://jsbin.com/kuxutodini/edit?html,console

From the output of your test, I assumed you were using a 2.75MB blob, added respectively 1, 10 and 50 times to a zip.

I managed to reproduce your performance issue (tested both Chrome 76 and Firefox 69), with version 3.2.5 taking from 10 to 35 times longer than 3.1.5 to generate a zipped output.

I hope this can help debugging the issue.

maximerety added a commit to maximerety/jszip that referenced this issue Sep 11, 2019
The naive implementation from the 'set-immediate-shim' package, based
on setTimeout(..., 0) is order of magnitude slower than the clever
ponyfill from core-js.

See Stuk#617
@tfcardoso
Copy link

tfcardoso commented Feb 21, 2020

I used this lib on a personal project today and was ready to give up when I found this issue. With 3.1.5 its quite fast generating one large zip that I need (+/- 10s) and in latest it takes above 10m and thats not an amount of time acceptable to download a file.

This to say that (in my opinion) this issue is a blocker and having a PR prepared, this should be top priority.

@caxton
Copy link

caxton commented Feb 23, 2020

I used this lib on a personal project today and was ready to give up when I found this issue. With 3.1.5 its quite fast generating one large zip that I need (+/- 10s) and in latest it takes above 10m and thats not an amount of time acceptable to download a file.

This to say that (in my opinion) this issue is a blocker and having a PR prepared, this should be top priority.

+1

@xqdoo00o
Copy link
Contributor

xqdoo00o commented Jul 21, 2020

Hi, @TestPolygon , I think the PR #702 may resolve this issue. Could you plz test with it ?

majia0123 pushed a commit to majia0123/Userscript that referenced this issue Aug 1, 2020
@TestPolygon
Copy link
Author

TestPolygon commented Aug 2, 2020

I can only check the first 2 commits. Using 64k block size is definitely the better thing, even ReadableStream uses such chunk size (or bigger).
And using the properly implemented setimmediate is the must have thing, it's the main reason of the performance issue.

I don't undetstand why the rep owner @Stuk ignores this issue. Maybe someone ping him on other platforms?

@Maxime-J
Copy link

Maxime-J commented Aug 14, 2020

Thank you @xqdoo00o !
Performance is way better with your commits
(went from more than 1 min to ~2 s for a huge archive)

@101arrowz
Copy link

101arrowz commented Nov 5, 2020

You might want to consider UZIP or fflate for higher performance since the maintainer of this repo seems to dropped support for it. Check #720 for more info.

funkyhippo added a commit to subject-f/guyamoe that referenced this issue Jan 23, 2021
Performance issues as documented here:
Stuk/jszip#617
kafkaien42 pushed a commit to kafkaien42/guyamoe that referenced this issue Aug 21, 2021
Performance issues as documented here:
Stuk/jszip#617
@klesun
Copy link

klesun commented May 15, 2022

Wow, just downgrading to 3.1.5 made it hundreds time faster...

@101arrowz
Copy link

101arrowz commented May 20, 2022

Looks like this is finally fixed in v3.10.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

8 participants