Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unzipping progress goes to 200% #5967

Closed
goodliff opened this issue Dec 16, 2019 · 15 comments · Fixed by #6009
Closed

Unzipping progress goes to 200% #5967

goodliff opened this issue Dec 16, 2019 · 15 comments · Fixed by #6009

Comments

@goodliff
Copy link

@goodliff goodliff commented Dec 16, 2019

Tried the last release (3.8.0) on docker to check the the unzipping change that was introduced by cli: unnzip Cypress using unzip utility on Linux #5851, when downloading and extracting cypress I noticed that the unzipping progress continued to 200% and time remaining went into negative figures.

Current behavior:

[14:09:00] Unzipping Cypress 200% -56s [title changed]

Desired behavior:

Ideally this would be accurate to the unzip progress.

Versions

Cypress package version: 3.8.0 | Cypress binary version: 3.8.0 | docker-compose version 1.24.1, build 4667896b

@GregLahaye

This comment has been minimized.

Copy link

@GregLahaye GregLahaye commented Dec 16, 2019

Experienced same issue with Cypress 3.8.0 on Windows 10 WSL Ubuntu

@jennifer-shehane

This comment has been minimized.

Copy link
Member

@jennifer-shehane jennifer-shehane commented Dec 17, 2019

😅 Yeah, that should not be happening. We have some tests around this, but no tests around the exact percentage display as far as I can tell.

Can you run the install with debug mode mode and print the entire set of logs here?

There are 2 pathways it could take for unzipping and this should tell us which pathway yours followed.

@GregLahaye

This comment has been minimized.

Copy link

@GregLahaye GregLahaye commented Dec 17, 2019

Here is debug log from Windows 10 WSL Ubuntu
https://pastebin.com/aqWyJ0aX

@goodliff

This comment has been minimized.

Copy link
Author

@goodliff goodliff commented Dec 17, 2019

Greg I'm a bit slow, did that take 14 minutes to unzip? I'm struggling to convince people to get it into our build cycles with a 4 minute (haven't checked the time since 3.8.0) unzip time. I'm assuming you're able to cache it between builds?

@GregLahaye

This comment has been minimized.

Copy link

@GregLahaye GregLahaye commented Dec 17, 2019

😄 yea it is a long time - this was on WSL (file I/O takes longer) and also yes - it will cache after first install so you should only need to install once (per version at least).

@flotwig

This comment has been minimized.

Copy link
Member

@flotwig flotwig commented Dec 17, 2019

@GregLahaye @goodliff It does still unzip correctly, right? Just the progress indicator is wrong?

@flotwig

This comment has been minimized.

Copy link
Member

@flotwig flotwig commented Dec 17, 2019

If you guys can do me a favor and send me the output of unzip -v - it would be helpful.

The progress indicator works basically by counting the number of lines output and dividing that by the total number of file in the archive - it assumes 1 line per file output. But maybe there is some version of unzip that has different stdout conventions.

My unzip:

UnZip 6.00 of 20 April 2009, by Debian. Original by Info-ZIP.

Latest sources and executables are at ftp://ftp.info-zip.org/pub/infozip/ ;
see ftp://ftp.info-zip.org/pub/infozip/UnZip.html for other sites.

Compiled with gcc 9.2.0 for Unix (Linux ELF).

UnZip special compilation options:
        ACORN_FTYPE_NFS
        COPYRIGHT_CLEAN (PKZIP 0.9x unreducing method not supported)
        SET_DIR_ATTRIB
        SYMLINKS (symbolic links supported, if RTL and file system permit)
        TIMESTAMP
        UNIXBACKUP
        USE_EF_UT_TIME
        USE_UNSHRINK (PKZIP/Zip 1.x unshrinking method supported)
        USE_DEFLATE64 (PKZIP 4.x Deflate64(tm) supported)
        UNICODE_SUPPORT [wide-chars, char coding: UTF-8] (handle UTF-8 paths)
        LARGE_FILE_SUPPORT (large files over 2 GiB supported)
        ZIP64_SUPPORT (archives using Zip64 for large files supported)
        USE_BZIP2 (PKZIP 4.6+, using bzip2 lib version 1.0.6, 6-Sept-2010)
        VMS_TEXT_CONV
        WILD_STOP_AT_DIR
        [decryption, version 2.11 of 05 Jan 2007]

UnZip and ZipInfo environment options:
           UNZIP:  [none]
        UNZIPOPT:  [none]
         ZIPINFO:  [none]
      ZIPINFOOPT:  [none]

And it looks fine:

unzip

@flotwig flotwig self-assigned this Dec 17, 2019
@goodliff

This comment has been minimized.

Copy link
Author

@goodliff goodliff commented Dec 17, 2019

Wow! Have you sped that up for demonstration purposes? :)

I'm just in the middle of something but when I'm free I'll try and get more information from docker (had to remove it from dependencies for the moment)

@flotwig

This comment has been minimized.

Copy link
Member

@flotwig flotwig commented Dec 17, 2019

Nope, it should be this fast if you have unzip installed! That was supposed to be an improvement in 3.8.0

re: unzip info - Sounds good - lemme know. It's also possible that your system doesn't have unzip, which could be causing the regression.

@goodliff

This comment has been minimized.

Copy link
Author

@goodliff goodliff commented Dec 17, 2019

Quite possibly, it's running via docker-compose at the moment and I've tried adding unzip to that. I think it may be defaulting to the npm unzip, I believe we have that as a dev-dep.

> cypress@3.8.0 postinstall /usr/src/app/node_modules/cypress
> node index.js --exec install

Installing Cypress (version: 3.8.0)

[16:41:16]  Downloading Cypress     [started]
[16:41:16]  Downloading Cypress      0% 0s [title changed]
[16:41:16]  Downloading Cypress      1% 20s [title changed]
[16:41:17]  Downloading Cypress      1% 30s [title changed]
[16:41:17]  Downloading Cypress      1% 40s [title changed]
...
[16:41:59]  Downloading Cypress      96% 2s [title changed]
[16:41:59]  Downloading Cypress      97% 1s [title changed]
[16:42:00]  Downloading Cypress      98% 1s [title changed]
[16:42:00]  Downloading Cypress      99% 0s [title changed]
[16:42:00]  Downloading Cypress      100% 0s [title changed]
[16:42:01]  Downloaded Cypress      [title changed]
[16:42:01]  Downloaded Cypress      [completed]
[16:42:01]  Unzipping Cypress       [started]
[16:42:02]  Unzipping Cypress        0% 0s [title changed]
[16:42:08]  Unzipping Cypress        1% 684s [title changed]
[16:42:08]  Unzipping Cypress        1% 685s [title changed]
[16:42:08]  Unzipping Cypress        1% 686s [title changed]
...
[16:43:33]  Unzipping Cypress        97% 3s [title changed]
[16:43:34]  Unzipping Cypress        98% 2s [title changed]
[16:43:35]  Unzipping Cypress        99% 1s [title changed]
[16:43:35]  Unzipping Cypress        100% 0s [title changed]
[16:43:36]  Unzipping Cypress        101% -1s [title changed]
[16:43:36]  Unzipping Cypress        102% -2s [title changed]
[16:43:38]  Unzipping Cypress        103% -3s [title changed]
[16:43:38]  Unzipping Cypress        104% -4s [title changed]
...
[16:44:38]  Unzipping Cypress        198% -78s [title changed]
[16:44:39]  Unzipping Cypress        199% -78s [title changed]
[16:44:39]  Unzipped Cypress        [title changed]
[16:44:39]  Unzipped Cypress        [completed]
[16:44:39]  Finishing Installation  [started]
[16:44:39]  Finished Installation   /root/.cache/Cypress/3.8.0 [title changed]
[16:44:39]  Finished Installation   /root/.cache/Cypress/3.8.0 [completed]

You can now open Cypress by running: node_modules/.bin/cypress open

https://on.cypress.io/installing-cypress

I've chopped large bits out of this to make it more readable, I'm still working on getting -v on unzip but I'll have to rebuild which will take some time.

Looking at docker-compose it doesn't mention unzip so I think you might be right.

Off topic, I'm trying to workout whether a PR from 8 months ago got released, where is the best place to ask or should I just raise a ticket?

@goodliff

This comment has been minimized.

Copy link
Author

@goodliff goodliff commented Dec 17, 2019

Yep, looks like you're right, I installed the unzip via apt before the installation and it completed in 11 secs instead of the few minutes without it. It also calculated the progress correctly.

[17:17:38]  Unzipping Cypress       [started]
[17:17:38]  Unzipping Cypress        0% 0s [title changed]
...
[17:17:48]  Unzipping Cypress        44% 12s [title changed]
[17:17:48]  Unzipping Cypress        45% 12s [title changed]
[17:17:48]  Unzipping Cypress        46% 11s [title changed]
[17:17:48]  Unzipping Cypress        47% 11s [title changed]
[17:17:48]  Unzipping Cypress        48% 11s [title changed]
[17:17:48]  Unzipping Cypress        49% 10s [title changed]
[17:17:48]  Unzipping Cypress        50% 10s [title changed]
[17:17:49]  Unzipping Cypress        51% 10s [title changed]
[17:17:49]  Unzipping Cypress        52% 9s [title changed]
[17:17:49]  Unzipping Cypress        52% 10s [title changed]
[17:17:49]  Unzipping Cypress        100% 0s [title changed]
[17:17:49]  Unzipped Cypress        [title changed]
[17:17:49]  Unzipped Cypress        [completed]
[17:17:49]  Finishing Installation  [started]
[17:17:49]  Finished Installation   /root/.cache/Cypress/3.8.0 [title changed]
[17:17:49]  Finished Installation   /root/.cache/Cypress/3.8.0 [completed]

You can now open Cypress by running: node_modules/.bin/cypress open
@GregLahaye

This comment has been minimized.

Copy link

@GregLahaye GregLahaye commented Dec 18, 2019

Same thing here - didn't have unzip installed so it seems like Cypress fell back to unzipping with Node.js:

2019-12-17T10:57:23.765Z cypress:cli:unzip unzipping from /tmp/cypress.zip
2019-12-17T10:57:23.766Z cypress:cli:unzip into /home/greg/.cache/Cypress/3.8.0
2019-12-17T10:57:23.785Z cypress:cli:unzip zipFile entries count 39363
2019-12-17T10:57:23.786Z cypress:cli:unzip unzipping via `unzip`
2019-12-17T10:57:23.868Z cypress:cli:unzip unzipping with node.js (slow)
2019-12-17T10:57:23.884Z cypress:cli:unzip `unzip` failed { code: -2 }
2019-12-17T10:57:23.884Z cypress:cli:unzip unzipping with node.js (slow)

Installing unzip through apt fixes the issue.

@goodliff

This comment has been minimized.

Copy link
Author

@goodliff goodliff commented Dec 19, 2019

Need anything else @flotwig?

@flotwig

This comment has been minimized.

Copy link
Member

@flotwig flotwig commented Dec 19, 2019

@goodliff Nope, we're good, thanks for the details so far. I've reproduced it locally and I have a PR open to fix it here: #6009

@cypress-bot

This comment has been minimized.

Copy link

@cypress-bot cypress-bot bot commented Dec 26, 2019

The code for this is done in cypress-io/cypress#6009, but has yet to be released.
We'll update this issue and reference the changelog when it's released.

@cypress-io cypress-io locked as resolved and limited conversation to collaborators Dec 30, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
4 participants
You can’t perform that action at this time.