Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test section raises BadZipFile #4842

Closed
2 tasks done
h-vetinari opened this issue Apr 6, 2023 · 10 comments
Closed
2 tasks done

test section raises BadZipFile #4842

h-vetinari opened this issue Apr 6, 2023 · 10 comments
Labels
source::community catch-all for issues filed by community members stale::closed [bot] closed after being marked as stale stale [bot] marked as stale due to inactivity type::bug describes erroneous operation, use severity::* to classify the type

Comments

@h-vetinari
Copy link
Contributor

h-vetinari commented Apr 6, 2023

Checklist

  • I added a descriptive title
  • I searched open reports and couldn't find a duplicate

What happened?

In conda-forge/google-cloud-cpp-feedstock#129, we see

START: D:\bld\win-64\google-cloud-cpp-2.9.0-hf2ff781_0.conda
Traceback (most recent call last):
  File "C:\Miniforge\lib\site-packages\conda_package_handling\streaming.py", line 35, in _extract
    stream = package_streaming.stream_conda_component(
  File "C:\Miniforge\lib\site-packages\conda_package_streaming\package_streaming.py", line 98, in stream_conda_component
    zf = zipfile.ZipFile(fileobj or filename)
  File "C:\Miniforge\lib\zipfile.py", line 1267, in __init__
    self._RealGetContents()
  File "C:\Miniforge\lib\zipfile.py", line 1334, in _RealGetContents
    raise BadZipFile("File is not a zip file")
zipfile.BadZipFile: File is not a zip file

The traceback points to code in the conda namespace, but I'm not sure which component is at fault, so raising an issue for the overall conda-build.

Conda Info

Conda-forge CI on windows 

Conda Config

Conda-forge CI

Conda list

Conda-forge CI

Additional Context

No response

@h-vetinari h-vetinari added the type::bug describes erroneous operation, use severity::* to classify the type label Apr 6, 2023
@dholth
Copy link
Contributor

dholth commented Apr 7, 2023

I see https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=685088&view=logs&j=8af117b0-dfa5-5d1c-b02e-a448dd16efdb&t=5255ce7d-8850-5a89-87a7-dc472af37343&l=14598 "Package at D:\bld\win-64\google-cloud-cpp-2.9.0-hf2ff781_0.conda did not contain valid index.json data. Please check the file and remove/redownload if necessary to obtain a valid package." which might be caused by the same issue. Some conda-package-streaming users report tricky garbage collection but conda-build, through conda-package-handling, should be using it correctly.

Does this happen every time? Where do we check the exact versions of conda-build etc. being used?

@dholth dholth closed this as completed Apr 7, 2023
@dholth dholth reopened this Apr 7, 2023
@dholth
Copy link
Contributor

dholth commented Apr 7, 2023

(closed by mistake)

@h-vetinari
Copy link
Contributor Author

Does this happen every time?

It happened 2 out of 2 times. I restarted it for a third run just now.

Where do we check the exact versions of conda-build etc. being used?

It's all in the logs, but here for your convenience:

From section "Install conda-build"
  Updating specs:

   - python=3.10
   - conda-build
   - conda
   - pip
   - boa
   - conda-forge-ci-setup=3
   - ca-certificates
   - certifi
   - openssl

  Package                                      Version  Build               Channel                 Size
----------------------------------------------------------------------------------------------------------
  Install:
----------------------------------------------------------------------------------------------------------

  + anaconda-client                             1.11.1  pyhd8ed1ab_0        conda-forge/noarch      69kB
  + anaconda-project                            0.11.1  pyhd8ed1ab_0        conda-forge/noarch     230kB
  + anyio                                        3.6.2  pyhd8ed1ab_0        conda-forge/noarch      85kB
  + attrs                                       22.2.0  pyh71513ae_0        conda-forge/noarch      54kB
  + backports                                      1.0  pyhd8ed1ab_3        conda-forge/noarch       6kB
  + backports.functools_lru_cache                1.6.4  pyhd8ed1ab_0        conda-forge/noarch       9kB
  + beautifulsoup4                              4.12.1  pyha770c72_0        conda-forge/noarch     114kB
  + boa                                         0.14.0  pyhd8ed1ab_4        conda-forge/noarch      63kB
  + chardet                                      5.1.0  py310h5588dad_0     conda-forge/win-64     261kB
  + click                                        8.1.3  win_pyhd8ed1ab_2    conda-forge/noarch      77kB
  + clyent                                       1.2.2  py_1                conda-forge/noarch      10kB
  + conda-build                                 3.24.0  py310h5588dad_0     conda-forge/win-64     570kB
  + conda-env                                    2.6.0  1                   conda-forge/noarch       2kB
  + conda-forge-ci-setup                        3.29.1  py310hcfc216e_100   conda-forge/win-64      78kB
  + conda-pack                                   0.7.0  pyh6c4a22f_0        conda-forge/noarch      30kB
  + dataclasses                                    0.8  pyhc8e2a94_3        conda-forge/noarch      10kB
  + defusedxml                                   0.7.1  pyhd8ed1ab_0        conda-forge/noarch      24kB
  + filelock                                    3.10.7  pyhd8ed1ab_0        conda-forge/noarch      14kB
  + freetype                                    2.12.1  h546665d_1          conda-forge/win-64     497kB
  + glob2                                          0.7  py_0                conda-forge/noarch      12kB
  + importlib-metadata                           6.1.0  pyha770c72_0        conda-forge/noarch      25kB
  + importlib_resources                         5.12.0  pyhd8ed1ab_0        conda-forge/noarch      31kB
  + jinja2                                       3.1.2  pyhd8ed1ab_1        conda-forge/noarch     101kB
  + joblib                                       1.2.0  pyhd8ed1ab_0        conda-forge/noarch     210kB
  + json5                                        0.9.5  pyh9f0ad1d_0        conda-forge/noarch      21kB
  + jsonschema                                  4.17.3  pyhd8ed1ab_0        conda-forge/noarch      70kB
  + jupyter_core                                 5.3.0  py310h5588dad_0     conda-forge/win-64     108kB
  + lcms2                                         2.15  h3e3b177_1          conda-forge/win-64     499kB
  + lerc                                         4.0.0  h63175ca_0          conda-forge/win-64     194kB
  + libdeflate                                    1.18  hcfcfb64_0          conda-forge/win-64     152kB
  + libjpeg-turbo                              2.1.5.1  hcfcfb64_0          conda-forge/win-64     688kB
  + liblief                                     0.12.3  h63175ca_0          conda-forge/win-64       5MB
  + libpng                                      1.6.39  h19919ed_0          conda-forge/win-64     344kB
  + libtiff                                      4.5.0  h6c8260b_6          conda-forge/win-64       1MB
  + libwebp-base                                 1.3.0  hcfcfb64_0          conda-forge/win-64     268kB
  + libxcb                                        1.13  hcd874cb_1004       conda-forge/win-64       1MB
  + m2-bash                                    4.3.042  5                   conda-forge/win-64       4MB
  + m2-ca-certificates                        20150426  103                 conda-forge/win-64     392kB
  + m2-coreutils                                  8.25  102                 conda-forge/win-64       5MB
  + m2-curl                                     7.47.1  3                   conda-forge/win-64     622kB
  + m2-db                                       5.3.28  3                   conda-forge/win-64      54kB
  + m2-expat                                     2.1.1  2                   conda-forge/win-64      19kB
  + m2-findutils                                 4.6.0  2                   conda-forge/win-64     625kB
  + m2-gcc-libs                                  5.3.0  4                   conda-forge/win-64       1MB
  + m2-gdbm                                       1.11  4                   conda-forge/win-64     139kB
  + m2-git                                       2.8.1  2                   conda-forge/win-64     104MB
  + m2-gmp                                       6.1.0  3                   conda-forge/win-64     419kB
  + m2-gzip                                        1.7  2                   conda-forge/win-64     106kB
  + m2-heimdal                                   1.5.3  10                  conda-forge/win-64     706kB
  + m2-heimdal-libs                              1.5.3  10                  conda-forge/win-64     778kB
  + m2-icu                                        56.1  2                   conda-forge/win-64      11MB
  + m2-info                                        6.0  2                   conda-forge/win-64     244kB
  + m2-less                                        481  2                   conda-forge/win-64     109kB
  + m2-libcrypt                                    1.3  2                   conda-forge/win-64       8kB
  + m2-libcurl                                  7.47.1  3                   conda-forge/win-64     206kB
  + m2-libdb                                    5.3.28  3                   conda-forge/win-64       1MB
  + m2-libedit                                     3.1  20150326            conda-forge/win-64     104kB
  + m2-libexpat                                  2.1.1  2                   conda-forge/win-64      63kB
  + m2-libffi                                    3.2.1  2                   conda-forge/win-64      30kB
  + m2-libgdbm                                    1.11  4                   conda-forge/win-64      28kB
  + m2-libiconv                                   1.14  3                   conda-forge/win-64     735kB
  + m2-libidn                                     1.32  2                   conda-forge/win-64     225kB
  + m2-libintl                                  0.19.7  4                   conda-forge/win-64      24kB
  + m2-libmetalink                               0.1.2  3                   conda-forge/win-64      29kB
  + m2-libopenssl                              1.0.2.g  2                   conda-forge/win-64       1MB
  + m2-libp11-kit                               0.23.2  2                   conda-forge/win-64      98kB
  + m2-libpcre                                    8.38  2                   conda-forge/win-64     102kB
  + m2-libreadline                             6.3.008  8                   conda-forge/win-64     283kB
  + m2-libsqlite                              3.10.0.0  2                   conda-forge/win-64     475kB
  + m2-libssh2                                   1.6.0  2                   conda-forge/win-64     175kB
  + m2-libtasn1                                    4.7  2                   conda-forge/win-64     106kB
  + m2-msys2-runtime               2.5.0.17080.65c939c  3                   conda-forge/win-64       3MB
  + m2-ncurses                            6.0.20160220  2                   conda-forge/win-64       2MB
  + m2-openssh                                   7.1p2  2                   conda-forge/win-64       2MB
  + m2-openssl                                 1.0.2.g  2                   conda-forge/win-64       2MB
  + m2-p11-kit                                  0.23.2  2                   conda-forge/win-64     128kB
  + m2-patch                                     2.7.5  2                   conda-forge/win-64      92kB
  + m2-perl                                     5.22.1  2                   conda-forge/win-64      16MB
  + m2-perl-authen-sasl                           2.16  3                   conda-forge/win-64      50kB
  + m2-perl-convert-binhex                       1.123  3                   conda-forge/win-64      36kB
  + m2-perl-encode-locale                         1.04  2                   conda-forge/win-64      14kB
  + m2-perl-error                              0.17024  2                   conda-forge/win-64      22kB
  + m2-perl-file-listing                          6.04  3                   conda-forge/win-64      12kB
  + m2-perl-html-parser                           3.71  4                   conda-forge/win-64      89kB
  + m2-perl-html-tagset                           3.20  3                   conda-forge/win-64      15kB
  + m2-perl-http-cookies                          6.01  3                   conda-forge/win-64      26kB
  + m2-perl-http-daemon                           6.01  3                   conda-forge/win-64      19kB
  + m2-perl-http-date                             6.02  3                   conda-forge/win-64      13kB
  + m2-perl-http-message                          6.06  3                   conda-forge/win-64      79kB
  + m2-perl-http-negotiate                        6.01  3                   conda-forge/win-64      16kB
  + m2-perl-io-socket-ssl                        2.016  2                   conda-forge/win-64     131kB
  + m2-perl-io-stringy                           2.111  2                   conda-forge/win-64      60kB
  + m2-perl-libwww                                6.13  2                   conda-forge/win-64     130kB
  + m2-perl-lwp-mediatypes                        6.02  3                   conda-forge/win-64      24kB
  + m2-perl-mailtools                             2.14  2                   conda-forge/win-64      65kB
  + m2-perl-mime-tools                           5.506  2                   conda-forge/win-64     191kB
  + m2-perl-net-http                              6.09  2                   conda-forge/win-64      25kB
  + m2-perl-net-smtp-ssl                          1.02  2                   conda-forge/win-64       7kB
  + m2-perl-net-ssleay                            1.72  2                   conda-forge/win-64     248kB
  + m2-perl-termreadkey                           2.33  2                   conda-forge/win-64      28kB
  + m2-perl-timedate                              2.30  3                   conda-forge/win-64      43kB
  + m2-perl-uri                                   1.68  2                   conda-forge/win-64      82kB
  + m2-perl-www-robotrules                        6.02  3                   conda-forge/win-64      17kB
  + m2-sed                                       4.2.2  3                   conda-forge/win-64     908kB
  + m2-vim                                    7.4.1721  2                   conda-forge/win-64      14MB
  + m2-zlib                                      1.2.8  4                   conda-forge/win-64      53kB
  + m2w64-gcc-libgfortran                        5.3.0  6                   conda-forge/win-64     351kB
  + m2w64-gcc-libs                               5.3.0  7                   conda-forge/win-64     532kB
  + m2w64-gcc-libs-core                          5.3.0  7                   conda-forge/win-64     219kB
  + m2w64-gmp                                    6.1.0  2                   conda-forge/win-64     744kB
  + m2w64-libwinpthread-git         5.0.0.4634.697f757  2                   conda-forge/win-64      32kB
  + markdown-it-py                               2.2.0  pyhd8ed1ab_0        conda-forge/noarch      62kB
  + markupsafe                                   2.1.2  py310h8d17308_0     conda-forge/win-64      26kB
  + mdurl                                        0.1.0  pyhd8ed1ab_0        conda-forge/noarch      14kB
  + msys2-conda-epoch                         20160418  1                   conda-forge/win-64       3kB
  + nbformat                                     5.8.0  pyhd8ed1ab_0        conda-forge/noarch     101kB
  + openjpeg                                     2.5.0  ha2aaf27_2          conda-forge/win-64     237kB
  + packaging                                     23.0  pyhd8ed1ab_0        conda-forge/noarch      41kB
  + pillow                                       9.5.0  py310h3dcae36_0     conda-forge/win-64      46MB
  + pkginfo                                      1.9.6  pyhd8ed1ab_0        conda-forge/noarch      28kB
  + pkgutil-resolve-name                        1.3.10  pyhd8ed1ab_0        conda-forge/noarch       9kB
  + platformdirs                                 3.2.0  pyhd8ed1ab_0        conda-forge/noarch      18kB
  + prompt-toolkit                              3.0.38  pyha770c72_0        conda-forge/noarch     269kB
  + prompt_toolkit                              3.0.38  hd8ed1ab_0          conda-forge/noarch       6kB
  + psutil                                       5.9.4  py310h8d17308_0     conda-forge/win-64     383kB
  + pthread-stubs                                  0.4  hcd874cb_1001       conda-forge/win-64       6kB
  + py-lief                                     0.12.3  py310h00ffb61_0     conda-forge/win-64       2MB
  + pygments                                    2.14.0  pyhd8ed1ab_0        conda-forge/noarch     824kB
  + pyrsistent                                  0.19.3  py310h8d17308_0     conda-forge/win-64      92kB
  + python-dateutil                              2.8.2  pyhd8ed1ab_0        conda-forge/noarch     246kB
  + python-fastjsonschema                       2.16.3  pyhd8ed1ab_0        conda-forge/noarch     225kB
  + python-libarchive-c                            4.0  py310h5588dad_2     conda-forge/win-64      36kB
  + pytz                                        2023.3  pyhd8ed1ab_0        conda-forge/noarch     187kB
  + pywin32                                        304  py310h00ffb61_2     conda-forge/win-64       9MB
  + pyyaml                                         6.0  py310h8d17308_5     conda-forge/win-64     151kB
  + rich                                        13.3.3  pyhd8ed1ab_0        conda-forge/noarch     183kB
  + ripgrep                                     13.0.0  h7f3b576_2          conda-forge/win-64       2MB
  + ruamel_yaml                                0.15.80  py310h8d17308_1008  conda-forge/win-64     278kB
  + shyaml                                       0.6.2  pyhd3deb0d_0        conda-forge/noarch      22kB
  + six                                         1.16.0  pyh6c4a22f_0        conda-forge/noarch      14kB
  + sniffio                                      1.3.0  pyhd8ed1ab_0        conda-forge/noarch      14kB
  + soupsieve                              2.3.2.post1  pyhd8ed1ab_0        conda-forge/noarch      35kB
  + tomli                                        2.0.1  pyhd8ed1ab_0        conda-forge/noarch      16kB
  + tornado                                        6.2  py310h8d17308_1     conda-forge/win-64     683kB
  + traitlets                                    5.9.0  pyhd8ed1ab_0        conda-forge/noarch      98kB
  + typing-extensions                            4.5.0  hd8ed1ab_0          conda-forge/noarch      10kB
  + typing_extensions                            4.5.0  pyha770c72_0        conda-forge/noarch      31kB
  + watchgod                                     0.8.2  pyhd8ed1ab_0        conda-forge/noarch      15kB
  + wcwidth                                      0.2.6  pyhd8ed1ab_0        conda-forge/noarch      29kB
  + xorg-libxau                                  1.0.9  hcd874cb_0          conda-forge/win-64      58kB
  + xorg-libxdmcp                                1.1.3  hcd874cb_0          conda-forge/win-64      68kB
  + yaml                                         0.2.5  h8ffe710_2          conda-forge/win-64      63kB
  + zipp                                        3.15.0  pyhd8ed1ab_0        conda-forge/noarch      17kB
host env
## Package Plan ##

  environment location: D:\bld\google-cloud-cpp-split_1680783055022\_h_env


The following NEW packages will be INSTALLED:

    c-ares:          1.18.1-h8ffe710_0           conda-forge
    ca-certificates: 2022.12.7-h5b45459_0        conda-forge
    krb5:            1.20.1-heb0366b_0           conda-forge
    libabseil:       20230125.0-cxx17_h63175ca_1 conda-forge
    libcrc32c:       1.1.2-h0e60522_0            conda-forge
    libcurl:         7.88.1-h68f0423_1           conda-forge
    libgrpc:         1.52.1-h32da247_1           conda-forge
    libprotobuf:     3.21.12-h12be248_0          conda-forge
    libssh2:         1.10.0-h9a1e1f7_3           conda-forge
    libzlib:         1.2.13-hcfcfb64_4           conda-forge
    nlohmann_json:   3.11.2-h39d44d4_0           conda-forge
    openssl:         3.1.0-hcfcfb64_0            conda-forge
    re2:             2023.02.02-h63175ca_0       conda-forge
    ucrt:            10.0.22621.0-h57928b3_0     conda-forge
    vc:              14.3-hb6edc58_10            conda-forge
    vs2015_runtime:  14.34.31931-h4c5c07a_10     conda-forge
    zlib:            1.2.13-hcfcfb64_4           conda-forge
build env
## Package Plan ##

  environment location: D:\bld\google-cloud-cpp-split_1680783055022\_build_env


The following NEW packages will be INSTALLED:

    c-ares:          1.18.1-h8ffe710_0           conda-forge
    ca-certificates: 2022.12.7-h5b45459_0        conda-forge
    cmake:           3.26.3-h1537add_0           conda-forge
    libabseil:       20230125.0-cxx17_h63175ca_1 conda-forge
    libgrpc:         1.52.1-h32da247_1           conda-forge
    libprotobuf:     3.21.12-h12be248_0          conda-forge
    libzlib:         1.2.13-hcfcfb64_4           conda-forge
    ninja:           1.11.1-h91493d7_0           conda-forge
    openssl:         3.1.0-hcfcfb64_0            conda-forge
    re2:             2023.02.02-h63175ca_0       conda-forge
    ucrt:            10.0.22621.0-h57928b3_0     conda-forge
    vc:              14.3-hb6edc58_10            conda-forge
    vs2015_runtime:  14.34.31931-h4c5c07a_10     conda-forge
    vs2019_win-64:   19.29.30139-hb9aee9d_10     conda-forge
    vswhere:         3.1.1-h57928b3_0            conda-forge
    zlib:            1.2.13-hcfcfb64_4           conda-forge

@dholth
Copy link
Contributor

dholth commented Apr 7, 2023

Assuming the newest conda, conda-build and conda-package-streaming, zstd, (python-)zstandard

@coryan
Copy link

coryan commented Apr 17, 2023

Any thoughts? Can we add any additional logging to troubleshoot this issue?

@h-vetinari
Copy link
Contributor Author

Still running into this with conda-forge/google-cloud-cpp-feedstock#130. Is there any version of conda, conda-build, etc. that changed something about this recently, or to which we can downgrade in the meantime?

@h-vetinari
Copy link
Contributor Author

Not sure if something changed, but conda-forge/google-cloud-cpp-feedstock#132 just passed. :)

@travishathaway travishathaway added the source::community catch-all for issues filed by community members label May 8, 2023
@travishathaway
Copy link
Contributor

travishathaway commented May 8, 2023

Should this be moved to https://github.com/conda/conda-package-streaming ?

Seems like an issue with not appropriately handling an exception raised by attempting to open a non-zip file 🤷‍♂️. We could easily transfer this issue over there.

@dholth
Copy link
Contributor

dholth commented May 8, 2023

No @travishathaway some part of the code attempted to create a zip file (a .conda file) and then couldn't extract it again. The file might have been empty or not flushed/closed properly before attempting to "open it again". The error is in the creation of the file not the opening of what was probably an empty file.

Copy link

github-actions bot commented May 8, 2024

Hi there, thank you for your contribution!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed automatically if no further activity occurs.

If you would like this issue to remain open please:

  1. Verify that you can still reproduce the issue at hand
  2. Comment that the issue is still reproducible and include:
    - What OS and version you reproduced the issue on
    - What steps you followed to reproduce the issue

NOTE: If this issue was closed prematurely, please leave a comment.

Thanks!

@github-actions github-actions bot added the stale [bot] marked as stale due to inactivity label May 8, 2024
@github-actions github-actions bot added the stale::closed [bot] closed after being marked as stale label Jun 7, 2024
@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Jun 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
source::community catch-all for issues filed by community members stale::closed [bot] closed after being marked as stale stale [bot] marked as stale due to inactivity type::bug describes erroneous operation, use severity::* to classify the type
Projects
Archived in project
Development

No branches or pull requests

4 participants