Yank older broken builds?

[cjdoris]

In short: Any Linux builds for pyarrow versions 6.0.1 and newer which do not have the dependency libstdcxx-ng >=12 should be yanked from Conda.

The following error occurs on Linux:

$ micromamba create -n test pyarrow 'libstdcxx-ng<11.4' -c conda-forge
.....
$ micromamba activate test
$ python
>>> import pyarrow
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/conda/envs/test/lib/python3.10/site-packages/pyarrow/__init__.py", line 65, in <module>
    import pyarrow.lib as _lib
ImportError: /opt/conda/envs/test/lib/python3.10/site-packages/pyarrow/../../../libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /opt/conda/envs/test/lib/python3.10/site-packages/pyarrow/../../../libarrow.so.800)

The problem is that there are some old builds of pyarrow which have an incorrect dependency on libstdcxx-ng. In this case, pyarrow=8.0.0=py310h225c066_1_cpu gets installed, which incorrectly depends on libstdcxx-ng >=10.3.0. Newer builds of the package correctly have libstdcxx-ng >=12.

The newest version of pyarrow in conda-forge which is actually compatible with libstdcxx-ng=11 appears to be 6.0.0 - insofar as the above error does not happen on import.

Original issue: JuliaPy/PythonCall.jl#247

[h-vetinari]

Thanks for the report. Since these builds have been around for quite a while, yanking isn't such a good idea. But the same effect can be achieved by a repodata patch, see https://github.com/conda-forge/conda-forge-repodata-patches-feedstock/

Could you open a PR?

[cjdoris]

Will do, thanks.

[jakirkham]

Please fill out the full template in the issue. In particular would like to see conda list

[cjdoris]

Will do, though it's easily reproducible with a wide range of dependencies, so I'm not sure if conda list will be that helpful. I have investigated the issue more and described it at conda-forge/conda-forge-repodata-patches-feedstock#359.

[cjdoris]

Here's the state of one particular conda env for which the error occurs.

Installed packages:

List of packages in environment: "/opt/conda/envs/test"
 
  Name                Version       Build                Channel    
──────────────────────────────────────────────────────────────────────
  _libgcc_mutex       0.1           conda_forge          conda-forge
  _openmp_mutex       4.5           2_gnu                conda-forge
  abseil-cpp          20210324.2    h9c3ff4c_0           conda-forge
  arrow-cpp           8.0.0         py310hf3556ec_1_cpu  conda-forge
  aws-c-cal           0.5.11        h95a6274_0           conda-forge
  aws-c-common        0.6.2         h7f98852_0           conda-forge
  aws-c-event-stream  0.2.7         h3541f99_13          conda-forge
  aws-c-io            0.10.5        hfb6a706_0           conda-forge
  aws-checksums       0.1.11        ha31a3da_7           conda-forge
  aws-sdk-cpp         1.8.186       hb4091e7_3           conda-forge
  bzip2               1.0.8         h7f98852_4           conda-forge
  c-ares              1.18.1        h7f98852_0           conda-forge
  ca-certificates     2022.9.24     ha878542_0           conda-forge
  gflags              2.2.2         he1b5a44_1004        conda-forge
  glog                0.6.0         h6f12383_0           conda-forge
  grpc-cpp            1.43.2        h9e046d8_3           conda-forge
  keyutils            1.6.1         h166bdaf_0           conda-forge
  krb5                1.19.3        h3790be6_0           conda-forge
  ld_impl_linux-64    2.39          hcc3a1bd_1           conda-forge
  libblas             3.9.0         16_linux64_openblas  conda-forge
  libbrotlicommon     1.0.9         h166bdaf_8           conda-forge
  libbrotlidec        1.0.9         h166bdaf_8           conda-forge
  libbrotlienc        1.0.9         h166bdaf_8           conda-forge
  libcblas            3.9.0         16_linux64_openblas  conda-forge
  libcrc32c           1.1.2         h9c3ff4c_0           conda-forge
  libcurl             7.86.0        h7bff187_1           conda-forge
  libedit             3.1.20191231  he28a2e2_2           conda-forge
  libev               4.33          h516909a_1           conda-forge
  libevent            2.1.10        h9b69904_4           conda-forge
  libffi              3.4.2         h7f98852_5           conda-forge
  libgcc-ng           12.2.0        h65d4601_19          conda-forge
  libgfortran-ng      12.2.0        h69a702a_19          conda-forge
  libgfortran5        12.2.0        h337968e_19          conda-forge
  libgomp             12.2.0        h65d4601_19          conda-forge
  libgoogle-cloud     1.36.0        h6945097_0           conda-forge
  liblapack           3.9.0         16_linux64_openblas  conda-forge
  libnghttp2          1.47.0        h727a467_0           conda-forge
  libnsl              2.0.0         h7f98852_0           conda-forge
  libopenblas         0.3.21        pthreads_h78a6416_3  conda-forge
  libprotobuf         3.19.4        h780b84a_0           conda-forge
  libsqlite           3.40.0        h753d276_0           conda-forge
  libssh2             1.10.0        haa6b8db_3           conda-forge
  libstdcxx-ng        11.3.0        h239ccf8_19          conda-forge
  libthrift           0.16.0        h519c5ea_1           conda-forge
  libutf8proc         2.8.0         h166bdaf_0           conda-forge
  libuuid             2.32.1        h7f98852_1000        conda-forge
  libzlib             1.2.13        h166bdaf_4           conda-forge
  lz4-c               1.9.3         h9c3ff4c_1           conda-forge
  ncurses             6.3           h27087fc_1           conda-forge
  numpy               1.22.3        py310h4ef5377_2      conda-forge
  openssl             1.1.1s        h166bdaf_0           conda-forge
  orc                 1.7.3         h1be678f_0           conda-forge
  parquet-cpp         1.5.1         2                    conda-forge
  pip                 22.3.1        pyhd8ed1ab_0         conda-forge
  pyarrow             8.0.0         py310h225c066_1_cpu  conda-forge
  python              3.10.8        h257c98d_0_cpython   conda-forge
  python_abi          3.10          3_cp310              conda-forge
  re2                 2022.02.01    h9c3ff4c_0           conda-forge
  readline            8.1.2         h0f457ee_0           conda-forge
  s2n                 1.0.10        h9b69904_0           conda-forge
  setuptools          65.5.1        pyhd8ed1ab_0         conda-forge
  snappy              1.1.9         hbd366e4_1           conda-forge
  tk                  8.6.12        h27826a3_0           conda-forge
  tzdata              2022f         h191b570_0           conda-forge
  wheel               0.38.4        pyhd8ed1ab_0         conda-forge
  xz                  5.2.6         h166bdaf_0           conda-forge
  zlib                1.2.13        h166bdaf_4           conda-forge
  zstd                1.5.2         h8a70e8d_1           conda-forge

Environment info:

                                           __
          __  ______ ___  ____ _____ ___  / /_  ____ _
         / / / / __ `__ \/ __ `/ __ `__ \/ __ \/ __ `/
        / /_/ / / / / / / /_/ / / / / / / /_/ / /_/ /
       / .___/_/ /_/ /_/\__,_/_/ /_/ /_/_.___/\__,_/
      /_/
 
 
            environment : test (active)
           env location : /opt/conda/envs/test
      user config files : /home/mambauser/.mambarc
 populated config files : 
       libmamba version : 1.0.0
     micromamba version : 1.0.0
           curl version : libcurl/7.86.0 OpenSSL/1.1.1s zlib/1.2.13 libssh2/1.10.0 nghttp2/1.47.0
     libarchive version : libarchive 3.6.1 zlib/1.2.13 bz2lib/1.0.8 libzstd/1.5.2
       virtual packages : __unix=0=0
                          __linux=5.15.49=0
                          __glibc=2.31=0
                          __archspec=1=x86_64
               channels : 
       base environment : /opt/conda
               platform : linux-64

[h-vetinari]

I'm sorry your repodata patch PR never got merged, that's not a good experience, especially as you did everything right.

Still, we're not supporting 6.0 arrow builds anymore (even about to drop 8.0 as soon as 12.0 gets migrated), so I'm closing this issue.