Unexpected `binary is None` in `liefldd.py` #4787

h-vetinari · 2023-02-26T06:10:19Z

Checklist

I added a descriptive title
I searched open reports and couldn't find a duplicate

What happened?

With the last conda-smithy release, conda-forge is now back to using lief 0.12.3, and failing in conda-forge/compiler-rt-feedstock#61 with:

  File "/Users/runner/miniforge3/lib/python3.10/site-packages/conda_build/os_utils/liefldd.py", line 406, in inspect_linkages_lief
    if binary.format == lief.EXE_FORMATS.ELF:
AttributeError: 'NoneType' object has no attribute 'format'

Conda Info

conda-forge CI for osx-64 / osx-arm64

Conda Config

conda-forge CI for osx-64 / osx-arm64

Conda list

host env:

environment location: /Users/runner/miniforge3/conda-bld/compiler-rt-packages_1677355103841/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol


The following NEW packages will be INSTALLED:

    clang:           16.0.0.rc3-h9d310f0_0         conda-forge/label/llvm_rc
    clang-16:        16.0.0.rc3-default_h5b99a12_0 conda-forge/label/llvm_rc
    clang-format:    16.0.0.rc3-default_h5b99a12_0 conda-forge/label/llvm_rc
    clang-format-16: 16.0.0.rc3-default_h5b99a12_0 conda-forge/label/llvm_rc
    clang-tools:     16.0.0.rc3-default_h5b99a12_0 conda-forge/label/llvm_rc
    clangdev:        16.0.0.rc3-default_h5b99a12_0 conda-forge/label/llvm_rc
    clangxx:         16.0.0.rc3-default_h5b99a12_0 conda-forge/label/llvm_rc
    icu:             70.1-h96cf925_0               conda-forge              
    libclang:        16.0.0.rc3-default_h5b99a12_0 conda-forge/label/llvm_rc
    libclang-cpp:    16.0.0.rc3-default_h5b99a12_0 conda-forge/label/llvm_rc
    libclang-cpp16:  16.0.0.rc3-default_h5b99a12_0 conda-forge/label/llvm_rc
    libclang13:      16.0.0.rc3-default_h4e9855e_0 conda-forge/label/llvm_rc
    libcxx:          14.0.6-hccf4f1f_0             conda-forge              
    libiconv:        1.17-hac89ed1_0               conda-forge              
    libllvm16:       16.0.0.rc3-h8456b07_1         conda-forge/label/llvm_rc
    libxml2:         2.10.3-hb9e07b5_0             conda-forge              
    libzlib:         1.2.13-hfd90126_4             conda-forge              
    llvm-tools:      16.0.0.rc3-h8456b07_1         conda-forge/label/llvm_rc
    llvmdev:         16.0.0.rc3-h8456b07_1         conda-forge/label/llvm_rc
    xz:              5.2.6-h775f41a_0              conda-forge              
    zlib:            1.2.13-hfd90126_4             conda-forge              
    zstd:            1.5.2-hbc0c0cd_6              conda-forge

build env:

  environment location: /Users/runner/miniforge3/conda-bld/compiler-rt-packages_1677355103841/_build_env

The following NEW packages will be INSTALLED:

    bzip2:                  1.0.8-h0d85af4_4          conda-forge
    c-ares:                 1.18.1-h0d85af4_0         conda-forge
    ca-certificates:        2022.12.7-h033912b_0      conda-forge
    clang_bootstrap_osx-64: 14.0.6-h6ddedbb_4         conda-forge
    cmake:                  3.25.2-h4032537_0         conda-forge
    expat:                  2.5.0-hf0c8a7f_0          conda-forge
    krb5:                   1.20.1-h049b76e_0         conda-forge
    libcurl:                7.88.1-h6df9250_0         conda-forge
    libcxx:                 14.0.6-hccf4f1f_0         conda-forge
    libedit:                3.1.20191231-h0678c8f_2   conda-forge
    libev:                  4.33-haf1e3a3_1           conda-forge
    libffi:                 3.4.2-h0d85af4_5          conda-forge
    libnghttp2:             1.51.0-he2ab024_0         conda-forge
    libsqlite:              3.40.0-ha978bb4_0         conda-forge
    libssh2:                1.10.0-h47af595_3         conda-forge
    libuv:                  1.44.2-hac89ed1_0         conda-forge
    libzlib:                1.2.13-hfd90126_4         conda-forge
    ncurses:                6.3-h96cf925_1            conda-forge
    ninja:                  1.11.1-hb8565cd_0         conda-forge
    openssl:                3.0.8-hfd90126_0          conda-forge
    python:                 3.11.0-he7542f4_1_cpython conda-forge
    readline:               8.1.2-h3899abd_0          conda-forge
    rhash:                  1.4.3-hac89ed1_0          conda-forge
    tk:                     8.6.12-h5dbffcc_0         conda-forge
    tzdata:                 2022g-h191b570_0          conda-forge
    xz:                     5.2.6-h775f41a_0          conda-forge
    zlib:                   1.2.13-hfd90126_4         conda-forge
    zstd:                   1.5.2-hbc0c0cd_6          conda-forge

Additional Context

Failing CI run here

The text was updated successfully, but these errors were encountered:

1player · 2023-02-28T17:31:31Z

Welp, just hit this as well, when it worked on Linux and Windows. Is it possible to work around it by downgrading to an earlier version of conda-build?

travishathaway · 2023-03-15T06:09:37Z

Hi @h-vetinari,

Thanks for filing this bug. I dug a little deeper on this, and it turns out our usage of this particular library in conda-build is a bit fragile. In my opinion, I think it's probably to time refactor https://github.com/conda/conda-build/blob/main/conda_build/os_utils/liefldd.py and reconsider the way we use the lief package as a dependency in that module.

I've added the https://github.com/conda/conda-build/labels/good-first-issue label because this could be a reasonable task for someone new to the repository. Otherwise, we unfortunately cannot promise when this will be addressed.

travishathaway · 2023-03-15T06:12:38Z

@1player,

I don't think it's possible to work around this with an earlier version of conda-build. After viewing a git blame, I could see this code was added five years ago.

I think the work around fix for this is to somehow force install the version of lief that conda-build works with. I am currently not aware of which version this is though. This is a very undesirable route to go though, in my opinion.

h-vetinari · 2023-03-15T23:18:53Z

Looks like there's an upstream bug to track for this: lief-project/LIEF#873

isuruf · 2023-03-16T04:27:16Z

The easiest way to work-around this would be to skip files ending with .a at

conda-build/conda_build/post.py

Line 738 in c5146a0

if not codefile_type(path):

Collecting of dependencies of .a static libraries have never worked and is not expected to work as they do not have dependency info on them.

h-vetinari · 2023-03-16T05:25:43Z

Thanks a lot for the insight @isuruf! I opened a PR: #4808

1player · 2023-03-16T09:15:50Z

The easiest way to work-around this would be to skip files ending with .a at

conda-build/conda_build/post.py

Line 738 in c5146a0

if not codefile_type(path):

Collecting of dependencies of .a static libraries have never worked and is not expected to work as they do not have dependency info on them.

I don't think that's a good approach.

I've been hitting this issue because we are packaging LLVM, and it is crucial the clang_rt-builtins.a file is present in the final package as it is needed for LLVM users. This issue affects only macOS, because we have no issues on Linux and that .a file.

h-vetinari · 2023-03-16T09:24:00Z

I think you misunderstand (or I do!)... This is not a question whether we package static libs (there are cases that are unavoidable, like compiler-builtins that you mention), but whether we run the lief-machinery on them that inspects the artefacts for their dependencies (and fails).

1player · 2023-03-31T08:43:11Z

Any update? We're stuck because we need to build compiler-rt, and there are two PR that fix this exact issue.

katietz · 2023-05-24T17:11:22Z

yes, I agree. I ran into same issue with llvm/clang 16 building ... I patched lief/conda-build for it

katietz · 2023-05-24T17:12:20Z

I will share a PR with conda-build on the changes I made to it. In general it would be pretty good to have a switch in conda-build, which could be used to turn off the lief checks

1player · 2023-05-24T19:54:17Z

There are two unmerged PRs that solve this bug already. They just need someone to review and merge one or the other.

…

On Wed, 24 May 2023, at 18:12, Kai Tietz wrote: I will share a PR with conda-build on the changes I made to it. In general it would be pretty good to have a switch in conda-build, which could be used to turn off the lief checks — Reply to this email directly, view it on GitHub <#4787 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFIPSGP5ZJNQK3B7RH5YLLXHY6P5ANCNFSM6AAAAAAVII4MKI>. You are receiving this because you were mentioned.Message ID: ***@***.***>

katietz · 2023-05-25T05:36:51Z

I just read through them. I think they both fix the issue here in particular, but there might be a way to resolve it more generic.
Actually we should try to stop to run on something binary is 'NoneType' . But yes, looking and filtering out specific file-endings would be good.

Btw why this filtering of file-endings was removed anyway. IIRC there was such a feature in the past, and now we attempt to run over any files in PREFIX with a lot of noise

katietz · 2023-05-25T06:04:52Z

see as suggestion to fix this the pull 4900, which does this in a more general way

h-vetinari · 2023-05-25T06:39:02Z

Doesn't hurt to link it: #4900

Fine by me, whatever gets this resolved, thanks! :)

h-vetinari added the type::bug describes erroneous operation, use severity::* to classify the type label Feb 26, 2023

craustin mentioned this issue Feb 27, 2023

Add recipe for deltalake library conda-forge/staged-recipes#22162

Closed

10 tasks

sklam mentioned this issue Mar 2, 2023

Add support for compiler-rt inclusion numba/llvmlite#909

Open

travishathaway added backlog issue has been triaged but has not been earmarked for any upcoming release good first issue great for new contributors, code change is envisioned to be trivial/relatively straight-forward labels Mar 15, 2023

travishathaway added the severity::2 critical; broken functionality with an unacceptably complex workaround label Mar 15, 2023

h-vetinari mentioned this issue Mar 15, 2023

BUG: unexpected binary is None in liefldd.py conda-forge/conda-forge.github.io#1916

Open

h-vetinari mentioned this issue Mar 16, 2023

don't try to run lief on static libraries #4808

Closed

1player mentioned this issue Mar 22, 2023

Don't run lief on UNIX static libraries #4819

Closed

travishathaway changed the title ~~BUG: unexpected binary is None in liefldd.py~~ Unexpected binary is None in liefldd.py May 8, 2023

travishathaway added the source::community catch-all for issues filed by community members label May 8, 2023

jezdez mentioned this issue Jun 5, 2023

Make sure we don't use binary on EmptyType with lief #4900

Merged

jezdez closed this as completed in #4900 Jun 6, 2023

h-vetinari mentioned this issue Jul 31, 2023

Fix path on windows conda-forge/compiler-rt-feedstock#79

Merged

h-vetinari mentioned this issue Sep 26, 2023

Remote-ci-setup handling cannot handle constraints anymore (since conda-smithy 3.26) conda-forge/conda-smithy#1773

Closed

peterjc mentioned this issue Sep 30, 2023

Another attempt to package qsv written in rust conda-forge/staged-recipes#24081

Open

10 tasks

github-actions bot added the locked [bot] locked due to inactivity label Jun 6, 2024

github-actions bot locked as resolved and limited conversation to collaborators Jun 6, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unexpected `binary is None` in `liefldd.py` #4787

Unexpected `binary is None` in `liefldd.py` #4787

h-vetinari commented Feb 26, 2023 •

edited

Loading

1player commented Feb 28, 2023

travishathaway commented Mar 15, 2023

travishathaway commented Mar 15, 2023

h-vetinari commented Mar 15, 2023

isuruf commented Mar 16, 2023

h-vetinari commented Mar 16, 2023

1player commented Mar 16, 2023 •

edited

Loading

h-vetinari commented Mar 16, 2023

1player commented Mar 31, 2023

katietz commented May 24, 2023

katietz commented May 24, 2023

1player commented May 24, 2023 via email

katietz commented May 25, 2023

katietz commented May 25, 2023

h-vetinari commented May 25, 2023

Unexpected binary is None in liefldd.py #4787

Unexpected binary is None in liefldd.py #4787

Comments

h-vetinari commented Feb 26, 2023 • edited Loading

Checklist

What happened?

Conda Info

Conda Config

Conda list

Additional Context

1player commented Feb 28, 2023

travishathaway commented Mar 15, 2023

travishathaway commented Mar 15, 2023

h-vetinari commented Mar 15, 2023

isuruf commented Mar 16, 2023

h-vetinari commented Mar 16, 2023

1player commented Mar 16, 2023 • edited Loading

h-vetinari commented Mar 16, 2023

1player commented Mar 31, 2023

katietz commented May 24, 2023

katietz commented May 24, 2023

1player commented May 24, 2023 via email

katietz commented May 25, 2023

katietz commented May 25, 2023

h-vetinari commented May 25, 2023

Unexpected `binary is None` in `liefldd.py` #4787

Unexpected `binary is None` in `liefldd.py` #4787

h-vetinari commented Feb 26, 2023 •

edited

Loading

1player commented Mar 16, 2023 •

edited

Loading