Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

package doesn't install lib_build.h #108

Closed
GeeCastro opened this issue Oct 31, 2022 · 10 comments
Closed

package doesn't install lib_build.h #108

GeeCastro opened this issue Oct 31, 2022 · 10 comments

Comments

@GeeCastro
Copy link

Installing xattr on a fresh python environment on MacOS 12.6 works but library throws an error on import. It breaks poetry which has a dependency on xattr (see this issue)

Steps to reproduce (on a fresh python 3.8.13 environment):

  1. pip install xattr
  2. `python -c 'import xattr'
Traceback (most recent call last):
  File "/Users/gauthiercastro/.pyenv/versions/3.8.13/lib/python3.8/site-packages/xattr/lib.py", line 7, in <module>
    from ._lib import lib, ffi
ImportError: dlopen(/Users/gauthiercastro/.pyenv/versions/3.8.13/lib/python3.8/site-packages/_cffi_backend.cpython-38-darwin.so, 0x0002): Library not loaded: '@rpath/libffi.8.dylib'
  Referenced from: '/Users/gauthiercastro/.pyenv/versions/3.8.13/lib/python3.8/site-packages/_cffi_backend.cpython-38-darwin.so'
  Reason: tried: '/Users/gauthiercastro/miniconda3/envs/speccheck_eval/lib/libffi.8.dylib' (no such file), '/Users/gauthiercastro/miniconda3/envs/speccheck_eval/lib/libffi.8.dylib' (no such file), '/Users/gauthiercastro/miniconda3/envs/speccheck_eval/lib/libffi.8.dylib' (no such file), '/Users/gauthiercastro/miniconda3/envs/speccheck_eval/lib/libffi.8.dylib' (no such file), '/opt/homebrew/lib/libffi.8.dylib' (no such file), '/opt/homebrew/lib/libffi.8.dylib' (no such file), '/opt/homebrew/lib/libffi.8.dylib' (no such file), '/opt/homebrew/lib/libffi.8.dylib' (no such file), '/opt/homebrew/lib/libffi.8.dylib' (no such file), '/opt/homebrew/lib/libffi.8.dylib' (no such file), '/usr/local/lib/libffi.8.dylib' (no such file), '/usr/lib/libffi.8.dylib' (no such file)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/gauthiercastro/.pyenv/versions/3.8.13/lib/python3.8/site-packages/xattr/__init__.py", line 13, in <module>
    from .lib import (XATTR_NOFOLLOW, XATTR_CREATE, XATTR_REPLACE,
  File "/Users/gauthiercastro/.pyenv/versions/3.8.13/lib/python3.8/site-packages/xattr/lib.py", line 9, in <module>
    from .lib_build import ffi, c_source
  File "/Users/gauthiercastro/.pyenv/versions/3.8.13/lib/python3.8/site-packages/xattr/lib_build.py", line 7, in <module>
    with open(os.path.join(PATH, 'lib_build.h')) as hf:
FileNotFoundError: [Errno 2] No such file or directory: '/Users/gauthiercastro/.pyenv/versions/3.8.13/lib/python3.8/site-packages/xattr/lib_build.h'

Any idea what could be wrong?
@GeeCastro GeeCastro mentioned this issue Oct 31, 2022
Closed
4 tasks
@etrepum
Copy link
Member

etrepum commented Oct 31, 2022

It looks like your installation of cffi is broken or corrupted somehow, which causes the xattr import to fail and fall back to trying to build the extension which won't work. lib_build.h is only used when building the extension. There's nothing really that can be done from xattr's perspective here.

@etrepum etrepum closed this as completed Oct 31, 2022
This was referenced Nov 11, 2022
Closed
Merged
Secrus added a commit to python-poetry/poetry that referenced this issue Nov 11, 2022
@moonsikpark @Secrus
Bump xattr version to 0.10.0. (#7005)
28b0b1d 
# Pull Request Check List

Resolves: #6928 

<!-- This is just a reminder about the most common mistakes. Please make
sure that you tick all *appropriate* boxes. But please read our
[contribution guide](https://python-poetry.org/docs/contributing/) at
least once, it will save you unnecessary review cycles! -->

- [ ] Added **tests** for changed code.
- [ ] Updated **documentation** for changed code.

<!-- If you have *any* questions to *any* of the points above, just
**submit and ask**! This checklist is here to *help* you, not to deter
you from contributing! -->

Bump `xattr` version to `0.10.0`.

This fixes #6891, which is not a problem with the user's environment as
suggested by the maintainer of xattr at xattr/xattr#108, but in fact the
problem of the package which was fixed by xattr/xattr#106.

Looking at the [changes in the version
bump](xattr/xattr@v0.9.9...v0.10.0), there
does not seem to be any significant changes other than shabang
xattr/xattr#106.

Co-authored-by: Moonsik Park <moonsik.park@estsoft.com>
Co-authored-by: Bartosz Sokorski <b.sokorski@gmail.com>
moonsikpark added a commit to moonsikpark/poetry that referenced this issue Nov 15, 2022
@moonsikpark
chore: update mypy >= 0.990
1bb1bba 
update lockfile.

pip install --no-input everywhere (python-poetry#6966)

This is an extension of
python-poetry#6724. I think `pip install`
invoked by poetry should never ask for user input.

Motivation is that it happened to myself and a number of colleagues many
times that poetry got seemingly stuck while it was just waiting for a
user input because of a private pypi repository that needed
authentication.

I hope this is a valuable contribution to a tool I like a lot and would
like to use more and more :)

docs: update Windows cache path to match 1.2

Bump `xattr` version to `0.10.0`. (python-poetry#7005)

Resolves: python-poetry#6928

<!-- This is just a reminder about the most common mistakes. Please make
sure that you tick all *appropriate* boxes. But please read our
[contribution guide](https://python-poetry.org/docs/contributing/) at
least once, it will save you unnecessary review cycles! -->

- [ ] Added **tests** for changed code.
- [ ] Updated **documentation** for changed code.

<!-- If you have *any* questions to *any* of the points above, just
**submit and ask**! This checklist is here to *help* you, not to deter
you from contributing! -->

Bump `xattr` version to `0.10.0`.

This fixes python-poetry#6891, which is not a problem with the user's environment as
suggested by the maintainer of xattr at xattr/xattr#108, but in fact the
problem of the package which was fixed by xattr/xattr#106.

Looking at the [changes in the version
bump](xattr/xattr@v0.9.9...v0.10.0), there
does not seem to be any significant changes other than shabang
xattr/xattr#106.

Co-authored-by: Moonsik Park <moonsik.park@estsoft.com>
Co-authored-by: Bartosz Sokorski <b.sokorski@gmail.com>

tests: add coverage to `poetry install`

update lockfile.

[pre-commit.ci] pre-commit autoupdate

updates:
- [github.com/asottile/pyupgrade: v3.2.0 → v3.2.2](asottile/pyupgrade@v3.2.0...v3.2.2)
- [github.com/hadialqattan/pycln: v2.1.1 → v2.1.2](hadialqattan/pycln@v2.1.1...v2.1.2)
moonsikpark added a commit to moonsikpark/poetry that referenced this issue Nov 15, 2022
@moonsikpark
chore: update mypy >= 0.990
f74ac1d 
update lockfile.

pip install --no-input everywhere (python-poetry#6966)

This is an extension of
python-poetry#6724. I think `pip install`
invoked by poetry should never ask for user input.

Motivation is that it happened to myself and a number of colleagues many
times that poetry got seemingly stuck while it was just waiting for a
user input because of a private pypi repository that needed
authentication.

I hope this is a valuable contribution to a tool I like a lot and would
like to use more and more :)

docs: update Windows cache path to match 1.2

Bump `xattr` version to `0.10.0`. (python-poetry#7005)

Resolves: python-poetry#6928

<!-- This is just a reminder about the most common mistakes. Please make
sure that you tick all *appropriate* boxes. But please read our
[contribution guide](https://python-poetry.org/docs/contributing/) at
least once, it will save you unnecessary review cycles! -->

- [ ] Added **tests** for changed code.
- [ ] Updated **documentation** for changed code.

<!-- If you have *any* questions to *any* of the points above, just
**submit and ask**! This checklist is here to *help* you, not to deter
you from contributing! -->

Bump `xattr` version to `0.10.0`.

This fixes python-poetry#6891, which is not a problem with the user's environment as
suggested by the maintainer of xattr at xattr/xattr#108, but in fact the
problem of the package which was fixed by xattr/xattr#106.

Looking at the [changes in the version
bump](xattr/xattr@v0.9.9...v0.10.0), there
does not seem to be any significant changes other than shabang
xattr/xattr#106.

Co-authored-by: Moonsik Park <moonsik.park@estsoft.com>
Co-authored-by: Bartosz Sokorski <b.sokorski@gmail.com>

tests: add coverage to `poetry install`

update lockfile.

[pre-commit.ci] pre-commit autoupdate

updates:
- [github.com/asottile/pyupgrade: v3.2.0 → v3.2.2](asottile/pyupgrade@v3.2.0...v3.2.2)
- [github.com/hadialqattan/pycln: v2.1.1 → v2.1.2](hadialqattan/pycln@v2.1.1...v2.1.2)
moonsikpark added a commit to moonsikpark/poetry that referenced this issue Nov 15, 2022
@moonsikpark
chore: update mypy >= 0.990
87fca65 
update lockfile.

pip install --no-input everywhere (python-poetry#6966)

This is an extension of
python-poetry#6724. I think `pip install`
invoked by poetry should never ask for user input.

Motivation is that it happened to myself and a number of colleagues many
times that poetry got seemingly stuck while it was just waiting for a
user input because of a private pypi repository that needed
authentication.

I hope this is a valuable contribution to a tool I like a lot and would
like to use more and more :)

docs: update Windows cache path to match 1.2

Bump `xattr` version to `0.10.0`. (python-poetry#7005)

Resolves: python-poetry#6928

<!-- This is just a reminder about the most common mistakes. Please make
sure that you tick all *appropriate* boxes. But please read our
[contribution guide](https://python-poetry.org/docs/contributing/) at
least once, it will save you unnecessary review cycles! -->

- [ ] Added **tests** for changed code.
- [ ] Updated **documentation** for changed code.

<!-- If you have *any* questions to *any* of the points above, just
**submit and ask**! This checklist is here to *help* you, not to deter
you from contributing! -->

Bump `xattr` version to `0.10.0`.

This fixes python-poetry#6891, which is not a problem with the user's environment as
suggested by the maintainer of xattr at xattr/xattr#108, but in fact the
problem of the package which was fixed by xattr/xattr#106.

Looking at the [changes in the version
bump](xattr/xattr@v0.9.9...v0.10.0), there
does not seem to be any significant changes other than shabang
xattr/xattr#106.

Co-authored-by: Moonsik Park <moonsik.park@estsoft.com>
Co-authored-by: Bartosz Sokorski <b.sokorski@gmail.com>

tests: add coverage to `poetry install`

update lockfile.

[pre-commit.ci] pre-commit autoupdate

updates:
- [github.com/asottile/pyupgrade: v3.2.0 → v3.2.2](asottile/pyupgrade@v3.2.0...v3.2.2)
- [github.com/hadialqattan/pycln: v2.1.1 → v2.1.2](hadialqattan/pycln@v2.1.1...v2.1.2)

update lockfile.

update lockfile.
@phdowling
Copy link

This is still causing issues for me, I'm using an M2 Mac. Tried reinstalling xattr, cffi as well as my entire python environment but python -c "import xattr" still fails:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/dowling/.pyenv/versions/3.12.0/lib/python3.12/site-packages/xattr/__init__.py", line 13, in <module>
    from .lib import (XATTR_NOFOLLOW, XATTR_CREATE, XATTR_REPLACE,
  File "/Users/dowling/.pyenv/versions/3.12.0/lib/python3.12/site-packages/xattr/lib.py", line 9, in <module>
    from .lib_build import ffi, c_source
  File "/Users/dowling/.pyenv/versions/3.12.0/lib/python3.12/site-packages/xattr/lib_build.py", line 7, in <module>
    with open(os.path.join(PATH, 'lib_build.h')) as hf:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/Users/dowling/.pyenv/versions/3.12.0/lib/python3.12/site-packages/xattr/lib_build.h'

Installed xattr version: 0.10.0
Install cffi version: 1.16.0

Any ideas?

@etrepum
Copy link
Member

etrepum commented Nov 17, 2023

If you post full instructions to reproduce your issue, particularly how you installed python and what commands you ran to get your installation into this state, I will look into it. Is there a reason why you are attempting to use 0.10.0 instead of 0.10.1?

@samuelchrist
Copy link

samuelchrist commented Nov 17, 2023 •
edited

I am using poetry on M1 mac os Sonoma. Tried reinstalling poetry, xattr, cffi as well as my entire python environment but python -c "import xattr" still fails:

I am using xattr 0.10.1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/user/Library/Caches/pypoetry/virtualenvs/venv/lib/python3.8/site-packages/xattr/__init__.py", line 13, in <module>
    from .lib import (XATTR_NOFOLLOW, XATTR_CREATE, XATTR_REPLACE,
  File "/Users/user/Library/Caches/pypoetry/virtualenvs/venv/lib/python3.8/site-packages/xattr/lib.py", line 9, in <module>
    from .lib_build import ffi, c_source
  File "/Users/user/Library/Caches/pypoetry/virtualenvs/venv/lib/python3.8/site-packages/xattr/lib_build.py", line 7, in <module>
    with open(os.path.join(PATH, 'lib_build.h')) as hf:
FileNotFoundError: [Errno 2] No such file or directory: '/Users/user/Library/Caches/pypoetry/virtualenvs/venv/lib/python3.8/site-packages/xattr/lib_build.h'

Please help

To recreate this install pyenv using homebrew. Install python 3.8.17 in pyenv. Set python 3.8.17 in pyenv as global. then install poetry 1.2.0 using python 3.8.17 in pyenv. Try doing poetry add flaml==1.0.8 or python -c "import xattr"

@etrepum etrepum reopened this Nov 17, 2023
@etrepum
Copy link
Member

etrepum commented Nov 18, 2023

I don't know why you're having an issue with Python 3.8, but it does seem like Python 3.12 causes some serious problems with the build as it has removed a bunch of functionality that is required by the current build process and cffi version. Will probably be a few days at least until that can be sorted out, unless someone else wants to contribute.

@samuelchrist
Copy link

I don't know why you're having an issue with Python 3.8, but it does seem like Python 3.12 causes some serious problems with the build as it has removed a bunch of functionality that is required by the current build process and cffi version. Will probably be a few days at least until that can be sorted out, unless someone else wants to contribute.

Even I am not sure but i see there is some issue with this while using poetry. Can you help to understand what could be the root cause.? Are you able to replicate the issue. ?

@etrepum
Copy link
Member

etrepum commented Nov 18, 2023

I have no experience using poetry, you may want to ask them.

My top priority will be addressing Python 3.12 support, but I have limited time to work on it right now. If anyone has the experience and time to contribute I'd be happy to review PRs. I will look into poetry and Python 3.8 after Python 3.12 is supported, whenever that happens.

@phdowling
Copy link

Sorry for the late response, will try to put more detailed steps to reproduce tomorrow, but basically I think it was:

  • uninstall all python versions from the system
  • Install pyenv
  • Install 3.12 via pyenv
  • pip install xattrs
  • import xattr fails as outlined in my last comment

I sort of got it to work by manually putting the missing .h and .c files into the appropriate site_packages directory, but I was still left unable to use poetry. Used a manual virtualenv + pip for now but that is not really a good permanent solution for me.

@etrepum
Copy link
Member

etrepum commented Nov 18, 2023

It sounds like poetry is part of the problem in all of these cases, perhaps you should bring it up with that project

@etrepum
Copy link
Member

etrepum commented Nov 20, 2023

I cut a new release that removes usage of deprecated setuptools/distutils/cffi features and should at least have a different error message if the extension wasn't installed. It builds and installs correctly with Python 3.12 on all platforms and architectures that cibuildwheel works with.

@etrepum etrepum closed this as completed Nov 20, 2023
@mrmeszaros mrmeszaros mentioned this issue Nov 22, 2023
Closed
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@etrepum @phdowling @GeeCastro @samuelchrist