New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid cffi build process if not necessary #20

Merged
merged 2 commits into from Nov 7, 2016

Conversation

Projects
None yet
3 participants
@hynek
Owner

hynek commented Nov 5, 2016

No @glyph, Python packaging is not good now. 馃槩

fixes #17

@codecov-io

This comment has been minimized.

codecov-io commented Nov 5, 2016

Current coverage is 100% (diff: 100%)

Merging #20 into master will not change coverage

@@           master   #20   diff @@
===================================
  Files           7     7          
  Lines         141   141          
  Methods         0     0          
  Messages        0     0          
  Branches       15    15          
===================================
  Hits          141   141          
  Misses          0     0          
  Partials        0     0          

Powered by Codecov. Last update baa61fe...5738d2b

@hynek

This comment has been minimized.

Owner

hynek commented Nov 5, 2016

@guettli can you verify if this fixes your problem?

@guettli

This comment has been minimized.

guettli commented Nov 7, 2016

I guess this can only be tested if a new version is here: https://pypi.python.org/pypi/argon2_cffi

I see the same error, but AFAIK the version on pypi is still the one without the fix.


(pypi)pypi@pypiserver:~> pip download  -d packages argon2-cffi

Collecting argon2-cffi
/home/pypi/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
  SNIMissingWarning
/home/pypi/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
  File was already downloaded /home/pypi/packages/argon2_cffi-16.2.0.tar.gz
    Complete output from command python setup.py egg_info:
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    c/_cffi_backend.c:15:17: fatal error: ffi.h: Datei oder Verzeichnis nicht gefunden
    compilation terminated.
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/home/pypi/tmp/pip-build-SkFE8Q/argon2-cffi/setup.py", line 206, in <module>
        extras_require=EXTRAS_REQUIRE,
      File "/usr/lib/python2.7/distutils/core.py", line 112, in setup
        _setup_distribution = dist = klass(attrs)
      File "/home/pypi/lib/python2.7/site-packages/setuptools/dist.py", line 265, in __init__
        self.fetch_build_eggs(attrs['setup_requires'])
      File "/home/pypi/lib/python2.7/site-packages/setuptools/dist.py", line 310, in fetch_build_eggs
        replace_conflicting=True,
      File "/home/pypi/lib/python2.7/site-packages/pkg_resources/__init__.py", line 824, in resolve
        dist = best[req.key] = env.best_match(req, ws, installer)
      File "/home/pypi/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1069, in best_match
        return self.obtain(req, installer)
      File "/home/pypi/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1081, in obtain
        return installer(requirement)
      File "/home/pypi/lib/python2.7/site-packages/setuptools/dist.py", line 377, in fetch_build_egg
        return cmd.easy_install(req)
      File "/home/pypi/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 629, in easy_install
        return self.install_item(spec, dist.location, tmpdir, deps)
      File "/home/pypi/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 659, in install_item
        dists = self.install_eggs(spec, download, tmpdir)
      File "/home/pypi/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 842, in install_eggs
        return self.build_and_install(setup_script, setup_base)
      File "/home/pypi/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1070, in build_and_install
        self.run_setup(setup_script, setup_base, args)
      File "/home/pypi/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1058, in run_setup
        raise DistutilsError("Setup script exited with %s" % (v.args[0],))
    distutils.errors.DistutilsError: Setup script exited with error: command 'gcc' failed with exit status 1

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /home/pypi/tmp/pip-build-SkFE8Q/argon2-cffi/

@hynek

This comment has been minimized.

Owner

hynek commented Nov 7, 2016

You can copy a checkout of the PR branch to your server and run the problematic line (python setup.py egg_info) there.

@guettli

This comment has been minimized.

guettli commented Nov 7, 2016

Yes, the branch seems to work. Here is what I did:

user@host:~/tmp> git clone https://github.com/hynek/argon2_cffi.git
Klone nach 'argon2_cffi'...
remote: Counting objects: 750, done.
remote: Compressing objects: 100% (7/7), done.
remote: Total 750 (delta 1), reused 0 (delta 0), pack-reused 743
Empfange Objekte: 100% (750/750), 134.74 KiB, done.
L枚se Unterschiede auf: 100% (455/455), done.

user@host:~/tmp> cd argon2_cffi/

user@host:~/tmp/argon2_cffi> git checkout side-effects
Branch side-effects set up to track remote branch side-effects from origin.
Gewechselt zu einem neuen Zweig 'side-effects'

user@host:~/tmp/argon2_cffi> git pull
Already up-to-date.

user@host:~/tmp/argon2_cffi> python setup.py egg_info
running egg_info
creating src/argon2_cffi.egg-info
writing requirements to src/argon2_cffi.egg-info/requires.txt
writing src/argon2_cffi.egg-info/PKG-INFO
writing top-level names to src/argon2_cffi.egg-info/top_level.txt
writing dependency_links to src/argon2_cffi.egg-info/dependency_links.txt
writing manifest file 'src/argon2_cffi.egg-info/SOURCES.txt'
reading manifest file 'src/argon2_cffi.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files found matching 'src/argon2/_ffi.py'
warning: no previously-included files found matching '.gitmodules'
warning: no previously-included files found matching 'extras/libargon2/.git'
warning: no previously-included files found matching 'appveyor.yml'
warning: no previously-included files matching '*.pyc' found under directory 'tests'
no previously-included directories found matching 'docs/_build'
writing manifest file 'src/argon2_cffi.egg-info/SOURCES.txt'

@hynek

This comment has been minimized.

Owner

hynek commented Nov 7, 2016

Thanks!

@hynek hynek merged commit 0cea6f7 into master Nov 7, 2016

6 checks passed

codecov/patch Coverage not affected when comparing baa61fe...5738d2b
Details
codecov/project 100% (+0.00%) compared to baa61fe
Details
continuous-integration/appveyor/branch AppVeyor build succeeded
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details
@guettli

This comment has been minimized.

guettli commented Nov 8, 2016

I say "thank you". I ask myself why packaging does required coding. In theory a package should only provide data. I have not understood packaging python modules up to now. I am unsure if I every will understand it.

What is missing from setuptools/distutils/whateverlibrary for you to make packaging argon2_cffi a data only thing (without coding)?

@hynek hynek deleted the side-effects branch Nov 8, 2016

@hynek

This comment has been minimized.

Owner

hynek commented Nov 8, 2016

The short answer is wheels. They are binary, data-only, and really easy to build for Python-only packages.

It鈥檚 a bit more difficult for macOS & Windows but argon2-cffi does ship them. Linux wheels are a rather recent development and the next release of argon2-cffi will ship them too if everything goes like I wanted it to do.

For that matter, I hope your PyPI mirror (have you tried devpi btw? we鈥檙e using it with great success) is wheel aware. Recent pips should pick the correct wheel automatically.


The reason we fixed this anyway is that sdist (source distributions) that require the execution of setup.py to gain meta data will always be around for now. (there are efforts to change that but we鈥檒l have to see what comes out of it)

@guettli

This comment has been minimized.

guettli commented Nov 9, 2016

@hynek thank you very much for fixing this. When will be the fix on pypi?

@hynek

This comment has been minimized.

Owner

hynek commented Nov 10, 2016

@guettli

This comment has been minimized.

guettli commented Nov 10, 2016

@hynek yes, works. Great, I am happy.

I got this message first:

distutils.errors.DistutilsError: Setup script exited with error: command 'x86_64-linux-gnu-gcc'

But after apt install python3.5-dev I got it installed.

Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment