Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

python-pynacl: compilation failed #13016

Closed
feckert opened this issue Aug 4, 2020 · 10 comments · Fixed by #13033
Closed

python-pynacl: compilation failed #13016

feckert opened this issue Aug 4, 2020 · 10 comments · Fixed by #13033

Comments

@feckert
Copy link
Member

feckert commented Aug 4, 2020

Maintainer: @jmarcet or @jefferyto
Environment: x86_64, APU3, openwrt master latest commit

Description:
I get the following output if I want to compile docker-compose.
It does work a view weeks ago, but suddenly after updateing the package feed I got the follwoing compil error.
I am not a python god and do not know what the problem is now!
I haven't read into the whole python build system of openwrt.
I don't know where to start. Is this a problem with the host, target or the pip?

Can anyone from the specialists @jefferyto or @commodo help me out?

Collecting cffi>=1.4.1
  Using cached cffi-1.14.1.tar.gz (468 kB)
Requirement already satisfied: pycparser in /home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/staging_dir/hostpkg/lib/python3.8/site-packages (from cffi>=1.4.1) (2.20)
Skipping wheel build for cffi, due to binaries being disabled for it.
Installing collected packages: cffi
    Running setup.py install for cffi: started
    Running setup.py install for cffi: finished with status 'done'
Successfully installed cffi-1.14.1
WARNING: The wheel package is not available.
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.8
creating build/lib.linux-x86_64-3.8/nacl
copying src/nacl/exceptions.py -> build/lib.linux-x86_64-3.8/nacl
copying src/nacl/__init__.py -> build/lib.linux-x86_64-3.8/nacl
copying src/nacl/hash.py -> build/lib.linux-x86_64-3.8/nacl
copying src/nacl/utils.py -> build/lib.linux-x86_64-3.8/nacl
copying src/nacl/public.py -> build/lib.linux-x86_64-3.8/nacl
copying src/nacl/encoding.py -> build/lib.linux-x86_64-3.8/nacl
copying src/nacl/hashlib.py -> build/lib.linux-x86_64-3.8/nacl
copying src/nacl/secret.py -> build/lib.linux-x86_64-3.8/nacl
copying src/nacl/signing.py -> build/lib.linux-x86_64-3.8/nacl
creating build/lib.linux-x86_64-3.8/nacl/pwhash
copying src/nacl/pwhash/__init__.py -> build/lib.linux-x86_64-3.8/nacl/pwhash
copying src/nacl/pwhash/_argon2.py -> build/lib.linux-x86_64-3.8/nacl/pwhash
copying src/nacl/pwhash/argon2i.py -> build/lib.linux-x86_64-3.8/nacl/pwhash
copying src/nacl/pwhash/argon2id.py -> build/lib.linux-x86_64-3.8/nacl/pwhash
copying src/nacl/pwhash/scrypt.py -> build/lib.linux-x86_64-3.8/nacl/pwhash
creating build/lib.linux-x86_64-3.8/nacl/bindings
copying src/nacl/bindings/__init__.py -> build/lib.linux-x86_64-3.8/nacl/bindings
copying src/nacl/bindings/crypto_generichash.py -> build/lib.linux-x86_64-3.8/nacl/bindings
copying src/nacl/bindings/crypto_secretbox.py -> build/lib.linux-x86_64-3.8/nacl/bindings
copying src/nacl/bindings/crypto_shorthash.py -> build/lib.linux-x86_64-3.8/nacl/bindings
copying src/nacl/bindings/utils.py -> build/lib.linux-x86_64-3.8/nacl/bindings
copying src/nacl/bindings/randombytes.py -> build/lib.linux-x86_64-3.8/nacl/bindings
copying src/nacl/bindings/crypto_pwhash.py -> build/lib.linux-x86_64-3.8/nacl/bindings
copying src/nacl/bindings/crypto_hash.py -> build/lib.linux-x86_64-3.8/nacl/bindings
copying src/nacl/bindings/sodium_core.py -> build/lib.linux-x86_64-3.8/nacl/bindings
copying src/nacl/bindings/crypto_aead.py -> build/lib.linux-x86_64-3.8/nacl/bindings
copying src/nacl/bindings/crypto_scalarmult.py -> build/lib.linux-x86_64-3.8/nacl/bindings
copying src/nacl/bindings/crypto_sign.py -> build/lib.linux-x86_64-3.8/nacl/bindings
copying src/nacl/bindings/crypto_box.py -> build/lib.linux-x86_64-3.8/nacl/bindings
copying src/nacl/bindings/crypto_kx.py -> build/lib.linux-x86_64-3.8/nacl/bindings
copying src/nacl/bindings/crypto_secretstream.py -> build/lib.linux-x86_64-3.8/nacl/bindings
copying src/nacl/bindings/crypto_core.py -> build/lib.linux-x86_64-3.8/nacl/bindings
warning: build_py: byte-compiling is disabled, skipping.

running build_ext
generating cffi module 'build/temp.linux-x86_64-3.8/_sodium.c'
creating build/temp.linux-x86_64-3.8
building '_sodium' extension
creating build/temp.linux-x86_64-3.8/build
creating build/temp.linux-x86_64-3.8/build/temp.linux-x86_64-3.8
x86_64-openwrt-linux-musl-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -Os -pipe -fno-caller-saves -fno-plt -fhonour-copts -Wno-error=unused-but-set-variable -Wno-error=unused-result -ffile-prefix-map=/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/build_dir/target-x86_64_musl/Python-3.8.5=Python-3.8.5 -Wformat -Werror=format-security -fpic -fstack-protector -D_FORTIFY_SOURCE=1 -Wl,-z,now -Wl,-z,relro -Os -pipe -fno-caller-saves -fno-plt -fhonour-copts -Wno-error=unused-but-set-variable -Wno-error=unused-result -ffile-prefix-map=/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/build_dir/target-x86_64_musl/Python-3.8.5=Python-3.8.5 -Wformat -Werror=format-security -fpic -fstack-protector -D_FORTIFY_SOURCE=1 -Wl,-z,now -Wl,-z,relro -DNDEBUG -fno-inline -Os -pipe -fno-caller-saves -fno-plt -fhonour-copts -Wno-error=unused-but-set-variable -Wno-error=unused-result -ffile-prefix-map=/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/build_dir/target-x86_64_musl/Python-3.8.5=Python-3.8.5 -Wformat -Werror=format-security -fpic -fstack-protector -D_FORTIFY_SOURCE=1 -Wl,-z,now -Wl,-z,relro -DNDEBUG -fno-inline -I/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/staging_dir/target-x86_64_musl/usr/include -I/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/staging_dir/toolchain-x86_64_gcc-8.4.0_musl/usr/include -I/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/staging_dir/toolchain-x86_64_gcc-8.4.0_musl/include/fortify -I/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/staging_dir/toolchain-x86_64_gcc-8.4.0_musl/include -Os -pipe -fno-caller-saves -fno-plt -fhonour-copts -Wno-error=unused-but-set-variable -Wno-error=unused-result -ffile-prefix-map=/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/build_dir/target-x86_64_musl/pypi/PyNaCl-1.4.0=PyNaCl-1.4.0 -Wformat -Werror=format-security -fpic -fstack-protector -D_FORTIFY_SOURCE=1 -Wl,-z,now -Wl,-z,relro -I/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/staging_dir/target-x86_64_musl/usr/include -I/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/staging_dir/toolchain-x86_64_gcc-8.4.0_musl/usr/include -I/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/staging_dir/toolchain-x86_64_gcc-8.4.0_musl/include/fortify -I/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/staging_dir/toolchain-x86_64_gcc-8.4.0_musl/include -I/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/staging_dir/target-x86_64_musl/usr/include/python3.8 -fPIC -I/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/staging_dir/target-x86_64_musl/usr/include/python3.8 -c build/temp.linux-x86_64-3.8/_sodium.c -o build/temp.linux-x86_64-3.8/build/temp.linux-x86_64-3.8/_sodium.o
cc1: note: someone does not honour COPTS correctly, passed 4 times
build/temp.linux-x86_64-3.8/_sodium.c:682:10: error: 'crypto_scalarmult_ed25519_bytes' redeclared as different kind of symbol
 size_t (*crypto_scalarmult_ed25519_bytes)() = NULL;
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/staging_dir/target-x86_64_musl/usr/include/sodium.h:36,
                 from build/temp.linux-x86_64-3.8/_sodium.c:540:
/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/staging_dir/target-x86_64_musl/usr/include/sodium/crypto_scalarmult_ed25519.h:15:8: note: previous declaration of 'crypto_scalarmult_ed25519_bytes' was here
 size_t crypto_scalarmult_ed25519_bytes(void);
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
build/temp.linux-x86_64-3.8/_sodium.c:683:10: error: 'crypto_scalarmult_ed25519_scalarbytes' redeclared as different kind of symbol
 size_t (*crypto_scalarmult_ed25519_scalarbytes)() = NULL;
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/staging_dir/target-x86_64_musl/usr/include/sodium.h:36,
                 from build/temp.linux-x86_64-3.8/_sodium.c:540:
/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/staging_dir/target-x86_64_musl/usr/include/sodium/crypto_scalarmult_ed25519.h:19:8: note: previous declaration of 'crypto_scalarmult_ed25519_scalarbytes' was here
 size_t crypto_scalarmult_ed25519_scalarbytes(void);
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
build/temp.linux-x86_64-3.8/_sodium.c:685:7: error: 'crypto_scalarmult_ed25519' redeclared as different kind of symbol
 int (*crypto_scalarmult_ed25519)(unsigned char *, const unsigned char *, const unsigned char *) = NULL;
       ^~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/staging_dir/target-x86_64_musl/usr/include/sodium.h:36,
                 from build/temp.linux-x86_64-3.8/_sodium.c:540:
/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/staging_dir/target-x86_64_musl/usr/include/sodium/crypto_scalarmult_ed25519.h:30:5: note: previous declaration of 'crypto_scalarmult_ed25519' was here
 int crypto_scalarmult_ed25519(unsigned char *q, const unsigned char *n,
     ^~~~~~~~~~~~~~~~~~~~~~~~~
build/temp.linux-x86_64-3.8/_sodium.c:686:7: error: 'crypto_scalarmult_ed25519_base' redeclared as different kind of symbol
 int (*crypto_scalarmult_ed25519_base)(unsigned char *, const unsigned char *) = NULL;
       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/staging_dir/target-x86_64_musl/usr/include/sodium.h:36,
                 from build/temp.linux-x86_64-3.8/_sodium.c:540:
/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/staging_dir/target-x86_64_musl/usr/include/sodium/crypto_scalarmult_ed25519.h:40:5: note: previous declaration of 'crypto_scalarmult_ed25519_base' was here
 int crypto_scalarmult_ed25519_base(unsigned char *q, const unsigned char *n)
     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
build/temp.linux-x86_64-3.8/_sodium.c:687:7: error: 'crypto_scalarmult_ed25519_noclamp' redeclared as different kind of symbol
 int (*crypto_scalarmult_ed25519_noclamp)(unsigned char *, const unsigned char *,
       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/staging_dir/target-x86_64_musl/usr/include/sodium.h:36,
                 from build/temp.linux-x86_64-3.8/_sodium.c:540:
/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/staging_dir/target-x86_64_musl/usr/include/sodium/crypto_scalarmult_ed25519.h:35:5: note: previous declaration of 'crypto_scalarmult_ed25519_noclamp' was here
 int crypto_scalarmult_ed25519_noclamp(unsigned char *q, const unsigned char *n,
     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
build/temp.linux-x86_64-3.8/_sodium.c:689:7: error: 'crypto_scalarmult_ed25519_base_noclamp' redeclared as different kind of symbol
 int (*crypto_scalarmult_ed25519_base_noclamp)(unsigned char *, const unsigned char *) = NULL;
       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/staging_dir/target-x86_64_musl/usr/include/sodium.h:36,
                 from build/temp.linux-x86_64-3.8/_sodium.c:540:
/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/staging_dir/target-x86_64_musl/usr/include/sodium/crypto_scalarmult_ed25519.h:44:5: note: previous declaration of 'crypto_scalarmult_ed25519_base_noclamp' was here
 int crypto_scalarmult_ed25519_base_noclamp(unsigned char *q, const unsigned char *n)
     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
error: command 'x86_64-openwrt-linux-musl-gcc' failed with exit status 1
Makefile:39: recipe for target '/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/build_dir/target-x86_64_musl/pypi/PyNaCl-1.4.0/.built' failed
make[4]: *** [/home/bbworker/bbworker/owrt_master_x86_64/System6/build/openwrt/build_dir/target-x86_64_musl/pypi/PyNaCl-1.4.0/.built] Error 1
@feckert
Copy link
Member Author

feckert commented Aug 4, 2020

The only thing I have changed to upstream is, that I have applied my patch to move the pip-cache to tmp.

@commodo
Copy link
Contributor

commodo commented Aug 4, 2020

I updated python3-cffi recently to 1.4.1
Will check with that.

@commodo
Copy link
Contributor

commodo commented Aug 4, 2020

I updated python3-cffi recently to 1.4.1
Will check with that.

wait; that shouldn't matter; i updated cffi to 1.14.1 recently; but that's on the target-side;
this looks host-side;

@commodo
Copy link
Contributor

commodo commented Aug 4, 2020

so, this patch in libsodium seems to have broken things here: 3ef28a4
removing that patch helps;
@dangowrt @jmarcet @damianorenfer ideas?

we can probably patch this into PyNaCl, but I wouldn't enforce a solution;
if possible i would let people discuss on this;

@neheb
Copy link
Contributor

neheb commented Aug 4, 2020

Should be fixed now. The problem was a missing autoreconf.

edit: never mind. This package needs patching too.

@jefferyto
Copy link
Member

I think patching out all of the stub definitions in src/bindings/minimal/crypto_scalarmult.h and always setting static const int PYNACL_HAS_CRYPTO_SCALARMULT_ED25519 = 1 should work, but I haven't tested this.

I would have preferred variants for libsodium (#12902) but I see @neheb was the one who suggested changing the minimal build API 😦

@neheb
Copy link
Contributor

neheb commented Aug 4, 2020

Looks right.

edit: I mentioned why I didn't like variants. They're a timing issue.

@jefferyto
Copy link
Member

I know - I got into an argument about variants and InstallDev a while ago (which I am not going to link to here 😂)

I think the best solution is some kind of lib-switching mechanism in the main repo, where dependent packages can specify which variant they want and the right InstallDev files are switched/linked/etc. just for them, but I really don't have time to look into something so obscure right now 😂

commodo added a commit to commodo/packages that referenced this issue Aug 5, 2020
Fixes openwrt#13016
Patch [1] broke compilation for python-pynacl.

The fix is to patch PyNaCl to consider that
PYNACL_HAS_CRYPTO_SCALARMULT_ED25519 is always available.

[1] openwrt@3ef28a4

Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
@commodo
Copy link
Contributor

commodo commented Aug 5, 2020

PR #13033

@feckert
Copy link
Member Author

feckert commented Aug 6, 2020

Thanks all for the quick fix 👍

1715173329 pushed a commit to immortalwrt/packages that referenced this issue Aug 7, 2020
Fixes openwrt/packages#13016
Patch [1] broke compilation for python-pynacl.

The fix is to patch PyNaCl to consider that
PYNACL_HAS_CRYPTO_SCALARMULT_ED25519 is always available.

[1] openwrt/packages@3ef28a4

Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
farmergreg pushed a commit to farmergreg/packages that referenced this issue Sep 8, 2020
Fixes openwrt#13016
Patch [1] broke compilation for python-pynacl.

The fix is to patch PyNaCl to consider that
PYNACL_HAS_CRYPTO_SCALARMULT_ED25519 is always available.

[1] openwrt@3ef28a4

Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
farmergreg pushed a commit to farmergreg/packages that referenced this issue Sep 8, 2020
Fixes openwrt#13016
Patch [1] broke compilation for python-pynacl.

The fix is to patch PyNaCl to consider that
PYNACL_HAS_CRYPTO_SCALARMULT_ED25519 is always available.

[1] openwrt@3ef28a4

Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
pprindeville pushed a commit to pprindeville/packages that referenced this issue Dec 19, 2020
Fixes openwrt#13016
Patch [1] broke compilation for python-pynacl.

The fix is to patch PyNaCl to consider that
PYNACL_HAS_CRYPTO_SCALARMULT_ED25519 is always available.

[1] openwrt@3ef28a4

Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
riptidewave93 referenced this issue Oct 10, 2021
Functions from ed25519_core are needed for GNUnet to build.
Include them in the minimal build of libsodium so we don't need to
switch to the full build just for that.

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants