FAIL: pwhash_scrypt on aarch64

[gaozhangfei]

$make dev-deploy k8s multinode

/tmp/pip-install-mDKYOT/pynacl/src/libsodium/build-aux/test-driver: line 107: 22087 Killed                  "$@" > $log_file 2>&1
FAIL: pwhash_scrypt
PASS: pwhash_scrypt_ll
PASS: siphashx24
PASS: xchacha20
============================================================================
Testsuite summary for libsodium 1.0.15
============================================================================
# TOTAL: 70
# PASS:  69
# SKIP:  0
# XFAIL: 0
# FAIL:  1
# XPASS: 0
# ERROR: 0
============================================================================
See test/default/test-suite.log
Please report to https://github.com/jedisct1/libsodium/issues
============================================================================
Makefile:1770: recipe for target 'test-suite.log' failed
make[4]: *** [test-suite.log] Error 1
make[4]: Leaving directory '/tmp/pip-install-mDKYOT/pynacl/build/temp.linux-aarch64-2.7/test/default'
Makefile:1876: recipe for target 'check-TESTS' failed
make[3]: *** [check-TESTS] Error 2
make[3]: Leaving directory '/tmp/pip-install-mDKYOT/pynacl/build/temp.linux-aarch64-2.7/test/default'
Makefile:2432: recipe for target 'check-am' failed
make[2]: *** [check-am] Error 2
make[2]: Leaving directory '/tmp/pip-install-mDKYOT/pynacl/build/temp.linux-aarch64-2.7/test/default'
Makefile:400: recipe for target 'check-recursive' failed
make[1]: *** [check-recursive] Error 1
make[1]: Leaving directory '/tmp/pip-install-mDKYOT/pynacl/build/temp.linux-aarch64-2.7/test'
Makefile:513: recipe for target 'check-recursive' failed
make: *** [check-recursive] Error 1
Traceback (most recent call last):

[jedisct1]

Probably because you don't have enough memory to run that test.

[gaozhangfei]

Thanks Frank

I try to install openstack-helm on aarch64
$make dev-deploy k8s multinode
Every time it fails at pwhash_scrypt, the system memory is big enough: 32G

$ free
total used free shared buff/cache available
Mem: 32938120 1795572 30219656 337460 922892 30464076
Swap: 0 0 0

[gaozhangfei]

The error happen when "pip --no-cache-dir install ansible "
what strange is
pip --no-cache-dir install ansible can succeed when I test docker run a container, where pynacl can be successfully installed as well as ansible.

But it failed in docker build -t dockerfile, which almost do the same thing.
got
Running setup.py install for pynacl: still running...
Running setup.py install for pynacl: still running...
Running setup.py install for pynacl: still running...
Running setup.py install for pynacl: finished with status 'error'

[gaozhangfei]

Hi, Frank,

I use docker stats xx to monitor the process, and found it costs lots of memory, 28G and still increasing, finally oom should be triggered.
"pip --no-cache-dir install ansible " ->
Running setup.py install for pynacl: finished with status 'error' ->
trigger lots of test:
is it issue of pynacl or ansible or libsodium?

Thanks

[lafonj]

Hi, I'm having this same problem on Ubuntu 16.04 aarch64. I'm on a server with 128GB of RAM. This problem also happens when running make && make check on the latest stable libsodium source. Let me know if there is any way I can help debug

[zhangfeigao]

Here is a workaround: using "apt-get install ansible" instead of "pip --no-cache-dir install ansible "

[lafonj]

Thanks! That helped me get around the problem. It still fails the test if you try to build libsodium from source but I am unblocked.

[lafonj]

This issue is still following me. What I'm building now requires a wheels build of PyNaCl and just installing ansible or libsodium-dev is not enough to get around that.

[jedisct1]

Unfortunately, I was not able to duplicate this.

Can you pinpoint what exact test is causing this?

[lafonj]

The test that failed was pwhash_scrypt. There is not much of an error log all I see is ../../build-aux/test-driver: line 107: 59022 Killed.

[jedisct1]

That test suite is in test/default/test_scrypt.c. Try removing individual test cases until you find the one that triggers this.

[jedisct1]

Also check the session limits (ulimit -a).

[jedisct1]

Maybe you also have a pwhash_scrypt.res file that can help see where it stopped.

[lafonj]

Output of ulimit -a:

core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 515329
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 515329
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

End of pwhash_scrypt.res:

pwhash_str failure: [10]
pwhash_str failure: [11]
pwhash_str fail

[jedisct1]

max locked memory       (kbytes, -l) 64

That looks very low. Try bumping it to 16384 (the default value on the Ubuntu systems I have access to).

[lafonj]

Yep. That did the trick. Not sure why it was so low on my system. Thanks for the tip!