New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Regression: MBEDTLS_THREADING_ALT defined, but not all prerequisites #390

Closed
robert-scheck opened this Issue Jan 10, 2016 · 9 comments

Comments

Projects
None yet
5 participants
@robert-scheck

robert-scheck commented Jan 10, 2016

Since mbedtls 2.2.1, building fails on RHEL 7, 6 and 5 like this (all architectures):

cd /builddir/build/BUILD/mbedtls-2.2.1/library && /usr/bin/gcc  -Dmbedcrypto_EXPORTS -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic  -Wall -Wextra -W -Wdeclaration-after-statement -Wwrite-strings -Wmissing-declarations -Wmissing-prototypes -O2 -fPIC -I/builddir/build/BUILD/mbedtls-2.2.1/include   -o CMakeFiles/mbedcrypto.dir/aes.c.o   -c /builddir/build/BUILD/mbedtls-2.2.1/library/aes.c
In file included from /builddir/build/BUILD/mbedtls-2.2.1/include/mbedtls/config.h:2509,
                 from /builddir/build/BUILD/mbedtls-2.2.1/library/aes.c:29:
/builddir/build/BUILD/mbedtls-2.2.1/include/mbedtls/check_config.h:487:2: error: #error "MBEDTLS_THREADING_ALT defined, but not all prerequisites"
In file included from /builddir/build/BUILD/mbedtls-2.2.1/include/mbedtls/config.h:2509,
                 from /builddir/build/BUILD/mbedtls-2.2.1/library/aes.c:29:
/builddir/build/BUILD/mbedtls-2.2.1/include/mbedtls/check_config.h:487:2: error: #error "MBEDTLS_THREADING_ALT defined, but not all prerequisites"
In file included from /builddir/build/BUILD/mbedtls-2.2.1/library/aes.c:38:
/builddir/build/BUILD/mbedtls-2.2.1/include/mbedtls/aes.h:279:21: error: aes_alt.h: No such file or directory
In file included from /builddir/build/BUILD/mbedtls-2.2.1/library/aes.c:43:
/builddir/build/BUILD/mbedtls-2.2.1/include/mbedtls/aesni.h:63: error: expected ')' before '*' token
In file included from /builddir/build/BUILD/mbedtls-2.2.1/include/mbedtls/config.h:2509,
                 from /builddir/build/BUILD/mbedtls-2.2.1/library/aesni.c:28:
/builddir/build/BUILD/mbedtls-2.2.1/include/mbedtls/check_config.h:487:2: error: #error "MBEDTLS_THREADING_ALT defined, but not all prerequisites"
In file included from /builddir/build/BUILD/mbedtls-2.2.1/include/mbedtls/aesni.h:26,
                 from /builddir/build/BUILD/mbedtls-2.2.1/library/aesni.c:35In file included from /builddir/build/BUILD/mbedtls-2.2.1/library/aes.c:38:
/builddir/build/BUILD/mbedtls-2.2.1/include/mbedtls/aes.h:279:21: error: aes_alt.h: No such file or directory
In file included from /builddir/build/BUILD/mbedtls-2.2.1/library/aes.c:43:
/builddir/build/BUILD/mbedtls-2.2.1/include/mbedtls/aesni.h:63: error: expected ')' before '*' token
:
/builddir/build/BUILD/mbedtls-2.2.1/include/mbedtls/aes.h:279:21: error: aes_alt.h: No such file or directory
In file included from /builddir/build/BUILD/mbedtls-2.2.1/library/aesni.c:35:
/builddir/build/BUILD/mbedtls-2.2.1/include/mbedtls/aesni.h:63: error: expected ')' before '*' token
/builddir/build/BUILD/mbedtls-2.2.1/library/aes.c: In function 'mbedtls_aes_self_test':
/builddir/build/BUILD/mbedtls-2.2.1/library/aes.c:1237: error: 'mbedtls_aes_context' undeclared (first use in this function)
/builddir/build/BUILD/mbedtls-2.2.1/library/aes.c:1237: error: (Each undeclared identifier is reported only once
/builddir/build/BUILD/mbedtls-2.2.1/library/aes.c:1237: error: for each function it appears in.)
/builddir/build/BUILD/mbedtls-2.2.1/library/aes.c:1237: error: expected ';' before 'ctx'
/builddir/build/BUILD/mbedtls-2.2.1/library/aes.c:1240: warning: implicit declaration of function 'mbedtls_aes_init'
/builddir/build/BUILD/mbedtls-2.2.1/library/aes.c:1240: error: 'ctx' undeclared (first use in this function)
/builddir/build/BUILD/mbedtls-2.2.1/library/aes.c:1258: warning: implicit declaration of function 'mbedtls_aes_setkey_dec'
/builddir/build/BUILD/mbedtls-2.2.1/library/aes.c:1261: warning: implicit declaration of function 'mbedtls_aes_crypt_ecb'
/builddir/build/BUILD/mbedtls-2.2.1/library/aes.c:1274: warning: implicit declaration of function 'mbedtls_aes_setkey_enc'
/builddir/build/BUILD/mbedtls-2.2.1/library/aes.c:1318: warning: implicit declaration of function 'mbedtls_aes_crypt_cbc'
/builddir/build/BUILD/mbedtls-2.2.1/library/aes.c:1384: warning: implicit declaration of function 'mbedtls_aes_crypt_cfb128'
/builddir/build/BUILD/mbedtls-2.2.1/library/aes.c:1442: warning: implicit declaration of function 'mbedtls_aes_crypt_ctr'
/builddir/build/BUILD/mbedtls-2.2.1/library/aes.c:1483: warning: implicit declaration of function 'mbedtls_aes_free'
make[2]: *** [library/CMakeFiles/mbedcrypto_static.dir/aes.c.o] Error 1
make[2]: Leaving directory `/builddir/build/BUILD/mbedtls-2.2.1'
/usr/bin/cmake -E cmake_progress_report /builddir/build/BUILD/mbedtls-2.2.1/CMakeFiles 
make[1]: *** [library/CMakeFiles/mbedcrypto_static.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
/builddir/build/BUILD/mbedtls-2.2.1/library/aes.c: In function 'mbedtls_aes_self_test':
/builddir/build/BUILD/mbedtls-2.2.1/library/aes.c:1237: error: 'mbedtls_aes_context' undeclared (first use in this function)
/builddir/build/BUILD/mbedtls-2.2.1/library/aes.c:1237: error: (Each undeclared identifier is reported only once
/builddir/build/BUILD/mbedtls-2.2.1/library/aes.c:1237: error: for each function it appears in.)
/builddir/build/BUILD/mbedtls-2.2.1/library/aes.c:1237: error: expected ';' before 'ctx'
/builddir/build/BUILD/mbedtls-2.2.1/library/aes.c:1240: warning: implicit declaration of function 'mbedtls_aes_init'
/builddir/build/BUILD/mbedtls-2.2.1/library/aes.c:1240: error: 'ctx' undeclared (first use in this function)
/builddir/build/BUILD/mbedtls-2.2.1/library/aes.c:1258: warning: implicit declaration of function 'mbedtls_aes_setkey_dec'
/builddir/build/BUILD/mbedtls-2.2.1/library/aes.c:1261: warning: implicit declaration of function 'mbedtls_aes_crypt_ecb'
/builddir/build/BUILD/mbedtls-2.2.1/library/aes.c:1274: warning: implicit declaration of function 'mbedtls_aes_setkey_enc'
/builddir/build/BUILD/mbedtls-2.2.1/library/aes.c:1318: warning: implicit declaration of function 'mbedtls_aes_crypt_cbc'
/builddir/build/BUILD/mbedtls-2.2.1/library/aes.c:1384: warning: implicit declaration of function 'mbedtls_aes_crypt_cfb128'
/builddir/build/BUILD/mbedtls-2.2.1/library/aes.c:1442: warning: implicit declaration of function 'mbedtls_aes_crypt_ctr'
/builddir/build/BUILD/mbedtls-2.2.1/library/aes.c:1483: warning: implicit declaration of function 'mbedtls_aes_free'
/builddir/build/BUILD/mbedtls-2.2.1/library/aesni.c:95: error: expected ')' before '*' token
make[2]: *** [library/CMakeFiles/mbedcrypto.dir/aes.c.o] Error 1
make[2]: *** Waiting for unfinished jobs....

Note that mbedtls 2.2.0 was building fine on RHEL 7, 6 and 5 on all architectures.

@ciarmcom

This comment has been minimized.

Member

ciarmcom commented Jan 11, 2016

ARM Internal Ref: IOTSSL-597

@ciarmcom ciarmcom added the mirrored label Jan 11, 2016

@mpg

This comment has been minimized.

Contributor

mpg commented Jan 11, 2016

This is weird. MBEDTLS_THREADING_ALT should not be defined in config.h.

% wget -q https://tls.mbed.org/download/mbedtls-2.2.1-apache.tgz
% tar xf mbedtls-2.2.1-apache.tgz
% grep THREADING_ALT mbedtls-2.2.1/include/mbedtls/config.h
 * \def MBEDTLS_THREADING_ALT
//#define MBEDTLS_THREADING_ALT
 * You will have to enable either MBEDTLS_THREADING_ALT or

As you can see, it's commented out. Did you run any other Make target
beforehand? We recently changed the apidoc target so that it now play tricks
with config.h - it's supposed to undo the changes though, but might fail to
if an error occurs - that could be a possible explanation.

@robert-scheck

This comment has been minimized.

robert-scheck commented Jan 11, 2016

https://kojipkgs.fedoraproject.org//work/tasks/7643/12487643/build.log is the complete build log. We are basically running:

/usr/bin/cmake -DCMAKE_C_FLAGS_RELEASE:STRING=-DNDEBUG -DCMAKE_CXX_FLAGS_RELEASE:STRING=-DNDEBUG -DCMAKE_Fortran_FLAGS_RELEASE:STRING=-DNDEBUG -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON -DCMAKE_INSTALL_PREFIX:PATH=/usr -DINCLUDE_INSTALL_DIR:PATH=/usr/include -DLIB_INSTALL_DIR:PATH=/usr/lib64 -DSYSCONF_INSTALL_DIR:PATH=/etc -DSHARE_INSTALL_PREFIX:PATH=/usr/share -DLIB_SUFFIX=64 -DBUILD_SHARED_LIBS:BOOL=ON -D CMAKE_BUILD_TYPE:String=Release -D USE_SHARED_MBEDTLS_LIBRARY:BOOL=1 .
make -j4 all apidoc
@mpg

This comment has been minimized.

Contributor

mpg commented Jan 11, 2016

Ok, the issue is you can't use -j if one of your targets is apidoc. If you replace that with make -j4 all && make apidoc that should work.

@mortenstevens

This comment has been minimized.

mortenstevens commented Jan 11, 2016

@mpg

But we are using the same build command on Fedora and it works fine?
See the Fedora buildlog: https://kojipkgs.fedoraproject.org//packages/mbedtls/2.2.1/1.fc23/data/logs/x86_64/build.log

The source RPM for RHEL 5, 6 and 7 is exactly the same as for Fedora.

@mpg

This comment has been minimized.

Contributor

mpg commented Jan 11, 2016

Well, the issue with make -j4 all apidoc is it's racy, so I'm not particularly surprised it sometimes works and sometimes breaks. (I'm more surprised if it always works on Fedora and always fails on RHEL, but well.)

@mortenstevens

This comment has been minimized.

mortenstevens commented Jan 11, 2016

@mpg
It's now building fine with make -j 1 on RHEL 6 and 7. The cause may be an older version of cmake. Fedora has cmake 3.3.x and RHEL cmake 2.8.x.

Overall, I think it's not the best solution to build with make -j1. Maybe you can revert these apidoc changes? Especially with a minor version like 2.2.1.

@jcowgill

This comment has been minimized.

Contributor

jcowgill commented Jan 11, 2016

The cause of this happing on RHEL only is definitely an old version of CMake. I tried with CMake 3.4 and 2.8.9 (easiest to get on Debian). CMake 3.4 inserts a .NOTPARALLEL target into the Makefile whereas 2.8.9 does not. This special target is enough to force make to build the all and apidoc targets in sequence.

@mpg

This comment has been minimized.

Contributor

mpg commented Jan 12, 2016

@mortenstevens Yes, as I was saying in #391, I think I'll revert that change, I failed to anticipate that it would break parallel builds with multiple targets, and I think that's a good enough reason to revert it.

However, as there are at least two workarounds (using -j1 or make -j4 all && make apidoc), it may have to wait until we have other reasons to make a new release. Sorry for the inconvenience in the meantime.

@mpg mpg closed this in c990189 Jan 12, 2016

gilles-peskine-arm added a commit to gilles-peskine-arm/mbedtls that referenced this issue Sep 10, 2018

Revert changes done to 'make apidoc' target
This partially reverts 1989caf (only the changes to Makefile and
CMakeLists, the addition to scripts/config.pl is kept).

Modifying config.h in the apidoc target creates a race condition with

    make -j4 all apidoc

where some parts of the library, tests or programs could be built with the
wrong config.h, resulting in all kinds of (semi-random) errors. Recent
versions of CMake mitigate this by adding a .NOTPARALLEL target to the
generated Makefile, but people would still get errors with older CMake
versions that are still in use (eg in RHEL 5), and with plain make.

An additional issue is that, by failing to use cp -p, the apidoc target was
updating the timestamp on config.h, which seems to cause further build issues.

Let's get back to the previous, safe, situation. The improved apidoc building
will be resurrected in a script in the next commit.

fixes ARMmbed#390
fixes ARMmbed#391
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment