Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update to ndk 26 #17863

Merged
merged 6 commits into from
Oct 22, 2023
Merged

Update to ndk 26 #17863

merged 6 commits into from
Oct 22, 2023

Conversation

licy183
Copy link
Member

@licy183 licy183 commented Sep 21, 2023

Closes #17429

@finagolfin
Copy link
Member

Thanks @licy183, @Grimler91 usually tests this locally by building all packages first. Let's see what he wants to do.

@Grimler91
Copy link
Member

Would be nice to check some packages at least to make sure there aren't any surprises. Give me a week and I'll do some local ./build-all.sh.

Thanks for working on this @licy183!

@Grimler91
Copy link
Member

Have built something like 100 random (well, pseudo-random, order by ./build-all.sh) packages now. Only ndk-upgrade related issue so far has been for libpopt due to the linker flag --no-undefined-version now being default. libpopt has been fixed in c952695.

Will continue building the coming days.

@finagolfin finagolfin removed their request for review September 26, 2023 06:12
@Grimler91
Copy link
Member

Seems that several packages fail to build with same type of error as noticed for libpopt, for example json-c and libtbb. Pretty easy fix though, just have to remove symbols from version scripts.

Not related to ndk update, but seems several packages fail to build with golang 1.21 also, for example ipfs:

vendor/github.com/quic-go/quic-go/internal/qtls/go121.go:5:13: cannot use "The version of quic-go you're using can't be built on Go 1.21 yet. For more details, please see https://github.com/quic-go/quic-go/wiki/quic-go-and-Go-versions." (untyped string constant "The version of quic-go you're using can't be built on Go 1.21 yet. F...) as int value in variable declaration

but I guess we have the TERMUX_PKG_GO_USE_OLDER variable to deal with this.

Many c++ packages fail to build with new c++17 standard (and/or some other change), we have for example:

gperf:

./getline.cc:58:7: error: ISO C++17 does not allow 'register' storage class specifier [-Wregister]
      register int c = getc (stream);
      ^~~~~~~~~
1 error generated.

freeimage:

Source/OpenEXR/Imath/ImathVec.h:833:28: error: ISO C++17 does not allow dynamic exception specifications [-Wdynamic-exception-spec]
Vec4<int>::normalizeExc () throw (IEX_NAMESPACE::MathExc);
                           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Source/OpenEXR/Imath/ImathVec.h:833:28: note: use 'noexcept(false)' instead
Vec4<int>::normalizeExc () throw (IEX_NAMESPACE::MathExc);
                           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                           noexcept(false)
Source/OpenEXR/Imath/ImathVec.h:842:35: error: ISO C++17 does not allow dynamic exception specifications [-Wdynamic-exception-spec]
Vec4<int>::normalizedExc () const throw (IEX_NAMESPACE::MathExc);
                                  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Source/OpenEXR/Imath/ImathVec.h:842:35: note: use 'noexcept(false)' instead
Vec4<int>::normalizedExc () const throw (IEX_NAMESPACE::MathExc);
                                  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                                  noexcept(false)
Source/OpenEXR/Imath/ImathVec.h:1212:26: error: ISO C++17 does not allow dynamic exception specifications [-Wdynamic-exception-spec]
Vec2<T>::normalizeExc () throw (IEX_NAMESPACE::MathExc)
                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Source/OpenEXR/Imath/ImathVec.h:1212:26: note: use 'noexcept(false)' instead
Vec2<T>::normalizeExc () throw (IEX_NAMESPACE::MathExc)
                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                         noexcept(false)

libzthread:

In file included from /home/builder/.termux-build/libzthread/src/src/Condition.cxx:24:
In file included from /home/builder/.termux-build/libzthread/src/src/ConditionImpl.h:29:
/home/builder/.termux-build/libzthread/src/src/Scheduling.h:54:39: error: no template named 'binary_function' in namespace 'std'; did you mean '__binary_function'?
  struct priority_order : public std::binary_function<ThreadImpl*, ThreadImpl*, bool> {
                                 ~~~~~^~~~~~~~~~~~~~~
                                      __binary_function
/home/builder/.termux-build/_cache/android-r26-api-24-v0/bin/../sysroot/usr/include/c++/v1/__functional/binary_function.h:49:1: note: '__binary_function' declared here
using __binary_function = __binary_function_keep_layout_base<_Arg1, _Arg2, _Result>;
^

libcoinor-utils:

/home/builder/.termux-build/libcoinor-utils/src/CoinUtils/src/CoinOslFactorization2.cpp:1517:22: error: ISO C++17 does not allow 'register' storage class specifier [-Wregister]
int c_ekkbtrn_ipivrw(COIN_REGISTER3 const EKKfactinfo *COIN_RESTRICT2 fact,
                     ^~~~~~~~~~~~~~~
/home/builder/.termux-build/libcoinor-utils/src/CoinUtils/src/CoinOslFactorization2.cpp:26:24: note: expanded from macro 'COIN_REGISTER3'
#define COIN_REGISTER3 register
                       ^

mimetic:

In file included from /home/builder/.termux-build/mimetic/src/mimetic/os/file_iterator.cxx:17:
/home/builder/.termux-build/mimetic/src/mimetic/os/file_iterator.h:16:36: warning: 'iterator<std::input_iterator_tag, char>' is deprecated [-Wdeprecated-declarations]
struct ifile_iterator: public std::iterator<std::input_iterator_tag, char>
                                   ^
/home/builder/.termux-build/_cache/android-r26-api-24-v0/bin/../sysroot/usr/include/c++/v1/__iterator/iterator.h:24:29: note: 'iterator<std::input_iterator_tag, char>' has been explicitly marked deprecated here
struct _LIBCPP_TEMPLATE_VIS _LIBCPP_DEPRECATED_IN_CXX17 iterator
                            ^
/home/builder/.termux-build/_cache/android-r26-api-24-v0/bin/../sysroot/usr/include/c++/v1/__config:798:41: note: expanded from macro '_LIBCPP_DEPRECATED_IN_CXX17'
#    define _LIBCPP_DEPRECATED_IN_CXX17 _LIBCPP_DEPRECATED
                                        ^
/home/builder/.termux-build/_cache/android-r26-api-24-v0/bin/../sysroot/usr/include/c++/v1/__config:771:49: note: expanded from macro '_LIBCPP_DEPRECATED'
#      define _LIBCPP_DEPRECATED __attribute__((deprecated))
                                                ^
1 warning generated.
In file included from /home/builder/.termux-build/mimetic/src/mimetic/rfc822/header.cxx:7:
/home/builder/.termux-build/mimetic/src/mimetic/rfc822/header.h:37:21: error: no template named 'unary_function' in namespace 'std'; did you mean '__unary_function'?
        public std::unary_function<const Field, bool>
               ~~~~~^~~~~~~~~~~~~~
                    __unary_function
/home/builder/.termux-build/_cache/android-r26-api-24-v0/bin/../sysroot/usr/include/c++/v1/__functional/unary_function.h:46:1: note: '__unary_function' declared here
using __unary_function = __unary_function_keep_layout_base<_Arg, _Result>;
^

libmdbx:

FAILED: CMakeFiles/mdbx-static.dir/src/mdbx.c++.o 
/home/builder/.termux-build/_cache/android-r26-api-24-v0/bin/clang++ --target=aarch64-none-linux-android --gcc-toolchain=/home/builder/.termux-build/_cache/android-r26-api-24-v0 --sysroot=/home/builder/.termux-build/_cache/android-r26-api-24-v0/sysroot -DMDBX_BUILD_SHARED_LIBRARY=0 -DMDBX_CONFIG_H=\"/home/builder/.termux-build/libmdbx/build/config.h\" -D_HAS_EXCEPTIONS=1 -D__STDC_CONSTANT_MACROS=1 -D__STDC_FORMAT_MACROS=1 -D__STDC_LIMIT_MACROS=1 -I/home/builder/.termux-build/libmdbx/src/src -I/home/builder/.termux-build/libmdbx/build -fstack-protector-strong -Oz --target=aarch64-linux-android24  -I/data/data/com.termux/files/usr/include -fexceptions -fcxx-exceptions -frtti -fno-common -ggdb -Wno-unknown-pragmas -ffunction-sections -fdata-sections -Wall -Wextra -flto=thin -O3 -DNDEBUG -std=gnu++23 -fPIC -ffast-math -fvisibility=hidden -MD -MT CMakeFiles/mdbx-static.dir/src/mdbx.c++.o -MF CMakeFiles/mdbx-static.dir/src/mdbx.c++.o.d -o CMakeFiles/mdbx-static.dir/src/mdbx.c++.o -c /home/builder/.termux-build/libmdbx/src/src/mdbx.c++
error: invalid value 'gnu++23' in '-std=gnu++23'
note: use 'c++98' or 'c++03' for 'ISO C++ 1998 with amendments' standard
note: use 'gnu++98' or 'gnu++03' for 'ISO C++ 1998 with amendments and GNU extensions' standard
note: use 'c++11' for 'ISO C++ 2011 with amendments' standard
note: use 'gnu++11' for 'ISO C++ 2011 with amendments and GNU extensions' standard
note: use 'c++14' for 'ISO C++ 2014 with amendments' standard
note: use 'gnu++14' for 'ISO C++ 2014 with amendments and GNU extensions' standard
note: use 'c++17' for 'ISO C++ 2017 with amendments' standard
note: use 'gnu++17' for 'ISO C++ 2017 with amendments and GNU extensions' standard
note: use 'c++20' for 'ISO C++ 2020 DIS' standard
note: use 'gnu++20' for 'ISO C++ 2020 DIS with GNU extensions' standard
note: use 'c++2b' for 'Working draft for ISO C++ 2023 DIS' standard
note: use 'gnu++2b' for 'Working draft for ISO C++ 2023 DIS with GNU extensions' standard

Most of those looks like issues that should and probably will be fixed in the upstream projects sooner or laters. However the errors for libzthread and mimetic (no template named 'binary_function/unary_function' in namespace) looks ndk specific, so would be nice to have a fix for that before merging.

I'll continue doing some builds.

fornwall added a commit to termux-play-store/termux-packages that referenced this pull request Oct 2, 2023
fornwall added a commit to termux-play-store/termux-packages that referenced this pull request Oct 2, 2023
This package was last updated in 2005, and contain really old code
which is no longer used.

As no termux package depends on it, and it fails to build with android
ndk 26 (termux#17863), let's
remove it instead of trying to maintain it for little value.
@fornwall
Copy link
Member

fornwall commented Oct 2, 2023

gperf:
./getline.cc:58:7: error: ISO C++17 does not allow 'register' storage class specifier [-Wregister]
register int c = getc (stream);
^~~~~~~~~
1 error generated.

Should be fixed by #18068.

@fornwall
Copy link
Member

fornwall commented Oct 2, 2023

libzthread:

In file included from /home/builder/.termux-build/libzthread/src/src/Condition.cxx:24:
In file included from /home/builder/.termux-build/libzthread/src/src/ConditionImpl.h:29:
/home/builder/.termux-build/libzthread/src/src/Scheduling.h:54:39: error: no template named 'binary_function' in namespace 'std'; did you mean '__binary_function'?
struct priority_order : public std::binary_function<ThreadImpl*, ThreadImpl*, bool> {
~~~~~^~~~~~~~~~~~~~~
__binary_function
/home/builder/.termux-build/_cache/android-r26-api-24-> v0/bin/../sysroot/usr/include/c++/v1/__functional/binary_function.h:49:1: note: '__binary_function' declared here
using __binary_function = __binary_function_keep_layout_base<_Arg1, _Arg2, _Result>;
^

Let's remove this package, as it's from 2005 and not used. Proposed that in #18070.

@fornwall
Copy link
Member

fornwall commented Oct 2, 2023

mimetic:

Should be fixed by #18071

fornwall added a commit to termux-play-store/termux-packages that referenced this pull request Oct 2, 2023
@fornwall
Copy link
Member

fornwall commented Oct 2, 2023

libcoinor-utils

Should be fixed by #18072

@Grimler91
Copy link
Member

Nice work there @fornwall!


libsodium fails with:

libtool: warning: '-version-info/-version-number' is ignored for convenience libraries
In file included from /home/builder/.termux-build/libsodium/src/src/libsodium/crypto_aead/aes256gcm/armcrypto/aead_aes256gcm_armcrypto.c:35:
/home/builder/.termux-build/_cache/android-r26-api-24-v0/lib/clang/17/include/arm_neon.h:62829:32: error: always_inline function 'vget_high_p64' requires target feature 'crypto', but would be inlined into function 'vmull_high_p64' that is compiled without support for 'crypto'
  __ret = vmull_p64((poly64_t)(vget_high_p64(__p0)), (poly64_t)(vget_high_p64(__p1)));
                               ^
/home/builder/.termux-build/_cache/android-r26-api-24-v0/lib/clang/17/include/arm_neon.h:62829:65: error: always_inline function 'vget_high_p64' requires target feature 'crypto', but would be inlined into function 'vmull_high_p64' that is compiled without support for 'crypto'
  __ret = vmull_p64((poly64_t)(vget_high_p64(__p0)), (poly64_t)(vget_high_p64(__p1)));
                                                                ^

Grimler91 pushed a commit to termux-play-store/termux-packages that referenced this pull request Oct 2, 2023
termux-pacman-bot added a commit to termux-pacman/termux-packages that referenced this pull request Oct 2, 2023
Grimler91 pushed a commit to termux-play-store/termux-packages that referenced this pull request Oct 2, 2023
This package was last updated in 2005, and contain really old code
which is no longer used.

As no termux package depends on it, and it fails to build with android
ndk 26 (termux#17863), let's
remove it instead of trying to maintain it for little value.
WTNLXTBL added a commit to WTNLXTBL/termux-packages that referenced this pull request Oct 2, 2023
WTNLXTBL added a commit to WTNLXTBL/termux-packages-pacman that referenced this pull request Oct 2, 2023
WTNLXTBL added a commit to WTNLXTBL/termux-packages-pacman that referenced this pull request Oct 2, 2023
This package was last updated in 2005, and contain really old code
which is no longer used.

As no termux package depends on it, and it fails to build with android
ndk 26 (termux/termux-packages#17863), let's
remove it instead of trying to maintain it for little value.
Grimler91 pushed a commit to termux-play-store/termux-packages that referenced this pull request Oct 2, 2023
termux-pacman-bot added a commit to termux-pacman/termux-packages that referenced this pull request Oct 2, 2023
This package was last updated in 2005, and contain really old code
which is no longer used.

As no termux package depends on it, and it fails to build with android
ndk 26 (termux/termux-packages#17863), let's
remove it instead of trying to maintain it for little value.
termux-pacman-bot added a commit to termux-pacman/termux-packages that referenced this pull request Oct 2, 2023
WTNLXTBL added a commit to WTNLXTBL/termux-packages that referenced this pull request Oct 2, 2023
This package was last updated in 2005, and contain really old code
which is no longer used.

As no termux package depends on it, and it fails to build with android
ndk 26 (termux/termux-packages#17863), let's
remove it instead of trying to maintain it for little value.
WTNLXTBL added a commit to WTNLXTBL/termux-packages that referenced this pull request Oct 2, 2023
@Grimler91
Copy link
Member

libsodium fails with:

Hm, I can't reproduce this @Grimler91 ? Do you still see this with the current version of this branch, after setting up the NDK again after a clean?

I have rebased the branch against master and still get the error when running ./build-package.sh -i libsodium, in a fresh docker image:

  CCLD     libaesni.la
  CCLD     libsse2.la
  CCLD     libssse3.la
libtool: warning: '-version-info/-version-number' is ignored for convenience libraries
In file included from /home/builder/.termux-build/libsodium/src/src/libsodium/crypto_aead/aes256gcm/armcrypto/aead_aes256gcm_armcrypto.c:35:
/home/builder/.termux-build/_cache/android-r26-api-24-v0/lib/clang/17/include/arm_neon.h:62829:32: error: always_inline function 'vget_high_p64' requires target feature 'crypto', but would be inlined into function 'vmull_high_p64' that is compiled without support for 'crypto'
  __ret = vmull_p64((poly64_t)(vget_high_p64(__p0)), (poly64_t)(vget_high_p64(__p1)));
                               ^
/home/builder/.termux-build/_cache/android-r26-api-24-v0/lib/clang/17/include/arm_neon.h:62829:65: error: always_inline function 'vget_high_p64' requires target feature 'crypto', but would be inlined into function 'vmull_high_p64' that is compiled without support for 'crypto'
  __ret = vmull_p64((poly64_t)(vget_high_p64(__p0)), (poly64_t)(vget_high_p64(__p1)));
                                                                ^
2 errors generated.
libtool: warning: '-version-info/-version-number' is ignored for convenience libraries
make[3]: *** [Makefile:2493: crypto_aead/aes256gcm/armcrypto/libarmcrypto_la-aead_aes256gcm_armcrypto.lo] Error 1
make[3]: *** Waiting for unfinished jobs....
libtool: warning: '-version-info/-version-number' is ignored for convenience libraries
libtool: warning: '-version-info/-version-number' is ignored for convenience libraries
libtool: warning: '-version-info/-version-number' is ignored for convenience libraries
libtool: warning: '-version-info/-version-number' is ignored for convenience libraries
make[3]: Leaving directory '/home/builder/.termux-build/libsodium/build/src/libsodium'
make[2]: *** [Makefile:3397: all-recursive] Error 1
make[2]: Leaving directory '/home/builder/.termux-build/libsodium/build/src/libsodium'
make[1]: *** [Makefile:409: all-recursive] Error 1
make[1]: Leaving directory '/home/builder/.termux-build/libsodium/build/src'
make: *** [Makefile:520: all-recursive] Error 1

Pushed the branch I tested with here now

@licy183
Copy link
Member Author

licy183 commented Oct 4, 2023

Pulling sources for every NDK-related packages will make CI failed due to No space left on device. Maybe we should override termux_step_get_source and only get source if performing an on-device build.

@finagolfin
Copy link
Member

I have a pull that massively increases the space by not using Docker, #18082, maybe you can remove the line limiting it to known large packages like rust and try applying it to this pull, just to see how it goes. I want to make that the default over time, moving Docker to being used only for a few local builds by those who prefer it.

fornwall added a commit that referenced this pull request Oct 4, 2023
This fixes building on Android NDK 26, which otherwise defaults to
C++ 17. See

#17863
@Grimler91
Copy link
Member

What we can also do is to put all cached files in ~/.termux-build/_cache (or some subfolder thereof), packages built from android-ndk and for example packages building from llvm's source code then only have to download it once (if built in the same go). I'll have a go at it

termux-pacman-bot added a commit to termux-pacman/termux-packages that referenced this pull request Oct 4, 2023
This fixes building on Android NDK 26, which otherwise defaults to
C++ 17. See

termux/termux-packages#17863
WTNLXTBL added a commit to WTNLXTBL/termux-packages that referenced this pull request Oct 4, 2023
This fixes building on Android NDK 26, which otherwise defaults to
C++ 17. See

termux/termux-packages#17863
WTNLXTBL added a commit to WTNLXTBL/termux-packages-pacman that referenced this pull request Oct 4, 2023
This fixes building on Android NDK 26, which otherwise defaults to
C++ 17. See

termux/termux-packages#17863
@truboxl
Copy link
Contributor

truboxl commented Oct 11, 2023

r26b has been released: https://github.com/android/ndk/wiki/Changelog-r26#r26b

@finagolfin
Copy link
Member

@Grimler91, any progress with caching? I'll take over if you didn't get very far.

@Grimler91
Copy link
Member

@Grimler91, any progress with caching? I'll take over if you didn't get very far.

I did not, please do :)
I was thinking to keep git clones in per package cache dir still, and put all other source archives in common directory

@finagolfin
Copy link
Member

I'm not going to touch source archives or directories, only downloaded binary packages, both ours and the Ubuntu packages we need. Feel free to add whatever source caching you want to the package download caching I add once I'm done.

@Grimler91
Copy link
Member

Some packages probably still have build issues (have not test built all), but the type of issues we have seen have been solvable, so I think we can go ahead and merge this. I will do so in 10 hours or so 👍

@finagolfin
Copy link
Member

Will you kick off a mass rebuild after that? If so, please leave the swift and cookcli packages out, as Swift requires some patching for NDK 26. I have it mostly building, but need to clean it up and test it.

@Grimler91
Copy link
Member

Will you kick off a mass rebuild after that?

Not planning to, I think we said last update that it shouldn't be necessary (other than for testing purposes)

@Grimler91 Grimler91 merged commit 01f0bcd into master Oct 22, 2023
@Grimler91 Grimler91 deleted the issue-17429 branch October 22, 2023 19:05
@Grimler91
Copy link
Member

Updated to 26b and removed the extra space saving commits.

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

The LTS NDK 26 has been released
5 participants