Can't cross-compile OTP/Erlang 25.3

[SergMartidi]

Describe the bug
Tried to cross-compile OTP/Erlang 25.3 but make returns an error.

To Reproduce
$ ./configure --enable-bootstrap-only
$ make
$ ./configure --build=x86_64-linux-gnu --host=arm-linux-gnueabi --disable-dynamic-ssl-lib --with-ssl=/var/lib/cross/targets/arm/sysroot/usr erl_xcomp_sysroot=/var/lib/cross/targets/arm/sysroot/usr --disable-smp-require-native-atomics
$ make

And I got an error:
In file included from beam/erl_process.h:2924,
from beam/beam_code.h:25,
from beam/module.h:25,
from beam/global.h:34,
from sys/unix/erl_main.c:25:
beam/erl_process_lock.h: In function 'erts_proc_lock_wait_until_released':
beam/erl_process_lock.h:994: error: 'pix_lck' undeclared (first use in this function)
beam/erl_process_lock.h:994: error: (Each undeclared identifier is reported only once
beam/erl_process_lock.h:994: error: for each function it appears in.)

Affected versions
OTP 25.3

Additional context
System info:
Linux 2.6.31.8
armv5tel
arm-linux-gnueabi
gcc-4.2.2

I've been already succesfully using OTP 20.0 on my ARM.
And now I've decided to update OTP.
I've succesfully built OTP 24.0 without smp-support and hipe but I can't build OTP 25.3
I used option: --disable-smp-require-native-atomics
Because without this option I can't even configure project.
My compiler gcc-4.2.2 hasn't gcc __atomic_* builtins.
And during configuration gcc __sync_* builtins also wasn't found.
I tried to build with libatomic_ops but I couldn't configure project.
I got an error: ' no compare and swap'
So I configured with option: --disable-smp-require-native-atomics
Earlier I succesfully built OTP 20.0 with cross-compiler 4.9.4 with smp-support but I couldn't start it on my ARM.
I got message:
"A newer kernel is required to run this binary. (__kernel_cmpxchg64 helper)"
So it seems my ARM doesn't support this function.

Last information:
This problem begins from OTP 25.0
OTP 24.3.4.10 is built successsfully

[rickard-green]

And I got an error:
In file included from beam/erl_process.h:2924,
from beam/beam_code.h:25,
from beam/module.h:25,
from beam/global.h:34,
from sys/unix/erl_main.c:25:
beam/erl_process_lock.h: In function 'erts_proc_lock_wait_until_released':
beam/erl_process_lock.h:994: error: 'pix_lck' undeclared (first use in this function)
beam/erl_process_lock.h:994: error: (Each undeclared identifier is reported only once
beam/erl_process_lock.h:994: error: for each function it appears in.)

I'm able to build without atomics, so I'm not able to reproduce this.

However, you really do not want to do this. Performance will be horrible. We also do not have any test machines without native atomics support, so it might be/become buggy without us noticing.

I tried to build with libatomic_ops but I couldn't configure project.

If you are building from the git repository, you need to run autogen.sh in the repository in order to get the configure script. Have you done that?

Earlier I succesfully built OTP 20.0 with cross-compiler 4.9.4 with smp-support but I couldn't start it on my ARM.
I got message:
"A newer kernel is required to run this binary. (__kernel_cmpxchg64 helper)"
So it seems my ARM doesn't support this function.

The cross tool-chain and configuration you're using must match the target OS and hardware. If not, you'll end up with issues like this.

I'd recommend upgrading both compiler and linux used since they are ancient. If not, you should at least use libatomic_ops for native atomics.

[SergMartidi]

If you are building from the git repository, you need to run autogen.sh in the repository in order to get the configure
script. Have you done that?

I'm building from https://www.erlang.org/downloads. There was configure file. It was not needed to run .autogen

I'd recommend upgrading both compiler and linux used since they are ancient. If not, you should at least use
libatomic_ops for native atomics.

It's a pity but I can't do this. I use Erlang in Switches. It is not compatible with hardware and library that connect to this hardware. So I have to use my version of linux.

However, you really do not want to do this. Performance will be horrible. We also do not have any test machines
without native atomics support, so it might be/become buggy without us noticing.

Earlier several years we use OTP 20 in our switches, which was configured with options --no-smp-support --no-hipe and all was ok.
Do you mean that renew OTP version from 20 with options --no-smp-support --no-hipe to 24 with option --disable-smp-require-native-atomics may lead to performance degradation and bugs? It is very important question for us. So we must decide to renew OTP version on our switches or no.

[rickard-green]

After looking closer at this I found the bug that caused your build to fail. A fix can be found in #7159. It will be released in the next patch on OTP 25. The build did not fail for me as it did for you since the configuration flag for disabling native atomics (and other stuff) behaves differently than if no native atomics could be found by the configure script.

If you are building from the git repository, you need to run autogen.sh in the repository in order to get the configure
script. Have you done that?

I'm building from https://www.erlang.org/downloads. There was configure file. It was not needed to run .autogen

I thought you had failed to configure libatomic_ops which needs execution of autogen.sh if built from repository. Looking closer at this I see that libatomic_ops is pointless here since armv5 lack instructions for implementing cmpxchg operations so it will do no good.

I'd recommend upgrading both compiler and linux used since they are ancient. If not, you should at least use
libatomic_ops for native atomics.

It's a pity but I can't do this. I use Erlang in Switches. It is not compatible with hardware and library that connect to this hardware. So I have to use my version of linux.

If I interpret various information on the net correct a newer gcc (v4.7 or newer) should implement the _atomic* API for armv5 using a fallback in the Linux kernel. This fallback is most likely the __kernel_cmpxchg64 helper that you got an error message about when trying to execute the binary that you compiled with gcc 4.9.4. By upgrading to a newer linux kernel this will most likely work on your hardware. This will also probably perform much better than building with the --disable-smp-require-native-atomics configure flag.

However, you really do not want to do this. Performance will be horrible. We also do not have any test machines
without native atomics support, so it might be/become buggy without us noticing.

Earlier several years we use OTP 20 in our switches, which was configured with options --no-smp-support --no-hipe and all was ok. Do you mean that renew OTP version from 20 with options --no-smp-support --no-hipe to 24 with option --disable-smp-require-native-atomics may lead to performance degradation and bugs? It is very important question for us. So we must decide to renew OTP version on our switches or no.

The runtime system without SMP support was removed as of OTP 21. Without native atomic support there will be a performance degradation. How much is hard to tell. On a system with only one scheduler the performance loss will be less than on a system with multiple schedulers, though. We do not run any tests at all on platforms without native atomic support, so bugs introduced on such platforms might go unnoticed by us (like the bug fixed by #7159).

OTP 20 has not been supported for three years. That is, it receives no bug fixes at all, so I don't think it is a good solution to stay on OTP 20.

[SergMartidi]

Thank you very much for your answer!!!
This is very useful information for us

[rickard-green]

The OTP 25.3.1 patch containing #7159 has been released now.