Can you provide the full toolchain and execution environment? Is this metal or stimulated? Is this running in docker? I can't reproduce on a RPi yet

I have yet to dig into the details, but very broadly, that build is done inside a docker image on linux/amd64 using a cross compiler..

The repro is to do a fresh clone status-im/nimbus-eth2@137d969, then run make dist-arm64a linux machine (which sets up the docker image and cross compiler - it needs to be on a fresh clone or the host and target platforms get mixed up).

There's probably an easier way to repro by drilling down into the flags used in that build, I just haven't had time yet for a more thorough investigation

Can you link with LDFLAGS += -Wl,-verbose and post the results, or even LDFLAGS += -Wl,-Map=output.map and post the map file? This does not seem to be the same as the openssl issue, it may be as simple as moving the .rodata section closer to text. Unfortunately I need to get a hold on an ARM machine to try myself, you could try the following linker script to see if places rodata closer to text. I'm surprised such a small program triggers relocation on 21 bit masks

SECTIONS
{
  .text : {
    *(.text)
    *(.text.*)
  }

  .rodata : {
    *(.rodata)
    *(.rodata.*)
  }

  .data : {
    *(.data)
    *(SORT_BY_ALIGNMENT(.data.*))
  }
}

`

Adding: this doesn't get triggered by cross-compiling on Arch either:

$ CC=aarch64-linux-gnu-gcc AR=aarch64-linux-gnu-ar make
make -C src
make[1]: Entering directory '/home/heluani/Documents/code/ethereum/hashtree/src'
aarch64-linux-gnu-gcc -g -fpic   -c -o sha256_armv8_neon_x4.o sha256_armv8_neon_x4.S
aarch64-linux-gnu-gcc -g -fpic   -c -o sha256_armv8_neon_x1.o sha256_armv8_neon_x1.S
aarch64-linux-gnu-gcc -g -fpic   -c -o sha256_armv8_crypto.o sha256_armv8_crypto.S
aarch64-linux-gnu-gcc -g -Wall -Werror   -c -o hashtree.o hashtree.c
aarch64-linux-gnu-ar rcs libhashtree.a sha256_armv8_neon_x4.o sha256_armv8_neon_x1.o sha256_armv8_crypto.o hashtree.o
make[1]: Leaving directory '/home/heluani/Documents/code/ethereum/hashtree/src'

We faced the same issues on our native linux-arm machines (no cross compilation). https://github.com/ethpandaops/eth-client-docker-image-builder/blob/master/nimbus-eth2/Dockerfile.beacon

Job: https://github.com/ethpandaops/eth-client-docker-image-builder/actions/runs/8966477410/job/24622090024

I'm sorry all I see are nimbus specific compilation scripts and not something related to this library. Can you post me the result of the linking face with the LDFLAGS as above? I can't reproduce this neither natively compiling on ARM nor cross compiling from x86, so it's hard to rule out if this is not Nimbus or Docker related.

Update, @barnabasbusa was kind enough to give me access to the same type of box they are using in ethPandaOps, no issue here, clean compile:

root@hashtree-debug:~# git clone https://github.com/prysmaticlabs/hashtree.git
Cloning into 'hashtree'...
remote: Enumerating objects: 319, done.
remote: Counting objects: 100% (172/172), done.
remote: Compressing objects: 100% (110/110), done.
remote: Total 319 (delta 82), reused 102 (delta 40), pack-reused 147
Receiving objects: 100% (319/319), 157.37 KiB | 1.39 MiB/s, done.
Resolving deltas: 100% (179/179), done.
root@hashtree-debug:~# cd hashtree/
root@hashtree-debug:~/hashtree# make
make -C src
make[1]: Entering directory '/root/hashtree/src'
cc -g -fpic   -c -o sha256_armv8_neon_x4.o sha256_armv8_neon_x4.S
cc -g -fpic   -c -o sha256_armv8_neon_x1.o sha256_armv8_neon_x1.S
cc -g -fpic   -c -o sha256_armv8_crypto.o sha256_armv8_crypto.S
cc -g -Wall -Werror   -c -o hashtree.o hashtree.c
ar rcs libhashtree.a sha256_armv8_neon_x4.o sha256_armv8_neon_x1.o sha256_armv8_crypto.o hashtree.o
make[1]: Leaving directory '/root/hashtree/src'

Inclined on closing this issue until I get a reproducible example.

Edit: my guess will be that the Nim compilation takes the .text sections of the library and mixes it with the .text sections of Nimbus, similarly with the .rodata sections, this will definitely cause the relocation warnings since the rodata section will be pushed further from the 21 bit allows for the adr Opcodes the library uses.

Edit: my guess will be that the Nim compilation takes the .text sections of the library and mixes it with the .text sections of Nimbus, similarly with the .rodata sections, this will definitely cause the relocation warnings since the rodata section will be pushed further from the 21 bit allows for the adr Opcodes the library uses.

indeed, we embed the genesis states in the executable which means there's a lot of data in general in there - this is my suspicion as well, that the issue would only be reproducible in large binaries. 21 bits is not a lot in this context.

anyway, I intend to add more compilation info and repros soonish - just needed an open issue to start the documentation process of what's going on.

yep, here's an easy repro..

#if defined(__APPLE__)
# define cdecl(s) _##s
#else
# define cdecl(s) s
#endif

#if defined(__linux__)
    .section  .note.GNU-stack, "", @progbits
    .section .rodata,"a",@progbits
#elif defined(__APPLE__)
    .section __TEXT,__const
#elif defined(__WIN32__)
    .section .rdata,"dr"
#else
    .text
#endif

# name_data = start of data
# name_end = end of data (without alignment)
# name = 64-bit pointer to data
# name_size = 64-bit length in bytes

eth2_mainnet_genesis_data:
    .incbin "genesis.ssz"
eth2_mainnet_genesis_end:
    .global cdecl(eth2_mainnet_genesis_size)
    .p2align 3
cdecl(eth2_mainnet_genesis_size):
    .quad    eth2_mainnet_genesis_end - eth2_mainnet_genesis_data

put this file in src as metadata.S together with https://github.com/eth-clients/eth2-networks/blob/master/shared/mainnet/genesis.ssz, then run gcc -shared -o libhashtree.so *.S to repro.

gcc test.c -fPIC mainnet.S libhashtree.a similarly fails, using the above example and the static lib built by the makefile.

gcc test.c -fPIC mainnet.S libhashtree.a similarly fails, using the above example and the static lib built by the makefile.

This seems to work with gcc test.c -fPIC libhashtree.a mainnet.S so there may be a way to solve this at the linking stage instead of touching the assembly.