aarch64 / arm64 support

[myhro]

Hi,

I was using xh on a Raspberry Pi 4 which was recently switched to the Raspberry Pi OS arm64 release. After that, the arm-unknown-linux-gnueabihf release doesn't work anymore:

$ uname -a
Linux raspberrypi 5.10.63-v8+ #1488 SMP PREEMPT Thu Nov 18 16:16:16 GMT 2021 aarch64 GNU/Linux
$ cd xh-v0.14.1-arm-unknown-linux-gnueabihf/
$ ls -lh
total 6.1M
drwxr-xr-x 2 pi pi 4.0K Nov 26 20:35 completions
drwxr-xr-x 2 pi pi 4.0K Nov 26 20:35 doc
-rw-r--r-- 1 pi pi 1.1K Nov 26 20:35 LICENSE
-rw-r--r-- 1 pi pi 9.7K Nov 26 20:35 README.md
-rwxr-xr-x 1 pi pi 6.1M Nov 26 20:35 xh
$ ./xh
bash: ./xh: No such file or directory

Would it be possible to also release aarch64/arm64 binaries?

Regards,
Tiago.

[jihchi]

Hi @myhro ,

It looks like you haven't added "execute" permission to xh, for example: chmod +x ./xh, is this the case?

[myhro]

Hi @jihchi,

Thank you for the suggestion. Unfortunately that's not the case.

As you can see in the output copied above, the xh binary has -rwxr-xr-x (755) permissions.

[blyxxyz]

I think the problem is that the executable is dynamically linked:

$ file xh
xh: ELF 32-bit LSB pie executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 3.2.0, stripped

I don't have a system to test, but these commands might fix it:

sudo dpkg --add-architecture armhf
sudo apt update
sudo apt install libc6:armhf

It would be nice to not have to do that. We may be able to manage that by building with musl (arm-unknown-linux-musleabihf) so the binary is statically linked.

Starting from 2023 ARM processors apparently won't have 32-bit support at all, so by that point we will need a separate 64-bit build. But for now I think we can still get by with just the one ARM build.

[ducaale]

It would be nice to not have to do that. We may be able to manage that by building with musl (arm-unknown-linux-musleabihf) so the binary is statically linked.

Sounds good. A Binary built with this target also seems to run fine on termux, except that it fails to establish any connection

$ ./xh httpbin.org/json
xh: error: error sending request for url (http://httpbin.org/json): error trying to connect: dns error: failed to lookup address information: Try again

[myhro]

I don't have a system to test, but these commands might fix it:

Indeed. Just tried in a container on the same machine and the problem was fixed:

root@e8e9364562eb:~/xh-v0.14.1-arm-unknown-linux-gnueabihf# ./xh
bash: ./xh: No such file or directory
root@e8e9364562eb:~/xh-v0.14.1-arm-unknown-linux-gnueabihf# dpkg --add-architecture armhf
root@e8e9364562eb:~/xh-v0.14.1-arm-unknown-linux-gnueabihf# apt update
Hit:1 http://deb.debian.org/debian bullseye InRelease
Hit:2 http://security.debian.org/debian-security bullseye-security InRelease
Hit:3 http://deb.debian.org/debian bullseye-updates InRelease
Get:4 http://deb.debian.org/debian bullseye/main armhf Packages [7943 kB]
(...)
root@e8e9364562eb:~/xh-v0.14.1-arm-unknown-linux-gnueabihf# apt install libc6:armhf
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
  gcc-10-base:armhf krb5-locales libcom-err2:armhf libcrypt1:armhf libgcc-s1:armhf libgssapi-krb5-2:armhf libidn2-0:armhf libk5crypto3:armhf libkeyutils1:armhf
  libkrb5-3:armhf libkrb5support0:armhf libnsl2:armhf libnss-nis:armhf libnss-nisplus:armhf libssl1.1:armhf libtirpc3:armhf libunistring2:armhf
(...)
root@e8e9364562eb:~/xh-v0.14.1-arm-unknown-linux-gnueabihf# ./xh httpbin.org/json
HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: *
Connection: keep-alive
Content-Length: 429
Content-Type: application/json
Date: Sun, 19 Dec 2021 11:49:02 GMT
Server: gunicorn/19.9.0
(...)

It would be nice to not have to do that.

Definitely. This reminds me having to add libc6:i386 to run proprietary 32-bit-only binaries on amd64 machines in a distant past.

We may be able to manage that by building with musl (arm-unknown-linux-musleabihf) so the binary is statically linked.

I don't have that much experience building Rust projects, but I'll play around with different targets to see if I can find a suitable option.

Starting from 2023 ARM processors apparently won't have 32-bit support at all, so by that point we will need a separate 64-bit build. But for now I think we can still get by with just the one ARM build.

Is there a reason to avoid an additional ARM build for now other than aiming to maintain less build targets? My suggestion is based on what I've seen other big Rust projects doing, like bat and bottom, where aarch64-unknown-linux-gnu binaries are released.

[myhro]

I don't have that much experience building Rust projects, but I'll play around with different targets to see if I can find a suitable option.

I've used cross to build the project for both aarch64-unknown-linux-gnu and arm-unknown-linux-musleabihf targets. This is the result:

$ cd aarch64-unknown-linux-gnu/
$ file xh
xh: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically
linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0,
BuildID[sha1]=0b6bc40c379bced39b53771c2ff012a246b8032b, with debug_info, not
stripped
$ ./xh httpbin.org/json
HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: *
Connection: keep-alive
Content-Length: 429
Content-Type: application/json
Date: Sun, 19 Dec 2021 13:30:15 GMT
Server: gunicorn/19.9.0
(...)

$ cd arm-unknown-linux-musleabihf/
$ file xh
xh: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked,
with debug_info, not stripped
$ ./xh httpbin.org/json
HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: *
Connection: keep-alive
Content-Length: 429
Content-Type: application/json
Date: Sun, 19 Dec 2021 13:32:16 GMT
Server: gunicorn/19.9.0
(...)

So both options work, although I'm inclined towards aarch64-unknown-linux-gnu for being the "native" one. Putting myself in the shoes on an end-user who haven't read this issue, it would be easier to differentiate between the former and arm-unknown-linux-gnueabihf targets, which is too similar to arm-unknown-linux-musleabihf. There's no hint the musl-based one would work on aarch64/arm64 hosts, for instance.

Do we need to test something else? Otherwise I can send a PR adding adding the architecture and its build options to test/deploy matrixes on GitHub Actions workflows.

[ducaale]

Sadly, aarch64-unknown-linux-gnu doesn't work on termux because the resulting binary becomes dynamically linked.

How about replacing arm-unknown-linux-gnueabihf with arm-unknown-linux-musleabihf and maybe adding aarch64-unknown-linux-musl?

[myhro]

Sadly, aarch64-unknown-linux-gnu doesn't work on termux because the resulting binary becomes dynamically linked.

Sorry, but I may be misunderstanding the effects on Termux. By the "A Binary built with this target also seems to run fine on termux, except that it fails to establish any connection" reply, I understood that it also doesn't play along with arm-unknown-linux-musleabihf.

Or are we considering "being a valid executable binary" and "connects to the target and works as expected" to be separate problems? Not sure if the name resolution problem was caused by musl or not.

How about replacing arm-unknown-linux-gnueabihf with arm-unknown-linux-musleabihf and maybe adding aarch64-unknown-linux-musl?

This looks reasonable to me. The only downside I see from replacing gnueabihf with musleabihf is that people who might have scripted/automated how the binaries are downloaded will have to fix the target name in future releases. I've been doing that myself, but there's no problem fixing it.

[ducaale]

Or are we considering "being a valid executable binary" and "connects to the target and works as expected" to be separate problems? Not sure if the name resolution problem was caused by musl or not.

Yes, I think this happens because /etc/resolv.conf is not available in termux but I could be wrong. See nojima/httpie-go#64 and termux/termux-packages#1174.

Edit: I got it to work on termux by running termux-chroot. See https://wiki.termux.com/wiki/Differences_from_Linux.

This looks reasonable to me. The only downside I see from replacing gnueabihf with musleabihf is that people who might have scripted/automated how the binaries are downloaded will have to fix the target name in future releases. I've been doing that myself, but there's no problem fixing it.

Ah, I haven't considered that. In that case, let's just add aarch64-unknown-linux-musl for now.

[alexanderadam]

@xeffyr just mentioned that /data/data/com.termux/files/user/etc/resolv.conf can simply be used instead.

[ducaale]

Once reqwest adds support passing custom config to trust-dns-resolver, I think we will be able to modify the path to system DNS configuration from /etc/resolv.conf to /data/data/com.termux/files/user/etc/resolv.conf.

[ducaale]

The latest release (v0.15.0) now includes builds for aarch64.