Fails to build on SPARC64

[Richard-Rogalski]

Both ring 0.16.20 and the latest master branch fail to build on SPARC64. I'll include what I believe to be the relevant output.

error: failed to run custom build command for `ring v0.17.0-not-released-yet (/jdk/conduit/ring)`

Caused by:

  process didn't exit successfully: `/jdk/conduit/ring/target/debug/build/ring-12602144b6b3c18f/build-script-build` (exit status: 101)
  --- stdout
  cargo:rerun-if-env-changed=RING_PREGENERATE_ASM
  cargo:rustc-env=RING_CORE_PREFIX=ring_core_0_17_0_not_released_yet_
  OPT_LEVEL = Some("0")
  TARGET = Some("sparc64-unknown-linux-gnu")
  HOST = Some("sparc64-unknown-linux-gnu")
  CC_sparc64-unknown-linux-gnu = None
  CC_sparc64_unknown_linux_gnu = None
  HOST_CC = None
  CC = None
  CFLAGS_sparc64-unknown-linux-gnu = None
  CFLAGS_sparc64_unknown_linux_gnu = None
  HOST_CFLAGS = None
  CFLAGS = None
  CRATE_CC_NO_DEFAULTS = None
  DEBUG = Some("true")
  CARGO_CFG_TARGET_FEATURE = None

  --- stderr

  running "cc" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "-I" "include" "-I" "/jdk/conduit/ring/target/debug/build/ring-abc99723acde52b7/out" "-Wall" "-Wextra" "-std=c1x" "-Wbad-function-cast" "-Wnested-externs" "-Wstrict-prototypes" "-pedantic" "-pedantic-errors" "-Wall" "-Wextra" "-Wcast-align" "-Wcast-qual" "-Wconversion" "-Wenum-compare" "-Wfloat-equal" "-Wformat=2" "-Winline" "-Winvalid-pch" "-Wmissing-field-initializers" "-Wmissing-include-dirs" "-Wredundant-decls" "-Wshadow" "-Wsign-compare" "-Wsign-conversion" "-Wundef" "-Wuninitialized" "-Wwrite-strings" "-fno-strict-aliasing" "-fvisibility=hidden" "-fstack-protector" "-g3" "-Werror" "-c" "-o/jdk/conduit/ring/target/debug/build/ring-abc99723acde52b7/out/curve25519.o" "crypto/curve25519/curve25519.c"
  In file included from include/ring-core/mem.h:60,
                   from crypto/curve25519/curve25519.c:22:
  include/ring-core/base.h:99:2: error: #error "Unknown target CPU"
     99 | #error "Unknown target CPU"
        |  ^~~~~
  In file included from crypto/curve25519/internal.h:20,
                   from crypto/curve25519/curve25519.c:24:
  crypto/curve25519/../internal.h:178:2: error: #error "Must define either OPENSSL_32_BIT or OPENSSL_64_BIT"
    178 | #error "Must define either OPENSSL_32_BIT or OPENSSL_64_BIT"
        |  ^~~~~
  crypto/curve25519/../internal.h:191:15: error: unknown type name ‘crypto_word’






    191 | static inline crypto_word value_barrier_w(crypto_word a) {
        |               ^~~~~~~~~~~
  crypto/curve25519/../internal.h:191:43: error: unknown type name ‘crypto_word’
    191 | static inline crypto_word value_barrier_w(crypto_word a) {
        |                                           ^~~~~~~~~~~
  crypto/curve25519/../internal.h:216:15: error: unknown type name ‘crypto_word’
    216 | static inline crypto_word constant_time_msb_w(crypto_word a) {
        |               ^~~~~~~~~~~
  crypto/curve25519/../internal.h:216:47: error: unknown type name ‘crypto_word’
    216 | static inline crypto_word constant_time_msb_w(crypto_word a) {
        |                                               ^~~~~~~~~~~
  crypto/curve25519/../internal.h:221:15: error: unknown type name ‘crypto_word’
    221 | static inline crypto_word constant_time_is_zero_w(crypto_word a) {
        |               ^~~~~~~~~~~
  crypto/curve25519/../internal.h:221:51: error: unknown type name ‘crypto_word’
    221 | static inline crypto_word constant_time_is_zero_w(crypto_word a) {
        |                                                   ^~~~~~~~~~~
  crypto/curve25519/../internal.h:236:15: error: unknown type name ‘crypto_word’
    236 | static inline crypto_word constant_time_is_nonzero_w(crypto_word a) {
        |               ^~~~~~~~~~~
  crypto/curve25519/../internal.h:236:54: error: unknown type name ‘crypto_word’
    236 | static inline crypto_word constant_time_is_nonzero_w(crypto_word a) {
        |                                                      ^~~~~~~~~~~
  crypto/curve25519/../internal.h:241:15: error: unknown type name ‘crypto_word’
    241 | static inline crypto_word constant_time_eq_w(crypto_word a,
        |               ^~~~~~~~~~~
  crypto/curve25519/../internal.h:241:46: error: unknown type name ‘crypto_word’
    241 | static inline crypto_word constant_time_eq_w(crypto_word a,
        |                                              ^~~~~~~~~~~
  crypto/curve25519/../internal.h:242:48: error: unknown type name ‘crypto_word’
    242 |                                                crypto_word b) {
        |                                                ^~~~~~~~~~~
  crypto/curve25519/../internal.h:249:15: error: unknown type name ‘crypto_word’
    249 | static inline crypto_word constant_time_select_w(crypto_word mask,
        |               ^~~~~~~~~~~
  crypto/curve25519/../internal.h:249:50: error: unknown type name ‘crypto_word’
    249 | static inline crypto_word constant_time_select_w(crypto_word mask,
        |                                                  ^~~~~~~~~~~
  crypto/curve25519/../internal.h:250:52: error: unknown type name ‘crypto_word’
    250 |                                                    crypto_word a,
        |                                                    ^~~~~~~~~~~
  crypto/curve25519/../internal.h:251:52: error: unknown type name ‘crypto_word’
    12: <alloc::vec::Vec<T> as core::iter::traits::collect::FromIterator<T>>::from_iter
               at /rustc/1.62.1/library/alloc/src/vec/mod.rs:2612:9
    13: core::iter::traits::iterator::Iterator::collect
               at /rustc/1.62.1/library/core/src/iter/traits/iterator.rs:1788:9
    14: build_script_build::build_library
               at ./build.rs:502:16
    15: build_script_build::build_c_code::{{closure}}
               at ./build.rs:485:13
    16: <core::slice::iter::Iter<T> as core::iter::traits::iterator::Iterator>::for_each
               at /rustc/1.62.1/library/core/src/slice/iter/macros.rs:211:21
    17: build_script_build::build_c_code
               at ./build.rs:482:5
    18: build_script_build::ring_build_rs_main
               at ./build.rs:350:5
    19: build_script_build::main
               at ./build.rs:300:48
    20: core::ops::function::FnOnce::call_once
               at /rustc/1.62.1/library/core/src/ops/function.rs:248:5

This is also weird, as I do have CFLAGS set.

I would be more than happy to help test, or provide SSH access to a SPARC64 machine :-)

[Richard-Rogalski]

Just curious, as multiple packages I want depend on this, how hard would a port to sparc64 be?

[briansmith]

I'm going to close this as a duplicate of the new issue #1555 about porting to big-endian architectures, since once all that work is done, there shouldn't be anything else necessary for SPARC to work.

[glaubitz]

I'm going to close this as a duplicate of the new issue #1555 about porting to big-endian architectures, since once all that work is done, there shouldn't be anything else necessary for SPARC to work.

That doesn't seem to be enough, though. I just tried building ring from git and it still fails on SPARC:

  cargo:warning=In file included from /home/glaubitz/ring/include/ring-core/base.h:74,
  cargo:warning=                 from /home/glaubitz/ring/include/ring-core/mem.h:60,
  cargo:warning=                 from /home/glaubitz/ring/crypto/curve25519/curve25519.c:22:
  cargo:warning=/home/glaubitz/ring/include/ring-core/target.h:64:2: error: #error "Unknown target CPU"
  cargo:warning=   64 | #error "Unknown target CPU"
  cargo:warning=      |  ^~~~~

[Richard-Rogalski]

@glaubitz hey there!! great great work. I did a fair share of SPARC fixes and packaging on gentoo, and 95% the time, when I looked into fixing bugs or adding support, I found a bug report you made and an attached patch. Made my life much easier :)

Assuming that brian is right, and there is native code (non-asm) fallbacks, and no SIGBUSes, I can send you a patch later today that gets past that. I've had to retire my sparc box, but feel free to test

[glaubitz]

@glaubitz hey there!! great great work. I did a fair share of SPARC fixes and packaging on gentoo, and 95% the time, when I looked into fixing bugs or adding support, I found a bug report you made and an attached patch. Made my life much easier :)

That's great to hear ;-).

Assuming that brian is right, and there is native code (non-asm) fallbacks, and no SIGBUSes, I can send you a patch later today that gets past that. I've had to retire my sparc box, but feel free to test

That would be great, thanks! I would be digging into myself, but I am currently busy with too many other tasks.

As mentioned before, if you need a SPARC box, you can sign up for the GCC Compile Farm. There are currently Solaris machines on SPARC only, but the Linux machine will come back in the near future.

[glaubitz]

Any update on this? Let me know if you have a patch to test.

[Richard-Rogalski]

@glaubitz

Hey there, I'm terribly sorry, I forgot all about this and had a lot of stuff come up IRL. The following patch should work.

I recommend setting mcpu in CFLAGS and RUSTFLAGS, especially if you get a compilation error. An example would be: CFLAGS="-mcpu=ultrasparc" CXXFLAGS="-mcpu=ultrasparc" RUSTFLAGS="-C target-cpu=ultrasparc", or higher if your machine supports it (niagara4 etc). Note that the valid targets available for CFLAGS and RUSTFLAGS are not identical: the possible values for your arch should be viewable with rustc --print target-cpus

I pasted this patch in #gentoo-sparc on libera.chat (which we would extremely love if you joined!) and will PR it when you or someone there reports back with a success.

diff --git a/include/ring-core/target.h b/include/ring-core/target.h
index d58eb1195..67a6f04c6 100644
--- a/include/ring-core/target.h
+++ b/include/ring-core/target.h
@@ -60,6 +60,8 @@
 #define OPENSSL_32_BIT
 #elif defined(__s390x__)
 #define OPENSSL_64_BIT
+#elif defined(__sparc_v9__)
+#define OPENSSL_64_BIT
 #else
 #error "Unknown target CPU"
 #endif
diff --git a/mk/cargo.sh b/mk/cargo.sh
index 567e4be18..52f08dad7 100755
--- a/mk/cargo.sh
+++ b/mk/cargo.sh
@@ -30,6 +30,7 @@ qemu_powerpc64="qemu-ppc64 -L /usr/powerpc64-linux-gnu"
 qemu_powerpc64le="qemu-ppc64le -L /usr/powerpc64le-linux-gnu"
 qemu_riscv64="qemu-riscv64 -L /usr/riscv64-linux-gnu"
 qemu_s390x="qemu-s390x -L /usr/s390x-linux-gnu"
+qemu_sparc64="qemu-sparc64 -L /usr/sparc64-linux-gnu"
 
 # Avoid putting the Android tools in `$PATH` because there are tools in this
 # directory like `clang` that would conflict with the same-named tools that may
@@ -170,6 +171,13 @@ case $target in
     export CARGO_TARGET_S390X_UNKNOWN_LINUX_GNU_LINKER=s390x-linux-gnu-gcc
     export CARGO_TARGET_S390X_UNKNOWN_LINUX_GNU_RUNNER="$qemu_s390x"
     ;;
+  sparc64-unknown-linux-gnu)
+    # Ultrasparc is the sparcv9 ISA, without this -mcpu compilers can
+    # default to lower ISA versions. 
+    export CFLAGS_sparc64_unknown_linux_gnu="--sysroot=/usr/sparc64-linux-gnu -mcpu=ultrasparc"
+    export CARGO_TARGET_SPARC64_UNKNOWN_LINUX_GNU_LINKER=sparc64-linux-gnu-gcc
+    export CARGO_TARGET_SPARC64_UNKNOWN_LINUX_GNU_RUNNER="$qemu_sparc64"
+    ;;
   x86_64-unknown-linux-musl)
     use_clang=1
     # XXX: Work around https://github.com/rust-lang/rust/issues/79555.
diff --git a/mk/install-build-tools.sh b/mk/install-build-tools.sh
index ee26037ae..bbe6104c8 100755
--- a/mk/install-build-tools.sh
+++ b/mk/install-build-tools.sh
@@ -165,6 +165,12 @@ s390x-unknown-linux-gnu)
     gcc-s390x-linux-gnu \
     libc6-dev-s390x-cross
   ;;
+sparc64-unknown-linux-gnu)
+  install_packages \
+    qemu-user \
+    gcc-sparc64-linux-gnu \
+    libc6-dev-sparc64-cross
+  ;;
 wasm32-unknown-unknown)
   cargo install wasm-bindgen-cli --bin wasm-bindgen-test-runner
   use_clang=1

[glaubitz]

@glaubitz

Hey there, I'm terribly sorry, I forgot all about this and had a lot of stuff come up IRL. The following patch should work.

No worries.

I recommend setting mcpu in CFLAGS and RUSTFLAGS, especially if you get a compilation error. An example would be: CFLAGS="-mcpu=ultrasparc" CXXFLAGS="-mcpu=ultrasparc" RUSTFLAGS="-C target-cpu=ultrasparc", or higher if your machine supports it (niagara4 etc). Note that the valid targets available for CFLAGS and RUSTFLAGS are not identical: the possible values for your arch should be viewable with rustc --print target-cpus>

-mcpu=ultrasparc is the default for sparc64 anyways, so no worries ;-).

I pasted this patch in #gentoo-sparc on libera.chat (which we would extremely love if you joined!) and will PR it when you or someone there reports back with a success.

I'm there as cbmuser and also in several other Gentoo channels. We can certainly discuss the patch there, but not before tomorrow as it's already late here.

[Richard-Rogalski]

Luckily found a volunteer, 100% tests pass! Already made the PR.

For the future, already submitted an application to the GCC compile farm.

While for C/CXX ultrasparc is (hopefully) implied, seems for rust it tends to avoid using v9 instructions unless specified, last I checked. Never hurts to make sure they're explicit

And nice! Didn't know that cbmuser was you :D

[glaubitz]

Wow, nice! Thanks so much for landing this so quickly! Highly appreciated!

[briansmith]

When I was reviewing the latest PR, I saw a couple general rust-lang issues for SPARC64 that look quite relevant:

• extern "C" fn on sparc64 targets does not respect repr(transparent) rust-lang/rust#115336. This is the most concerning issue I saw, as we liberally use #[repr(transparent)] in ring. Maybe we should work to get rid of the #[repr(transparent)] stuff in ring since MIPS64 also seems to have issues, but it would be good to have somebody fix the rust-lang issue for Sparc and re-enable the relevant tests.
• [edit: NVM; it was fixed long ago.] sparc ABI issue when C program calls Rust function which returns structure rust-lang/rust#52638. This seems like less of an issue, because we only have one place where extern C code calls into Rust (IIRC), and that function doesn't return a structure.

[briansmith]

See also rust-lang/rust#113739 (comment). If there are people who are available to help maintain rustc for Sparc64, or at least answer questions, it would be helpful to head over there and volunteer.

[Richard-Rogalski]

PR updated. As for the rustc bugs relating to sparc, could be outside my zone of expertise but I can give then an honest whack-- put on my todo. For me maintainership is out of the question, but I can keep an eye out over there and answer any questions if i see em

[briansmith]

As for the rustc bugs relating to sparc, could be outside my zone of expertise but I can give then an honest whack-- put on my todo. For me maintainership is out of the question, but I can keep an eye out over there and answer any questions if i see em

That would be awesome. Here are the rust-lang/rust tests that are disabled for sparc64 that look most immediately relevant to ring: https://github.com/rust-lang/rust/blob/cfb47ca5df93c82983563fa37673f7108eb94df4/tests/ui/abi/compatibility.rs#L309-L330