Support QNX 7.1 with `io-sock`+libstd and QNX 8.0 (`no_std` only) #133631

flba-eb · 2024-11-29T14:05:23Z

Changes of this pull request:

Refactor code for qnx nto targets to share more code in file nto_qnx.rs
Add support for an additional network stack on nto qnx 7.1.

QNX 7.1 supports two network stacks:
1. io-pkt, which is default
2. io-sock, which is optional on 7.1 but default in QNX 8.0
As one can see in the io-sock migration notes, this changes the libc API in a way similar to e.g. linux-gnu vs. linux-musl.

This change adds a new target which has a different value for target_env, so that e.g. libc can distinguish between both APIs.
Add initial support for QNX 8.0, thanks @AkhilTThomas. As it turned out, the problem with forking many processes still exists in QNX 8.0. Because if this, we are now using it for any QNX version (i.e. not check for target_env anymore).

@rustbot label +O-neutrino
CC: @jonathanpallant @japaric @gh-tr @AkhilTThomas

rustbot · 2024-11-29T14:05:31Z

rustbot has assigned @pnkfelix.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

jonathanpallant · 2024-11-29T14:20:41Z

So, first up, excellent! The AWS QNX Neutrino 7.1.0 AMIs appears to use iosock and so I have to keep copying over libsocket.so.3 to run Rust code there. With this change hopefully I won't.

However, I note that there are currently no five-part-triples in the target list, so this would be the first. Perhaps aarch64-unknown-nto-qnx710_iosock would be better.

Will there be a x86_64-unknown-nto-qnx710_iosock to match?

flba-eb · 2024-11-29T14:26:54Z

So, first up, excellent! The AWS QNX Neutrino 7.1.0 AMIs appears to use iosock and so I have to keep copying over libsocket.so.3 to run Rust code there. With this change hopefully I won't.

Beware that version 3 and 4 have different APIs -- they may seem to work, but there might be subtle issues up to undefined behavior.

However, I note that there are currently no five-part-triples in the target list, so this would be the first. Perhaps aarch64-unknown-nto-qnx710_iosock would be better.

Very good idea -- I found it confusing that target_env uses an underscore while the target name does not. I will rename it!

Will there be a x86_64-unknown-nto-qnx710_iosock to match?

Not planned so far, but should be quite easy to do. How big is the need for it?

jonathanpallant · 2024-11-29T14:29:28Z

How big is the need for it?

No idea. Who was asking for this target?

src/doc/rustc/src/platform-support/nto-qnx.md

jonathanpallant · 2024-11-29T15:16:25Z

I was thinking to use an obvious dummy value rather than panicking if the environment variable isn't set. But maybe the project people have a better idea.

Are there another other targets that need to read environment variables?

flba-eb · 2024-12-18T08:45:57Z

Updated the nto-qnx.md document to reflect changes done by #134211 (and rebased).
@workingjubilee I would appreciate very much if you could have a quick look if there is something obvious which needs to be fixed. Changing this PR this week would be easier for me than around Christmas and new years eve.

bors · 2024-12-23T12:58:39Z

☔ The latest upstream changes (presumably #134677) made this pull request unmergeable. Please resolve the merge conflicts.

bors · 2024-12-27T16:27:21Z

☔ The latest upstream changes (presumably #134822) made this pull request unmergeable. Please resolve the merge conflicts.

AkhilTThomas · 2025-01-04T18:56:11Z

src/doc/rustc/src/platform-support/nto-qnx.md

-    AR_x86_64_pc_nto_qnx710=ntox86_64-ar'
+    - `CC_<target>=qcc`
+    - `CFLAGS_<target>=<nto_cflag>`
+    - `CXX_<target>=qcc`


'AR_<target>=<nto<arch>-ar' is missed here as well as in the example below

This should not be required anymore on recent master of Rust (at least to compile libstd). Do you observe issues on this branch?

I faced this issue while compiling this branch against the latest cc-rs. Since rust-lang/cc-rs#1319 is merged i followed the new build steps.
Is this change missed in the cc-rs ?
Similar to the modified get_base_compiler shouldn't the get_base_archiver modified to return qcc

It should work without setting the archiever, at least it worked for me 😄 . To not risk delaying this PR further, I've restored the previous documentation, it does not hurt to still set the env variable.

workingjubilee

why does a networking stack change require an entirely new target? I'm not seeing any library code changes that justify it?

flba-eb · 2025-01-05T09:27:48Z

why does a networking stack change require an entirely new target? I'm not seeing any library code changes that justify it?

Good point. Sorry for missing out to link to the prepared libc changes: A lot of (networking related) constants have different values, i.e. libstd requires a recompilation at least. In libc, we need to distinguish between the QNX versions, including the network stack 😒 . The versions are not ABI compatible. Also, we need to specify an additional linking search path when using io-sock.

workingjubilee · 2025-01-05T09:37:04Z

Lovely.

Thank you for the additional context and your patience, I was very busy in December unfortunately so didn't get around to a lot. Will try to review this soon.

…rk stack on aarch64 Signed-off-by: Florian Bartels <Florian.Bartels@elektrobit.com>

Signed-off-by: Florian Bartels <Florian.Bartels@elektrobit.com>

AkhilTThomas · 2025-01-11T12:27:20Z

Also, heads up - tomorrow is my last day of work in 2024, so if I can't do a build tomorrow, it'll have to wait until next year or you'll have to find other people with licences to verify this target change.

@jonathanpallant Hope you're back from vacation. Could you take a look at this PR ?

jonathanpallant · 2025-01-13T09:58:30Z

I have build issues on QNX Neutrino 7.0, which I think is because libc needs updating.
I cannot build QNX Neutrino 7.1 because my licence expired.
I cannot build QNX Neutrino 8.0 because macOS support has been dropped.

But, as I agreed with Florian before Christmas, we should get this merged because the targets existing is a blocker on libc and cc-rs changes.

jonathanpallant · 2025-01-13T11:06:07Z

Build command:

#!/bin/bash
source ~/qnx700/qnxsdp-env.sh
export build_env='
         CC_aarch64_unknown_nto_qnx700=qcc
        CXX_aarch64_unknown_nto_qnx700=qcc
     CFLAGS_aarch64_unknown_nto_qnx700=-Vgcc_ntoaarch64le_cxx
         AR_aarch64_unknown_nto_qnx700=ntoaarch64-ar
         CC_i586_pc_nto_qnx700=qcc
        CXX_i586_pc_nto_qnx700=qcc
     CFLAGS_i586_pc_nto_qnx700=-Vgcc_ntox86_cxx
         AR_i586_pc_nto_qnx700=ntox86-ar
'

env $build_env ./x.py build \
  --target aarch64-apple-darwin,aarch64-unknown-nto-qnx700,i586-pc-nto-qnx700 \
  rustc library/core library/alloc library/std src/tools/remote-test-server

Error:

...
Building stage2 library artifacts {alloc, core, std} (aarch64-apple-darwin -> aarch64-unknown-nto-qnx700)
    Finished `release` profile [optimized] target(s) in 0.02s
Building stage2 library artifacts {alloc, core, std} (aarch64-apple-darwin -> i586-pc-nto-qnx700)
error[E0432]: unresolved imports `crate::c_char`, `crate::c_long`, `crate::c_ulong`
   --> /Users/jonathan/.cargo/registry/src/index.crates.io-6f17d22bba15001f/libc-0.2.169/src/macros.rs:84:17
    |
65  | / macro_rules! prelude {
66  | |     () => {
67  | |         /// Frequently-used types that are available on all platforms
68  | |         ///
...   |
84  | |                 c_char, c_double, c_float, c_int, c_long, c_longlong, c_short, c_uchar, c_uint,
    | |                 ^^^^^^                            ^^^^^^ no `c_long` in the root
    | |                 |
    | |                 no `c_char` in the root
    | |                 help: a similar name exists in the module: `c_schar`
85  | |                 c_ulong, c_ulonglong, c_ushort, c_void, intptr_t, size_t, ssize_t, uintptr_t,
    | |                 ^^^^^^^ no `c_ulong` in the root
...   |
88  | |     };
89  | | }
    | |_- in this expansion of `prelude!`
    |
   ::: /Users/jonathan/.cargo/registry/src/index.crates.io-6f17d22bba15001f/libc-0.2.169/src/lib.rs:138:9
    |
138 |           prelude!();
    |           ---------- in this macro invocation
    |
    = help: consider importing this type alias instead:
            rustc_std_workspace_core::ffi::c_char
    = help: consider importing this type alias instead:
            rustc_std_workspace_core::ffi::c_long
    = help: consider importing this type alias instead:
            rustc_std_workspace_core::ffi::c_ulong

jonathanpallant · 2025-01-13T11:15:17Z

I think that's a libc bug. I'll open an issue for it.

Edit: rust-lang/libc#4241

QNX SDP 8.0 comes with newly renamed QNX OS 8.0, so update the page to talk about QNX, QNX Neutrino 7.0, QNX Neutrino 7.1 or QNX OS 8.0. Also actually add a list of target triples.

jonathanpallant · 2025-01-13T14:59:07Z

I'm proposing some wording changes to the nto-qnx.md README now that QNX are calling the new version QNX OS 8.0 (dropping the 'Neutrino' label that 7.1 and earlier versions had):

https://github.com/jonathanpallant/rust/tree/add_nto_qnx71_iosock_support_readme_updates

@flba-eb do you want to merge them into this PR, or should we do that and then I'll do that separately?

Edit: also the build failures above are simply because there's no libstd support for i586-pc-nto-qnx700 - it's libcore only, as noted in the support table.

flba-eb · 2025-01-15T11:59:04Z

I'm proposing some wording changes to the nto-qnx.md README now that QNX are calling the new version QNX OS 8.0 (dropping the 'Neutrino' label that 7.1 and earlier versions had):

https://github.com/jonathanpallant/rust/tree/add_nto_qnx71_iosock_support_readme_updates

@flba-eb do you want to merge them into this PR, or should we do that and then I'll do that separately?

Edit: also the build failures above are simply because there's no libstd support for i586-pc-nto-qnx700 - it's libcore only, as noted in the support table.

I would love to do this separately in case it helps getting this PR merged sooner, but if there are no review comments in the next few days I will probably merge your improvements.

workingjubilee · 2025-01-15T15:57:13Z

I would be happier to see the correct nomenclature in place.

workingjubilee · 2025-01-15T15:57:40Z

I must confess I am an insufferable pedant at heart.

flba-eb · 2025-01-15T16:30:25Z

I would be happier to see the correct nomenclature in place.

If this is about the change proposed by @jonathanpallant : It is now merged and pushed 😄 .

jonathanpallant · 2025-01-15T17:06:25Z

I must confess I am an insufferable pedant at heart.

As is the Rust compiler, so we're all in good company :)

workingjubilee · 2025-01-17T21:10:34Z

src/doc/rustc/src/platform-support/nto-qnx.md

@@ -18,30 +20,29 @@ and [Blackberry QNX][BlackBerry].

 ## Requirements

-Currently, the following QNX Neutrino versions and compilation targets are supported:
+Currently, the following QNX versions and compilation targets are supported:


Maybe the target_os should be migrated to "qnx"... (we don't require the target tuple and the target_os to line up perfectly)

just a random thought right now, I won't blockade anything on that.

workingjubilee · 2025-01-17T21:18:04Z

library/std/Cargo.toml

@@ -139,7 +139,8 @@ test = true
 level = "warn"
 check-cfg = [
    'cfg(bootstrap)',
-    'cfg(target_arch, values("xtensa"))',
+    'cfg(target_arch, values("xtensa", "aarch64-unknown-nto-qnx710_iosock"))',


...this seems incorrect?

workingjubilee · 2025-01-17T21:23:42Z

compiler/rustc_target/src/spec/targets/aarch64_unknown_nto_qnx710_iosock.rs

+        Some("ARM64 QNX Neutrino 7.1 RTOS with io-sock network stack".into());
+    target.options.pre_link_args =
+        nto_qnx::pre_link_args(nto_qnx::ApiVariant::IoSock, nto_qnx::Arch::Aarch64);
+    target.options.env = "nto71_iosock".into();


I feel it is almost a misuse of the field but we could use the target_abi for the iosock distinction, or a target_family even? All of these targets are slightly Wrong, because they should be using os_version_min instead, but as that is an RFC and not implemented yet, and thus these targets are defined by crude hackery, I feel like some more crude hackery might be acceptable? It seems incorrect to have to #[cfg(any(target_env = "nto71", target_env = "nto71_iosock"))] everywhere the iosock change doesn't matter.

library/std/src/sys/pal/unix/process/process_unix.rs

workingjubilee · 2025-01-17T21:25:51Z

src/doc/rustc/src/platform-support/nto-qnx.md

-```
+    ```toml
+    profile = "compiler"
+    change-id = 115898


tbh I usually just use

change-id = 999999

workingjubilee · 2025-01-17T21:32:34Z

compiler/rustc_target/src/spec/base/nto_qnx.rs

+        .unwrap_or_else(|_| "QNX_TARGET_not_set_please_source_qnxsdp-env.sh".into());
+    let linker_param = format!("-L{target_dir}/{arch_lib_dir}/io-sock/lib");
+
+    linker_param.leak()


fixing this is not necessarily in-scope for this PR, but

Suggested change

linker_param.leak()

// FIXME: leaking this is kind of weird: we're feeding these into something that expects an `AsRef<OsStr>`, but often converts to `OsString` anyways, so shouldn't we just demand an `OsString`?

linker_param.leak()

workingjubilee · 2025-01-17T21:36:51Z

compiler/rustc_target/src/spec/base/nto_qnx.rs

+        llvm_target: "x86_64-pc-unknown".into(),
+        metadata: meta(),
+        pointer_width: 64,
+        data_layout:
+            "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-i128:128-f80:128-n8:16:32:64-S128".into(),
+        arch: "x86_64".into(),
+        options: TargetOptions {
+            cpu: "x86-64".into(),
+            plt_by_default: false,
+            max_atomic_width: Some(64),


does QNX really support AMD Phenoms?

rustbot assigned pnkfelix Nov 29, 2024