fix(stringtheory): support big-endian platforms for `MetaString` by thieman · Pull Request #2032 · DataDog/saluki

thieman · 2026-07-06T20:12:33Z

Summary

Adds big-endian support to stringtheory::MetaString while keeping the existing optimized three-word union representation.

This is one of the Saluki-side blockers from DADP-142 / the AIX Confluence notes. After getting past process-memory, the AIX ADP build stopped at stringtheory because the previous MetaString implementation explicitly required 64-bit little-endian platforms.

Key changes

Keep MetaString on the existing compact three-word representation for both little- and big-endian 64-bit targets.
Make the discriminant/tag byte endian-aware:
- little-endian uses byte 23, preserving the existing layout and 23-byte inline capacity.
- big-endian uses byte 16, the high byte of the third machine word, with a 16-byte inline capacity so inline string bytes remain contiguous.
Preserve owned/static/interned/shared tagging through the high byte of the third usize across endianness.
Remove the separate big-endian fallback implementation in favor of the unified layout in lib/stringtheory/src/lib.rs.
Add tests for endian-specific layout constants, discriminant byte values, inline capacity boundaries, contiguous inline storage, and the public big-endian API surface.

Design notes

The optimized MetaString layout stores all variants in three machine words and determines the active variant from a single tag byte.
On little-endian targets, that tag byte is the final byte of the 24-byte layout. On big-endian targets, the high byte of the third usize is instead the first byte of the third machine word, so the tag byte index must change.
Big-endian inline capacity is intentionally reduced from 23 bytes to 16 bytes. This keeps inline string contents as a contiguous prefix, so Deref<Target = str> can still return a borrowed &str without allocating.
Owned strings remain non-cheap to clone on both endiannesses because From<String> now uses the same owned-string variant everywhere.

Continuation context

This PR is part of a three-PR Saluki-side AIX enablement stack:

fix(process-memory): support RSS queries on AIX #2031: process-memory AIX RSS support.
This PR: stringtheory big-endian MetaString support.
fix(datadog-agent-commons): add AIX platform settings #2034: datadog-agent-commons AIX platform settings.

A separate upstream/fork fix is still needed for aws-lc-rs; during validation I used the local checkout of Travis's fork/branch on the AIX host. That temporary override is not included in this PR.

Temporary AIX validation setup used on soaix499:

export PATH=/opt/freeware/lib/RustSDK/1.92/bin:/opt/freeware/bin:/usr/sbin:/usr/bin:/bin
export CC=/opt/freeware/bin/gcc-8
export CXX=/opt/freeware/bin/g++-8
export AR=/usr/bin/ar
export RANLIB=/usr/bin/ranlib
export CARGO_HOME=$HOME/dd/cargo-home
export CARGO_TARGET_DIR=$HOME/dd/saluki-target
export CARGO_REGISTRIES_CRATES_IO_PROTOCOL=sparse

Test plan

RED on AIX before implementation:
- Added temporary big-endian coverage while developing the initial fix.
- cargo test -p stringtheory --test big_endian --no-run failed at the existing little-endian compile gate.
- The final PR folds that coverage into the normal crate unit tests and removes the temporary integration test.
Local/native little-endian:
- make fmt
- cargo nextest run -p stringtheory
- Focused inline-boundary tests for the tag-byte off-by-one case:
  - cargo test -p stringtheory inlined_string_uses_every_byte_before_tag_byte -- --exact
  - cargo test -p stringtheory inlined_string_rejects_one_byte_past_tag_byte -- --exact
- cargo check --workspace
- cargo check --workspace --tests
- make check-deny
- git diff --check
Big-endian compile target:
- rustup target add s390x-unknown-linux-gnu
- cargo check -p stringtheory --target s390x-unknown-linux-gnu
- cargo check -p stringtheory --tests --target s390x-unknown-linux-gnu
AIX 7.3 (soaix499) on latest PR head 90a2aa6560 archived to ~/dd/saluki-pr2032-latest:
- cargo test -p stringtheory --lib --ignore-rust-version
- Result: 45 passed; 0 failed, including the endian-specific layout and public API unit tests.
AIX 7.3 (soaix499) with temporary local aws-lc-rs patch and the process-memory fix synced:
- cargo build -p agent-data-plane got past stringtheory and failed at the next known blocker, AIX platform constants in datadog-agent-commons.

Notes

This PR does not include the temporary local aws-lc-rs patch used for AIX validation.
The current branch commits were created with normal pre-commit checks enabled. Earlier notes about bypassing local hooks are obsolete after the rebase and follow-up commits.

pr-commenter · 2026-07-06T20:20:35Z

Binary Size Analysis (Agent Data Plane)

Baseline: f546aa0 · Comparison: 90a2aa6 · diff
Analysis Configuration: stripped binaries · Pass/Fail Threshold: +5%
Sizes: 41.66 MiB (baseline) vs 41.66 MiB (comparison)
Size Change: +464 B (+0.00%)

✅ Binary size difference within threshold

Changes by Module

Module	File Size	Symbols
`anon.5f040b9cf0e85edcd98481965dce5e0f.55.llvm.9953283433741861364`	+17.32 KiB	1
`anon.4fce6c7d2aa45dd9a6f02a2c0752f076.498.llvm.9573164220153040075`	-17.32 KiB	1
`anon.9e7fe21941eb14193087d4323bb508b7.51.llvm.18246841136514536931`	-10.25 KiB	1
`anon.9d74b0aa49281f31c2e13ca4fdb52337.65.llvm.9026379143731643429`	+10.25 KiB	1
`anon.84c792902d8f7aa6b85e3dcbd3e80db0.4.llvm.11863342121551748718`	+8.25 KiB	1
`anon.c4832a9e941eede0b3fac8601fc5dbed.149.llvm.16028478477781504075`	-8.25 KiB	1
`anon.6e13b50c2656d803ff9e7b11c69ad6e3.70.llvm.14998899457945641293`	+7.24 KiB	1
`anon.ad91e51c62512cff203a6c63aaecf20c.398.llvm.18240059250610999947`	-7.24 KiB	1
`anon.ad91e51c62512cff203a6c63aaecf20c.129.llvm.3199571197643162242`	+4.35 KiB	1
`anon.8fa54edd1cf7a386aa836d2c40430173.261.llvm.13073442000246678767`	-4.35 KiB	1
`anon.4fce6c7d2aa45dd9a6f02a2c0752f076.582.llvm.3704504571926930519`	+3.45 KiB	1
`anon.dda7c35bd9fc06cfd0b33bed524dd3ec.0.llvm.3247006006764087800`	-3.45 KiB	1
`anon.e51ff24844fb89fcc0c2bfdacceee053.227.llvm.17755263051163836990`	+3.37 KiB	1
`anon.e51ff24844fb89fcc0c2bfdacceee053.227.llvm.7818023259164723953`	-3.37 KiB	1
`anon.4fce6c7d2aa45dd9a6f02a2c0752f076.313.llvm.3704504571926930519`	+3.19 KiB	1
`anon.b4e720ff31d34137fbb65f39e9824087.393.llvm.8649721099890346063`	-3.19 KiB	1
`anon.516275e112a2fc97b662fd2b26c9d3ea.54.llvm.2088891235859587916`	-2.98 KiB	1
`anon.abd2b3bbd88016e74e1d5db97c3d2d11.109.llvm.213882830329292623`	+2.98 KiB	1
`anon.1e6cc6bdf84c04424786008b8bd85bda.5.llvm.13178757084669791416`	+2.81 KiB	1
`anon.2071d4c55a0430dc04b91d513cffeda5.9.llvm.5852619959000316956`	-2.80 KiB	1

Detailed Symbol Changes

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  [NEW] +17.3Ki  [NEW]     +81    anon.5f040b9cf0e85edcd98481965dce5e0f.55.llvm.9953283433741861364
  [NEW] +10.2Ki  [NEW]     +79    anon.9d74b0aa49281f31c2e13ca4fdb52337.65.llvm.9026379143731643429
  [NEW] +8.25Ki  [NEW]     +77    anon.84c792902d8f7aa6b85e3dcbd3e80db0.4.llvm.11863342121551748718
  [NEW] +7.24Ki  [NEW]     +79    anon.6e13b50c2656d803ff9e7b11c69ad6e3.70.llvm.14998899457945641293
  [NEW] +4.35Ki  [NEW]     +16    anon.ad91e51c62512cff203a6c63aaecf20c.129.llvm.3199571197643162242
  [NEW] +3.45Ki  [NEW]      +2    anon.4fce6c7d2aa45dd9a6f02a2c0752f076.582.llvm.3704504571926930519
  [NEW] +3.37Ki  [NEW] +3.28Ki    anon.e51ff24844fb89fcc0c2bfdacceee053.227.llvm.17755263051163836990
  [NEW] +3.19Ki  [NEW]     +89    anon.4fce6c7d2aa45dd9a6f02a2c0752f076.313.llvm.3704504571926930519
  [NEW] +2.98Ki  [NEW]     +75    anon.abd2b3bbd88016e74e1d5db97c3d2d11.109.llvm.213882830329292623
  [NEW] +2.81Ki  [NEW]     +80    anon.1e6cc6bdf84c04424786008b8bd85bda.5.llvm.13178757084669791416
  +0.0%    +454  [ = ]       0    [10359 Others]
  [DEL] -2.80Ki  [DEL]     -80    anon.2071d4c55a0430dc04b91d513cffeda5.9.llvm.5852619959000316956
  [DEL] -2.98Ki  [DEL]     -75    anon.516275e112a2fc97b662fd2b26c9d3ea.54.llvm.2088891235859587916
  [DEL] -3.19Ki  [DEL]     -89    anon.b4e720ff31d34137fbb65f39e9824087.393.llvm.8649721099890346063
  [DEL] -3.37Ki  [DEL] -3.28Ki    anon.e51ff24844fb89fcc0c2bfdacceee053.227.llvm.7818023259164723953
  [DEL] -3.45Ki  [DEL]      -2    anon.dda7c35bd9fc06cfd0b33bed524dd3ec.0.llvm.3247006006764087800
  [DEL] -4.35Ki  [DEL]     -16    anon.8fa54edd1cf7a386aa836d2c40430173.261.llvm.13073442000246678767
  [DEL] -7.24Ki  [DEL]     -79    anon.ad91e51c62512cff203a6c63aaecf20c.398.llvm.18240059250610999947
  [DEL] -8.25Ki  [DEL]     -77    anon.c4832a9e941eede0b3fac8601fc5dbed.149.llvm.16028478477781504075
  [DEL] -10.2Ki  [DEL]     -79    anon.9e7fe21941eb14193087d4323bb508b7.51.llvm.18246841136514536931
  [DEL] -17.3Ki  [DEL]     -81    anon.4fce6c7d2aa45dd9a6f02a2c0752f076.498.llvm.9573164220153040075
  +0.0%    +464  [ = ]       0    TOTAL

pr-commenter · 2026-07-06T20:36:23Z

Regression Detector (Agent Data Plane)

Run ID: 0fb64fbb-ed42-44d9-84ca-c9519633f91a
Baseline: f546aa02 · Comparison: 90a2aa65 · diff

Optimization Goals: ✅ No significant changes detected

Fine details of change detection per experiment (5)

Experiments configured erratic: true are tagged (ignored) and skipped when determining which experiments regressed or improved. Experiments which are detected as erratic at runtime are tagged (erratic) to flag that the run's sample dispersion was high, but their regression / improvement signal still counts.

experiment	goal	Δ mean %	links
quality_gates_rss_dsd_low	memory	⚪ +0.45	metrics profiles logs
quality_gates_rss_dsd_heavy	memory	⚪ +0.18	metrics profiles logs
quality_gates_rss_dsd_medium	memory	⚪ +0.09	metrics profiles logs
quality_gates_rss_idle	memory	⚪ -0.14	metrics profiles logs
quality_gates_rss_dsd_ultraheavy	memory	⚪ -0.24	metrics profiles logs

Bounds Checks: ✅ Passed (5)

experiment	check	replicates	observed	links
quality_gates_rss_dsd_heavy	memory_usage	10/10	✅ 133 MiB ≤ 140 MiB	metrics profiles logs
quality_gates_rss_dsd_low	memory_usage	10/10	✅ 43 MiB ≤ 50 MiB	metrics profiles logs
quality_gates_rss_dsd_medium	memory_usage	10/10	✅ 64.6 MiB ≤ 75 MiB	metrics profiles logs
quality_gates_rss_dsd_ultraheavy	memory_usage	10/10	✅ 192 MiB ≤ 200 MiB	metrics profiles logs
quality_gates_rss_idle	memory_usage	10/10	✅ 29 MiB ≤ 40 MiB	metrics profiles logs

Explanation

A change is flagged as a regression when |Δ mean %| > 5.00% in the regressing direction for its optimization goal AND SMP marks the experiment as a regression (is_regression: true). Improvements use the matching criteria for the improving direction. Experiments configured erratic: true (tagged (ignored)) are skipped outright; experiments detected as erratic at runtime (tagged (erratic)) still count, since that flag describes sample dispersion rather than directional certainty. The Δ mean % cell is colored accordingly: 🟢 = improvement, 🔴 = regression, ⚪ = neutral. Reduction in CPU or memory is an improvement; reduction in ingress throughput is a regression.

Add a big-endian MetaString fallback backed by shared string storage while keeping the existing tagged-pointer implementation for little-endian targets. Cover the big-endian public API with an AIX integration test and verify the ADP build moves to the next platform constants blocker.

Keep MetaString on the existing three-word union representation for both little- and big-endian 64-bit targets. The discriminant byte now follows the high byte of the third machine word, which keeps owned/static/interned/shared tagging consistent across endianness while reducing big-endian inline capacity to 16 contiguous bytes. This removes the separate big-endian fallback implementation and updates tests/docs to reflect the endian-specific inline capacity.

thieman · 2026-07-10T14:47:56Z


 // High-level invariant checks to ensure `stringtheory` isn't being used on an unsupported platform.
-#[cfg(not(all(target_pointer_width = "64", target_endian = "little")))]
+#[cfg(not(all(target_pointer_width = "64", any(target_endian = "little", target_endian = "big"))))]


any(little, big) seems extraneous, can you simplify

[GPT-5] Simplified to #[cfg(not(target_pointer_width = "64"))]. Also added focused layout tests for the endian-specific tag byte, inline capacity, and contiguous inline storage boundary.

Add unit coverage for the endian-specific MetaString discriminant byte, inline capacity, and contiguous inline storage boundary. These tests run on the native little-endian target and compile under the s390x big-endian test target. Also simplify the platform invariant check per PR feedback since the compile error only enforces the 64-bit platform requirement.

Add focused tests for the off-by-one boundary between inline string data and the endian-specific tag byte. The tests assert that strings use every byte before the tag byte and reject one byte past it.

Introduce TOP_BYTE_INDEX to make the endian-dependent byte offset explicit. MetaString uses this offset to locate the top byte of the third machine word, where the union discriminant is stored.

tobz · 2026-07-10T16:18:37Z

+
+    #[test]
+    fn inlined_string_uses_every_byte_before_tag_byte() {
+        let input = "a".repeat(INLINED_STR_TAG_INDEX);


Maybe use the length constant for clarity? (I recognize that index == length.)

[GPT-5] Updated to use INLINED_STR_MAX_LEN in the boundary tests. The tests still validate the off-by-one behavior, but now read in terms of length rather than the underlying tag-byte index.

tobz · 2026-07-10T16:19:57Z

+    #[test]
+    fn inlined_string_respects_endian_capacity() {
+        let max_len = "a".repeat(INLINED_STR_MAX_LEN);
+        let too_long = "a".repeat(INLINED_STR_MAX_LEN + 1);
+
+        assert!(MetaString::try_inline(&max_len).is_some());
+        assert!(MetaString::try_inline(&too_long).is_none());
+    }


Seems duplicative: we're already testing this above with the two individual inlining tests.

[GPT-5] Removed this duplicate test. The two explicit boundary tests cover the max accepted length and the first rejected length.

tobz · 2026-07-10T16:20:58Z

@@ -0,0 +1,55 @@
+#![cfg(target_endian = "big")]


It's ... unclear to me why this has to be a separate test. Are we somehow not testing the public API in the normal unit tests when run on a big-endian platform?

Confirm these public methods are all covered in the module unit tests and if so delete this file

[GPT-5] Confirmed the normal unit tests cover the public API on big-endian now that the implementation is unified. I moved the remaining coverage from this integration test into src/lib.rs (ordering, protobuf Chars conversions, and CheapMetaString::try_cheap_clone) and deleted this cfg-gated test file.

tobz · 2026-07-10T16:22:16Z

+    #[test]
+    fn endian_layout_constants_match_discriminant_strategy() {
+        let expected_top_byte_index = if cfg!(target_endian = "little") {
+            std::mem::size_of::<usize>() - 1
+        } else {
+            0
+        };
+        let expected_tag_index = std::mem::size_of::<usize>() * 2 + expected_top_byte_index;
+
+        assert_eq!(TOP_BYTE_INDEX, expected_top_byte_index);
+        assert_eq!(INLINED_STR_TAG_INDEX, expected_tag_index);
+        assert_eq!(INLINED_STR_MAX_LEN, expected_tag_index);
+    }


I think we can drop this? We're just restating the const definitions, and other tests should break if we mess those up somehow.

[GPT-5] Removed this test. It was restating the constant definitions rather than testing behavior; the inline capacity and raw discriminant tests cover the behavior those constants drive.

Address review feedback by removing tests that only restated constant definitions or duplicated the inline boundary checks. Keep the off-by-one coverage focused on the maximum inline length and the first rejected length.

Move the remaining public API coverage from the big-endian integration test into the crate unit tests, which run on big-endian targets now that the optimized implementation is unified. This removes the redundant cfg-gated integration test file.

## Summary Adds AIX platform defaults to `datadog-agent-commons` so platform settings resolve when building ADP on AIX. This is one of the Saluki-side blockers from DADP-142 / the AIX Confluence notes. After getting past `process-memory` and `stringtheory`, the AIX ADP build stopped in `datadog-agent-commons` because `platform/mod.rs` referenced platform constants/functions that are not defined for AIX. ## Key changes - Add `platform/aix_impl.rs` with AIX config, log, and syslog defaults. - Wire the AIX platform module into `platform/mod.rs`. - Add an AIX-only unit test for the expected platform paths. ## AIX defaults The AIX defaults match the standard Datadog Agent paths used in the AIX build notes: - Config directory: `/etc/datadog-agent` - Log directory: `/var/log/datadog` - Local syslog URI: `unixgram:///dev/log` ## Continuation context This PR is part of a three-PR Saluki-side AIX enablement stack: 1. #2031: `process-memory` AIX RSS support. 2. #2032: `stringtheory` big-endian `MetaString` fallback. 3. This PR: `datadog-agent-commons` AIX platform settings. A separate upstream/fork fix is still needed for `aws-lc-rs`; during validation I used the local checkout of Travis's fork/branch on the AIX host. That temporary override is **not** included in this PR. Temporary AIX validation setup used on `soaix499`: ```sh export PATH=/opt/freeware/lib/RustSDK/1.92/bin:/opt/freeware/bin:/usr/sbin:/usr/bin:/bin export CC=/opt/freeware/bin/gcc-8 export CXX=/opt/freeware/bin/g++-8 export AR=/usr/bin/ar export RANLIB=/usr/bin/ranlib export CARGO_HOME=$HOME/dd/cargo-home export CARGO_TARGET_DIR=$HOME/dd/saluki-target export CARGO_REGISTRIES_CRATES_IO_PROTOCOL=sparse ``` ## Test plan - RED on AIX before implementation: - Added the AIX-specific platform test. - With the proper AIX build env, `cargo test -p datadog-agent-commons platform::tests::aix_uses_standard_agent_paths --no-run` failed because AIX platform constants/functions were missing. - Local: - `cargo fmt --package datadog-agent-commons --all` - `cargo check -p datadog-agent-commons` - `cargo test -p datadog-agent-commons platform::tests` - `git diff --check` - AIX 7.3 (`soaix499`): - `cargo test -p datadog-agent-commons platform::tests::aix_uses_standard_agent_paths` - Result: `aix_uses_standard_agent_paths ... ok` - AIX 7.3 (`soaix499`) with temporary local `aws-lc-rs` patch plus #2031 and #2032 synced: - `cargo build -p agent-data-plane` - Result: completed successfully in debug profile and produced `/home/zkvhn/dd/saluki-target/debug/agent-data-plane`. - Binary identified by `file` as `64-bit XCOFF executable or object module not stripped`. ## Notes - This PR does not include the temporary local `aws-lc-rs` patch used for AIX validation. - I committed with `--no-verify` because the local hook/check path attempted broad workspace/nightly checks and was killed by the local machine while unrelated to this targeted crate change. The targeted commands above were run separately. Co-authored-by: travis.thieman <travis.thieman@datadoghq.com>

## Summary Adds AIX support to the `process-memory` crate by reading `/proc/<pid>/psinfo` and extracting the resident set size from the AIX `psinfo_t` layout. This is one of the Saluki-side blockers from DADP-142 / the AIX Confluence notes. With only the temporary `aws-lc-rs` AIX branch applied, the AIX ADP build previously stopped at `process-memory` because the crate rejected all non-Linux/macOS/Windows targets. ## Key changes - Add an AIX `Querier` backend for resident set size queries. - Include AIX in the crate's supported target list. - Parse `psinfo_t.pr_rssize` as a native-endian `uint64_t` value in KiB units, then convert to bytes with overflow checks. - Add AIX parser coverage for successful parsing, zero RSS, truncated `psinfo`, and overflow. - Add AIX-only integration coverage that compares `Querier` against the raw `/proc/<pid>/psinfo` RSS field. - Keep the `dhat` allocation test active on non-AIX targets and make the harness-free test binary a no-op on AIX, where `dhat` is intentionally not enabled. ## AIX implementation notes - AIX on `soaix499` does not expose `/proc/self/psinfo`; it exposes `/proc/<pid>/psinfo`. - IBM's AIX `/proc` documentation states that `/proc` files provide 64-bit mode-invariant data to observers and that future structure growth appends fields. - IBM documents `psinfo_t.pr_rssize` as `uint64_t pr_rssize`, resident set size in KiB (1024) units. - I verified the AIX 7.3 `psinfo_t` layout on `soaix499` with a small C program using `<sys/procfs.h>`: - `sizeof(psinfo_t) = 448` - `offsetof(psinfo_t, pr_size) = 96` - `offsetof(psinfo_t, pr_rssize) = 104` - `offsetof(psinfo_t, pr_start) = 112` - Manual magnitude validation on `soaix499` showed `Querier`, raw `pr_rssize * 1024`, and `ps v <pid>` report the same RSS magnitude. Treating `pr_rssize` as pages would report roughly 4x too high on the 4 KiB page-size test host. - The implementation is fail-closed: open/read/truncation/overflow failures return `None` rather than panicking. ## Continuation context This PR is part of a three-PR Saluki-side AIX enablement stack: 1. This PR: `process-memory` AIX RSS support. 2. #2032: `stringtheory` big-endian `MetaString` fallback. 3. #2034: `datadog-agent-commons` AIX platform settings. A separate upstream/fork fix is still needed for `aws-lc-rs`; during validation I used the local checkout of Travis's fork/branch on the AIX host. That temporary override is **not** included in this PR. ## Test plan Local: - `make fmt` - `cargo check -p process-memory` - `cargo check -p process-memory --tests` - `cargo test -p process-memory` - Direct parser test compile/run for `lib/process-memory/src/aix.rs`. AIX 7.3 (`soaix499`): - `cargo test -p process-memory` - Result: - AIX unit tests: `4 passed` - AIX integration tests: `1 passed` - `tests/no_runtime_allocations.rs` runs successfully as a no-op on AIX - Doc tests: `0 failed` - Manual RSS magnitude check: - `Querier` and raw `pr_rssize * 1024` reported about `5.9-6.1 MiB` for the test process. - `ps v <pid>` reported the same RSS magnitude. - Treating `pr_rssize` as a page count would have reported about `23.7-24.4 MiB`. ## Notes - This PR does not include the temporary local `aws-lc-rs` patch used for broader AIX validation. - The latest commits were made with the normal pre-commit hook path; the hook completed successfully, including formatting, clippy, license/advisory checks, prose checks, and API docs build. Co-authored-by: toby.lawrence <toby.lawrence@datadoghq.com>

## Human Summary Adds a script and Makefile target to build a release ADP binary on AIX. This comes with two intentional divergences from how we build on the other platforms: - We can't use `cargo auditable` because it does not support XCOFF. Clanker tried to get it to work with some manual patching but wasn't able to. Seems like this will be a more significant contribution to upstream if we want this to work. - We can't use link-time optimization (LTO) because it doesn't seem to work on AIX at all, even for a hello-world crate. This is actually called out in IBM's docs [here](https://www.ibm.com/docs/en/osfroa/1.90.0?topic=started-introducing-rustc-compiler). Neither of these should be showstoppers, but please comment if they are. The plan will be to call this script from within the Agent's `packaging/aix` machinery when we do AIX builds. Working on that follow-up now. ## Summary - Adds `make build-adp-aix` for native AIX ADP builds using the IBM Rust SDK and AIX Toolbox GCC toolchain. - Adds `ci/tooling/build-adp-aix.sh` to record the AIX build environment and run the native Cargo build. - Adds an `aix-optimized-release` Cargo profile that keeps the optimized-release binary settings that work on AIX while disabling LTO. - Checks the major toolchain versions for cargo/rustc/gcc/g++ so AIX builds fail loudly if the toolchain drifts unexpectedly. ## Notes - The AIX enablement and AWS-LC dependency changes from #2031, #2032, #2034, #2060, and #2082 are now on `main`; this PR is rebased on top of them and only carries the build helper/profile changes. - `aws-lc-sys` is built from the bumped dependency without patching Cargo's registry cache. - On the AIX test host, `gcc-8` fails in an AWS-LC compiler feature probe; `/opt/freeware/bin/gcc` (`gcc 13.3`) succeeds, so the helper defaults to that compiler. - AIX currently does not use `cargo auditable`: cargo-auditable emits ELF/generic Unix linker inputs and flags that AIX's XCOFF linker rejects. - AIX currently uses `aix-optimized-release` instead of `optimized-release`: the IBM Rust SDK 1.92 fails LTO even for a tiny hello-world crate with `failed to get bitcode from object file for LTO (Can't find section .ipa)`. - The helper still uses the same metadata inputs that affect the ADP binary (`APP_FULL_NAME`, `APP_SHORT_NAME`, `APP_IDENTIFIER`, `APP_VERSION`, `APP_GIT_HASH`, `APP_BUILD_TIME`, `APP_DEV_BUILD`) and passes `BUILD_FEATURES` through to Cargo. - Toolchain version prefixes are overrideable through `ADP_AIX_EXPECTED_*_PREFIX` variables when an intentional toolchain update occurs. ## Test Plan - [x] `bash -n ci/tooling/build-adp-aix.sh` - [x] `ADP_AIX_BUILD_DRY_RUN=true make build-adp-aix` - [x] `cargo metadata --no-deps --format-version 1` - [x] AIX: `CARGO_HOME=/opt/cargo-home-aix-adp-pr-stack CARGO_TARGET_DIR=/opt/saluki-aix-adp-pr-stack-target make build-adp-aix` - [x] AIX: `/opt/saluki-aix-adp-pr-stack-target/aix-optimized-release/agent-data-plane --help` - [x] AIX: `/opt/saluki-aix-adp-pr-stack-target/aix-optimized-release/agent-data-plane version` - [x] Pre-commit hook: `fmt`, `check-clippy`, `check-licenses`, `check-deny`, `check-docs`, `generate-api-docs` AIX toolchain observed: - `cargo 1.92.0 (IBM Open SDK for Rust on AIX 1.92.0.0)` - `rustc 1.92.0 (IBM Open SDK for Rust on AIX 1.92.0.0)` - `gcc (GCC) 13.3.0` - `g++ (GCC) 13.3.0` AIX build result: - `Finished aix-optimized-release profile [optimized + debuginfo] target(s) in 26m 22s` - Binary: `64-bit XCOFF executable or object module not stripped` Co-authored-by: travis.thieman <travis.thieman@datadoghq.com>

dd-octo-sts Bot added the area/memory Memory bounds and memory management. label Jul 6, 2026

This comment has been minimized.

Sign in to view

This was referenced Jul 6, 2026

fix(process-memory): support RSS queries on AIX #2031

Merged

fix(datadog-agent-commons): add AIX platform settings #2034

Merged

thieman force-pushed the thieman/dadp-142-stringtheory-big-endian branch from 694cc1b to 064c9fa Compare July 7, 2026 16:43

thieman commented Jul 10, 2026

View reviewed changes

thieman marked this pull request as ready for review July 10, 2026 15:07

thieman requested a review from a team as a code owner July 10, 2026 15:07

tobz requested changes Jul 10, 2026

View reviewed changes

Comment thread lib/stringtheory/src/lib.rs

thieman added 2 commits July 10, 2026 11:55

test(stringtheory): cover inline tag boundary

e17146a

Add focused tests for the off-by-one boundary between inline string data and the endian-specific tag byte. The tests assert that strings use every byte before the tag byte and reject one byte past it.

refactor(stringtheory): name endian-specific top byte index

1d2b878

Introduce TOP_BYTE_INDEX to make the endian-dependent byte offset explicit. MetaString uses this offset to locate the top byte of the third machine word, where the union discriminant is stored.

tobz requested changes Jul 10, 2026

View reviewed changes

thieman added 2 commits July 10, 2026 12:26

test(stringtheory): simplify inline boundary coverage

fc7b118

Address review feedback by removing tests that only restated constant definitions or duplicated the inline boundary checks. Keep the off-by-one coverage focused on the maximum inline length and the first rejected length.

tobz approved these changes Jul 10, 2026

View reviewed changes

tobz changed the title ~~fix(stringtheory): support big-endian MetaString~~ fix(stringtheory): support big-endian platforms for MetaString Jul 10, 2026

gh-worker-dd-devflow-36fce6 Bot added mergequeue-status: queued mergequeue-status: in_progress and removed mergequeue-status: queued labels Jul 10, 2026

gh-worker-dd-mergequeue-cf854d Bot merged commit 4d14ca1 into main Jul 10, 2026
89 checks passed

gh-worker-dd-devflow-36fce6 Bot removed the mergequeue-status: in_progress label Jul 10, 2026

gh-worker-dd-mergequeue-cf854d Bot deleted the thieman/dadp-142-stringtheory-big-endian branch July 10, 2026 19:01

gh-worker-dd-devflow-36fce6 Bot added the mergequeue-status: done label Jul 10, 2026

thieman mentioned this pull request Jul 10, 2026

chore(agent-data-plane): add AIX build helper #2084

Merged

7 tasks

Uh oh!

Conversation

thieman commented Jul 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Key changes

Design notes

Continuation context

Test plan

Notes

Uh oh!

This comment has been minimized.

pr-commenter Bot commented Jul 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Binary Size Analysis (Agent Data Plane)

✅ Binary size difference within threshold

Uh oh!

pr-commenter Bot commented Jul 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Regression Detector (Agent Data Plane)

Optimization Goals: ✅ No significant changes detected

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

thieman commented Jul 6, 2026 •

edited

Loading

pr-commenter Bot commented Jul 6, 2026 •

edited

Loading

pr-commenter Bot commented Jul 6, 2026 •

edited

Loading