Skip to content

Fix and harden edlcodegen Rust struct/wstring out-param stubs - #207

Merged
Branden Bonaby (bbonaby) merged 1 commit into
mainfrom
user/gudge/edlcodegen/fix-struct-out-param
Jul 25, 2026
Merged

Fix and harden edlcodegen Rust struct/wstring out-param stubs#207
Branden Bonaby (bbonaby) merged 1 commit into
mainfrom
user/gudge/edlcodegen/fix-struct-out-param

Conversation

@MGudgin

Copy link
Copy Markdown
Member

Summary

Fixes a Rust code-generation bug in edlcodegen: every enclave ecall (or host callback) with an out-only, non-array struct or wstring parameter faulted. The inbound dispatch stub borrowed the out-param from an ABI Option<T> (which is None on entry) via .as_mut().expect(...), which panicked — and in a no_std enclave the panic handler is loop {}, so the call spun at ~99% CPU (presenting as a hang). This also hardens the symmetric result-extraction path against malformed input.

Changes

  • Inbound dispatch (GetClosureFunctionStatement): emit abi_type.m_x.insert(Default::default()) instead of .as_mut().expect(...). Inserts a default so the closure can borrow &mut T, and (because it overwrites unconditionally) discards any value the other side of the ABI supplied for an out-only param, so caller-controlled contents never reach the implementation.
  • Result extraction (GetMoveFromAbiStructToParamStatements): emit result.m_x.ok_or(<crate>::AbiError::Hresult(0x80070057u32 as i32))? instead of .expect(...). When an enclave unpacks a host callback's result this runs on untrusted, host-controlled data; a host omitting the field previously re-triggered the same enclave panic/loop (a DoS). It now fails the call as an AbiError. 0x80070057 is E_INVALIDARG (written as a u32i32 cast, matching the library, since the value doesn't fit in i32).
  • Scope: only non-array, out-only struct/wstring params take these Option<T> ABI paths; string, arrays, vectors, and in/inout params are unaffected and already correct.

Regression coverage

The prior codegen test EDL (CurrentCodeGenerationState/CodeGenerationState.edl) had no out-only non-array struct/wstring param, so these paths were never captured in the checked-in baselines — which is why the bug shipped. This PR adds [out] TestStruct1 and [out] wstring params (trusted and untrusted) and regenerates the baselines, locking the fixed dispatch/extraction code against regression.

Testing

  • Rebuilt ToolingExecutable; the post-build GenerateCodeGenCurrentState target regenerated the C++ and Rust baselines with no errors.
  • Confirmed regenerated Rust: dispatch closures emit insert(Default::default()); extraction stubs emit ok_or(<crate>::AbiError::Hresult(0x80070057u32 as i32))?.
  • Type-checked both patterns with rustc: insert yields &mut T; ok_or(...)? returns Err on a missing (None) field (0x80070057u32 as i32 == -2147024809).
  • Verified end-to-end in a downstream Rust enclave sample: the previously-hanging struct out-param ecall now runs to completion.

Follow-ups (tracked separately)

  • CI: assert baselines are in sync (git diff --exit-code) and add executable Rust codegen coverage that would catch a straight revert.
  • Pre-existing: [out]/[in,out] optional<T> params are effectively dead in Rust (Option<&mut T> can never allocate); needs a broader codegen change.

This commit fixes a code-generation bug that made every Rust enclave ecall (or
host callback) with an out-only, non-array struct or wstring parameter fault,
hardens the symmetric result-extraction path against malformed input, and adds
regression coverage for both.

Details:
- Inbound dispatch: the enclave/host closure borrowed a struct/wstring [out]
  param from an ABI Option<T> that is None on entry (the caller never sends an
  out value) via .as_mut().expect(...). That panicked; in a no_std enclave the
  panic handler is loop {}, so the call spun at ~99% CPU. Generate
  abi_type.m_x.insert(Default::default()) instead: it inserts a default so the
  closure can borrow &mut T, and because it overwrites unconditionally it also
  discards any value the other side of the ABI supplied for an out-only param,
  so caller-controlled contents never reach the implementation.
- Result extraction: the caller stub copied a returned struct/wstring out param
  back with result.m_x.expect(...). When an enclave unpacks a host callback's
  result this runs on untrusted, host-controlled data, so a host omitting the
  field would re-trigger the same panic/loop inside the enclave. Generate
  result.m_x.ok_or(<crate>::AbiError::Hresult(0x80070057u32 as i32))? instead
  (0x80070057 is E_INVALIDARG; the u32-to-i32 cast matches the library, as the
  value does not fit in i32), so a missing field fails the call as an ABI error
  rather than faulting the caller.
- Only non-array, out-only struct and wstring params take these Option<T> ABI
  paths; string, arrays, vectors, and in/inout params are unaffected and already
  correct. Scalar/optional out-params were also unaffected.
- Extend CurrentCodeGenerationState/CodeGenerationState.edl with [out]
  TestStruct1 and [out] wstring params (trusted and untrusted). The prior test
  EDL had no out-only non-array struct/wstring param, so these codegen paths
  were never captured in the checked-in baselines, which is why the bug shipped.
- Regenerate the CurrentCodeGenerationState baselines so the fixed dispatch and
  extraction code is locked in against regression.

Test:
- Rebuilt ToolingExecutable; its post-build GenerateCodeGenCurrentState target
  regenerated the C++ and Rust baselines with no errors.
- Confirmed the regenerated Rust dispatch closures emit insert(Default::default())
  and the extraction stubs emit
  ok_or(<crate>::AbiError::Hresult(0x80070057u32 as i32))? for the new
  struct/wstring out-params.
- Type-checked both generated patterns with rustc: insert yields &mut T for the
  closure, and ok_or(...)? returns Err on a missing (None) field
  (0x80070057u32 as i32 == -2147024809).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 3f90c337-7087-44f5-98d7-c0534d1f49b1
if (!is_array && param.IsOutParameterOnly() && IsStructOrWStringType(param))
{
abi_struct_fields << std::format(
"abi_type.m_{}.as_mut().expect(\"Unexpected empty Option: m_{}\")",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't remember why I actually added .expect last year but I don't see how doing Option.insert would be a bad idea.

@bbonaby
Branden Bonaby (bbonaby) merged commit ce5dac4 into main Jul 25, 2026
15 checks passed
@MGudgin
Gudge (MGudgin) deleted the user/gudge/edlcodegen/fix-struct-out-param branch July 31, 2026 23:33
Branden Bonaby (bbonaby) pushed a commit that referenced this pull request Aug 3, 2026
## Summary

Follow-up to #207. That PR fixed the Rust out-only struct/wstring
out-param codegen (panic-loop on `None`, plus a symmetric
result-extraction DoS). This PR adds the regression protection that was
called out as missing: an executable test of the generator and a CI gate
against baseline drift.

## Changes

### 1. Generator unit test (executable, catches a revert)

`tests/UnitTests/ToolingExecutableTests/CodeGenerationRustOutParamTests.cpp`
parses `TestFiles/OutParamCodeGenTest.edl` and asserts the Rust emit
functions produce the panic-safe forms:
- **Dispatch closure** (`GetClosureFunctionStatement`): out-only
struct/wstring params emit `insert(Default::default())` — never
`.as_mut().expect(...)` or `get_or_insert_with`.
- **Extraction** (`GetMoveFromAbiStructToParamStatements`): out-only
struct/wstring emit `ok_or(<crate>::AbiError::Hresult(0x80070057u32 as
i32))?` — never `.expect(...)`, with the crate-appropriate error path
(`edlcodegen_enclave` vs `edlcodegen_host`).
- An `[in, out]` struct is included as a **contrast case** (plain borrow
/ plain move), so the test also pins that these paths are *not*
over-applied.

Reverting either fix in `CodeGenerationHelpers.h` fails this test. It
runs in the existing `cpp_ci` EdlCodeGen unit-test job (no new infra).

### 2. Baseline drift gate
A `cpp_ci` step runs after the build (which regenerates
`CurrentCodeGenerationState`) and fails if the checked-in baselines
differ from the freshly-built generator's output — catching a forgotten
regeneration or an unexpected codegen change in review. x64 only, since
the generator only runs on x64.

## Testing

- Built `UnitTests` via the solution and ran the new tests with
`vstest`: **3/3 pass**
(`Dispatch_OutStructAndWString_UseInsert_NotExpect`,
`Extract_OutStructAndWString_UseOkOr_NotExpect`,
`Extract_HostDirection_UsesHostCrate`).
- Verified the baseline drift gate passes on the current tree (no
drift).

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 3f90c337-7087-44f5-98d7-c0534d1f49b1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants