PR4: per-tier GPU-evidence requirement policy (attested 3p must present GPU evidence) - #749
Conversation
PR4 of the Chutes attested-provider integration (follows #745, #746, #748). Completes the per-tier verifier policy story (TCB floor #746, report_data #748, GPU evidence now) before the Chutes provider code. verify_gpu_evidence is best-effort today: absent nvidia_payload -> Ok(None) ("acceptable for non-GPU CVMs"). Correct for NEAR's mixed fleet, but an attested third party that advertises confidential GPU compute (e.g. Chutes/Hopper) could omit GPU evidence and silently pass with gpu_verdict=None. Add MeasurementPolicy.require_gpu_evidence (private + accessor): - near()/near_from_env() -> false (historical best-effort, NEAR unchanged). - attested3p() -> true. verify_gpu_evidence's absent-payload branch now consults the policy: Ok(None) when not required (NEAR), Err(GpuVerificationFailed) when required (attested 3p). The mandatory nonce binding + NRAS verification on present evidence are unchanged. Tests: gpu_evidence_required_only_for_attested3p (policy); gpu_evidence_absent_ok_for_near_but_rejected_for_attested3p (deterministic verify test, no network — absent branch returns before NRAS). GLM-5/Qwen fixtures unchanged; services lib 354 + e2e_all 546 green; fmt + clippy clean. No NEAR behavior change.
Review: PR4 — per-tier GPU-evidence requirement policyReviewed the diff against the priority checks. This is a tight, additive change and I found no critical issues. Verified:
Minor (non-blocking) nit:
Tests cover both the policy accessor and the verifier branch for NEAR vs. attested-3p. Good privacy hygiene too — error strings carry no customer data. ✅ Approved |
There was a problem hiding this comment.
Code Review
This pull request introduces a require_gpu_evidence policy option to enforce the presence of GPU evidence (NVIDIA NRAS) for third-party attested providers, while maintaining best-effort verification (returning Ok(None) if absent) for NEAR's own fleet. A security review identified a potential bypass vulnerability where a non-string nvidia_payload (e.g., a JSON object or array) would cause v.as_str() to return None, silently skipping GPU verification for NEAR nodes instead of rejecting the invalid format.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| _ => { | ||
| // No GPU evidence. Acceptable for NEAR's non-GPU CVMs, but an | ||
| // attested provider that must prove confidential GPU compute | ||
| // (policy.require_gpu_evidence) is rejected rather than passing | ||
| // with no GPU verdict. | ||
| if self.policy.require_gpu_evidence() { | ||
| return Err(AttestationVerificationError::GpuVerificationFailed( | ||
| "GPU evidence (nvidia_payload) required by policy but absent from the \ | ||
| attestation report" | ||
| .to_string(), | ||
| )); | ||
| } | ||
| return Ok(None); | ||
| } |
There was a problem hiding this comment.
Security Vulnerability: GPU Verification Bypass via Type Confusion
If nvidia_payload is present in the attestation report but is not a string (for example, if it is a JSON object or array), v.as_str() returns None. This causes the match to fall into the catch-all _ arm.
For NEAR nodes (where require_gpu_evidence is false), the function will then return Ok(None), silently skipping GPU verification even though GPU evidence was technically provided (but in a non-string format).
This creates a potential security bypass where a compromised or rogue NEAR node with a GPU can bypass mandatory GPU verification (such as the nonce check and NRAS verification) by simply formatting nvidia_payload as a JSON object instead of a stringified JSON.
To fix this, we should explicitly check if nvidia_payload is present and not null. If it is present but is not a valid string, we should return an InvalidFormat error rather than silently treating it as absent.
_ => {
// If the payload is present but not a string (e.g. an object),
// reject it as invalid format instead of silently skipping verification.
if let Some(v) = attestation_report.get("nvidia_payload") {
if !v.is_null() && !v.is_string() {
return Err(AttestationVerificationError::InvalidFormat(
"nvidia_payload must be a JSON string".to_string(),
));
}
}
// No GPU evidence. Acceptable for NEAR's non-GPU CVMs, but an
// attested provider that must prove confidential GPU compute
// (policy.require_gpu_evidence) is rejected rather than passing
// with no GPU verdict.
if self.policy.require_gpu_evidence() {
return Err(AttestationVerificationError::GpuVerificationFailed(
"GPU evidence (nvidia_payload) required by policy but absent from the \
attestation report"
.to_string(),
));
}
return Ok(None);
}References
- Carry error verdicts and specific failure conditions as typed enum variants rather than using helper functions or match guards across modules to inspect error details.
There was a problem hiding this comment.
Pull request overview
Implements a per-tier policy for whether NVIDIA GPU evidence (nvidia_payload) must be present during attestation verification. This preserves NEAR’s existing best-effort behavior for mixed CPU/GPU fleets while ensuring attested third-party providers cannot silently omit GPU evidence when their tier requires it.
Changes:
- Add
require_gpu_evidencetoMeasurementPolicywith a public accessor; defaultfalsefor NEAR policies andtrueforattested3p. - Update
AttestationVerifier::verify_gpu_evidenceto error when GPU evidence is required by policy but absent. - Add unit tests covering policy behavior and the absent-evidence branch for NEAR vs attested-3p.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| crates/services/src/attestation/verification.rs | Enforces “GPU evidence required” on the missing-evidence branch and adds a deterministic test for NEAR vs attested-3p behavior. |
| crates/services/src/attestation/measurement.rs | Adds a per-tier require_gpu_evidence policy flag plus accessor and unit test coverage. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| @@ -416,7 +416,20 @@ impl AttestationVerifier { | |||
| .and_then(|v| v.as_str()) | |||
| { | |||
There was a problem hiding this comment.
Reviewed the diff and surrounding attestation policy/verifier code. The new require_gpu_evidence policy is wired through MeasurementPolicy::attested3p, and the missing-nvidia_payload branch fails closed for attested 3P while preserving NEAR best-effort behavior.
Local checks passed: cargo fmt --check; cargo test -p services attestation::measurement::tests --no-fail-fast; cargo test -p services gpu_evidence --no-fail-fast; cargo test -p services attestation::verification::tests --no-fail-fast.
GitHub checks observed: lint, security_audit, and claude-review passed; Test Suite was still pending at review time. I found no substantive issues.
…of skipping
gemini (security-high) + Copilot flagged a type-confusion gap in verify_gpu_evidence:
`get("nvidia_payload").and_then(|v| v.as_str())` returns None when the field is
PRESENT but not a string (object/number/bool), so a malformed payload was treated
as "absent" and silently skipped GPU verification (for NEAR / best-effort tiers).
Distinguish the three cases explicitly:
- absent / null / empty string -> no GPU evidence offered (policy decides:
Ok(None) for NEAR, Err for an attested 3p that requires it)
- non-empty string -> the evidence to verify
- present but not a string -> MALFORMED, always Err for every tier
Test extended: a present-but-non-string nvidia_payload errors for both NEAR and
attested3p (never silently skipped). services lib green; fixtures unchanged;
fmt + clippy clean.
|
Good catch @gemini-code-assist (security-high) / @copilot — fixed in The gap: Now the three cases are explicit, and present-but-malformed is never mistaken for absent: let nvidia_payload_str = match attestation_report.get("nvidia_payload") {
None | Some(Value::Null) => None, // absent
Some(Value::String(s)) if s.is_empty() => None, // absent
Some(Value::String(s)) => Some(s.as_str()), // verify this
Some(_) => return Err(GpuVerificationFailed( // MALFORMED -> always reject
"nvidia_payload is present but is not a string (malformed report); refusing to skip GPU verification")),
};
Test extended ( |
There was a problem hiding this comment.
Code Review
This pull request introduces a require_gpu_evidence policy option to enforce the presence of GPU evidence (NVIDIA NRAS) for attested third parties, while preserving best-effort verification for NEAR's non-GPU CVMs. It also hardens the verification logic to explicitly handle malformed, non-string nvidia_payload values to prevent type-confusion bypasses. Feedback suggests avoiding a catch-all pattern (_) when matching on serde_json::Value to ensure compile-time safety if new variants are added.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| let nvidia_payload_str = match attestation_report.get("nvidia_payload") { | ||
| None | Some(serde_json::Value::Null) => None, | ||
| Some(serde_json::Value::String(s)) if s.is_empty() => None, | ||
| Some(serde_json::Value::String(s)) => Some(s.as_str()), | ||
| Some(_) => { | ||
| return Err(AttestationVerificationError::GpuVerificationFailed( | ||
| "nvidia_payload is present but is not a string (malformed report); \ | ||
| refusing to skip GPU verification" | ||
| .to_string(), | ||
| )); | ||
| } | ||
| }; |
There was a problem hiding this comment.
To avoid logic drift and maintain consistency with the rest of the codebase (such as nras_failed_checks and decode_nras_jwt_claims), avoid using a catch-all pattern (_) when matching on serde_json::Value. Explicitly enumerating the other variants ensures that any future additions to the serde_json::Value enum will trigger a compile-time error, forcing a deliberate decision on how to handle them.
let nvidia_payload_str = match attestation_report.get("nvidia_payload") {
None | Some(serde_json::Value::Null) => None,
Some(serde_json::Value::String(s)) if s.is_empty() => None,
Some(serde_json::Value::String(s)) => Some(s.as_str()),
Some(serde_json::Value::Bool(_))
| Some(serde_json::Value::Number(_))
| Some(serde_json::Value::Array(_))
| Some(serde_json::Value::Object(_)) => {
return Err(AttestationVerificationError::GpuVerificationFailed(
"nvidia_payload is present but is not a string (malformed report); \
refusing to skip GPU verification"
.to_string(),
));
}
};References
- When matching on an enum to classify its variants, avoid a catch-all case (
_). This forces a compile-time error if a new variant is added, ensuring the classification logic is updated accordingly.
PierreLeGuen
left a comment
There was a problem hiding this comment.
Reviewed the attestation policy and verifier changes in crates/services/src/attestation/measurement.rs and crates/services/src/attestation/verification.rs, including the policy constructors, absent/malformed nvidia_payload handling, and surrounding attestation flow.
Checks run locally:
cargo fmt --checkcargo test -p services attestationcargo clippy -p services --all-targets -- -D warnings
Remote checks were also green when refreshed: Lint, Test Suite, and security_audit. I did not find any blocking or substantive issues.
Code ReviewVerdict: Approve Clean, well-scoped change. The explicit 3-way match ( One gap not in prior commentsEmpty-string The test let mut empty_payload = serde_json::Map::new();
empty_payload.insert("nvidia_payload".to_string(), json!(""));
// NEAR: best-effort → Ok(None)
assert!(matches!(near.verify_gpu_evidence(&empty_payload, "00").await, Ok(None)));
// attested 3p: require_gpu_evidence → rejected
assert!(a3p.verify_gpu_evidence(&empty_payload, "00").await.is_err());Not a correctness bug (the code handles it correctly), but the coverage gap means a future refactor could accidentally merge the empty-string and non-empty-string branches without a test catching it. Nits already noted
|
…, hard-off) (#752) * inference_providers: add attested::chutes::Provider data-path skeleton PR5 of the Chutes attested-provider integration — first Chutes provider code (PR1-PR4 were verifier groundwork). Adds attested::chutes::Provider, an OpenAI-compatible inference client over https://llm.chutes.ai/v1. The data path is delegated to an inner ExternalProvider (ProviderConfig::OpenAiCompatible) — Chutes' inference API is OpenAI-compatible, so chat/stream/text/image/audio/score/rerank/embeddings/privacy/models reuse the proven, already-tested path; no duplicated HTTP/SSE code. It's a distinct type under attested::chutes (not just an external provider) because a later PR bolts a ChutesBackendVerifier onto it (fetch /evidence, transform TDX quote + GPU evidence into NEAR's attestation format, run the shared verifier under the strict Attested3p policy from #746/#748/#749). get_attestation_report and get_signature are explicitly stubbed as "not yet wired (hard-off gate)" — NOT delegated to the inner external provider's "unsupported" path — so the skeleton can never be mistaken for verified. Hard-off: nothing constructs a chutes::Provider yet (no pool wiring), so it's unreachable at runtime. Pool wiring + enable flag land with the verifier PR; production enablement stays gated on confirming Chutes' evidence format. Tests: construction keeps the model id + defaults to the Chutes base URL; attestation/signature return the "not yet wired" error. inference_providers lib 238 + e2e_all 546 green; fmt + clippy clean. No runtime behavior change. * address review: redact Config api_key in Debug, private fields, guard timeout, match error variants PR5 review feedback (#752): - Pierre (blocking) + Copilot + claude[bot]: Config derived Debug while holding a plaintext api_key -> secret leak via any {:?}. Make Config fields private (only constructor is Config::new) and implement a custom Debug that redacts api_key. - Copilot: timeout_seconds: i64 is cast `as u64` in the HTTP backend; a negative value would underflow to an effectively infinite timeout. new() now replaces a non-positive timeout with DEFAULT_TIMEOUT_SECONDS (300); private fields prevent a struct-literal bypass. - Copilot: tests asserted on Debug strings (brittle). Match the concrete error variant (AttestationError::FetchError / CompletionError::CompletionError) and assert on the message instead. Added config_non_positive_timeout_falls_back_to_default (also asserts the api_key is redacted, not leaked). inference_providers lib green; fmt + clippy clean. * address review: add Config::with_base_url builder + timeout_seconds() accessor; robust test assertions Pierre's follow-up observations on #752: - Config::new hardcoded base_url though the field is configurable + exposed via base_url(). Add a `with_base_url(..)` builder so the verifier-wiring PR + its integration/mock-server tests can override the URL (and the accessor is now meaningful). - The timeout test substring-matched Debug output and used a 1-char key. Add a timeout_seconds() accessor and assert on it (not Debug); split out config_debug_redacts_api_key using a realistic-length secret so the leak check is meaningful. Add with_base_url_overrides_default. inference_providers lib green; fmt + clippy clean. Still additive + unwired.
Summary
PR4 of the Chutes attested-provider integration (follows #745
ProviderTier, #746MeasurementPolicy, #748 strictReportDataVerifier). Makes GPU-evidence-required a per-tier policy. NEAR keeps its historical best-effort behavior; an attested third party that advertises confidential GPU compute must actually present verifiable, nonce-bound GPU evidence. NEAR behavior is unchanged.This completes the per-tier verifier policy story — TCB floor (#746), report_data binding (#748), and now GPU evidence — before the Chutes provider code (PR5–PR7).
Background
verify_gpu_evidenceis best-effort today: a report with nonvidia_payloadreturnsOk(None)("acceptable for non-GPU CVMs"). When GPU evidence is present, the caller-nonce binding (payload.nonce == request_nonce) and NVIDIA NRAS verification are already mandatory.For NEAR's own fleet (mix of GPU and non-GPU CVMs) the absent-is-OK behavior is correct. But a third party like Chutes advertises NVIDIA confidential compute (Hopper) — if it simply omitted the GPU evidence, the current code would pass it with
gpu_verdict = None, silently dropping the GPU half of the attestation. An attested 3p must prove what it claims.What changed
MeasurementPolicygains a privaterequire_gpu_evidence: bool+require_gpu_evidence()accessor:near()/near_from_env()→false(historical best-effort).attested3p()→true.verify_gpu_evidence: the absent-nvidia_payloadbranch now consults the policy —Ok(None)when not required (NEAR),Err(GpuVerificationFailed)when required (attested 3p). The mandatory nonce binding + NRAS verification on present evidence are unchanged.Tests
measurement::tests::gpu_evidence_required_only_for_attested3p—near/near_from_env→ false,attested3p→ true.verification::tests::gpu_evidence_absent_ok_for_near_but_rejected_for_attested3p— deterministic (no network; the absent branch returns before any NRAS call): NEAR verifier →Ok(None), attested-3p verifier →Err.serviceslib green;cargo build --workspacegreen;cargo test --test e2e_allgreen; fmt + clippy clean.Impact
require_gpu_evidence = false, identical to today). NoAttested3pprovider is wired yet.attested3p()policy now requires the GPU evidence Chutes advertises.🤖 Generated with Claude Code