[REVIEW] model-supply-chain: add ONNX external-data and runtime-provider evidence gates

[MAUROCERON]

Skill Being Reviewed

Skill name: model-supply-chain
Skill path: skills/ai-security/model-supply-chain/

False Positive Analysis

Benign-looking ONNX model package that can be over-credited:

model:
  format: onnx
  source: internal-artifact-registry
  sha256: recorded
  runtime: onnxruntime
  providers:
    - CUDAExecutionProvider
    - CPUExecutionProvider
external_data:
  present: true
  files:
    - weights.bin
conversion:
  source_model: pytorch/checkpoint.pt
  opset: 18

Why this is a false positive:

The skill checks model format, checksum verification, unsafe deserialization, and registry provenance, but an ONNX model can still carry supply-chain risk through external tensor files, conversion/opset drift, and runtime provider fallback. A report can credit the ONNX artifact as safe because the .onnx file hash is recorded while the referenced external data files are not independently bound, normalized, and path-contained.

Coverage Gaps

Missed variant 1: external data is not bound to the model digest

Large ONNX models can store tensor data outside the .onnx protobuf. If the reviewer only verifies the main .onnx file, an attacker can replace weights.bin, use unexpected external-data locations, or exploit path/symlink/hardlink behavior in packaging and loading workflows.

Missed variant 2: conversion and opset provenance is not captured

A PyTorch/TensorFlow-to-ONNX conversion may change operators, shapes, dynamic axes, custom ops, or preprocessing assumptions. Without converter version, opset, source model revision, validation dataset, and parity-test evidence, a converted artifact may not be equivalent to the approved source model.

Missed variant 3: execution provider fallback changes the effective runtime

ONNX Runtime execution providers partition work across CPU/GPU/NPU/provider libraries. If the intended provider fails or only handles part of the graph, fallback to CPU or another provider can change performance, isolation, supported operators, logging, and vulnerability exposure. The review should record provider order, allowed providers, fallback mode, custom/provider plugin provenance, and runtime test evidence.

Edge Cases

• Internal-only ONNX exports still need external-data digest binding and path containment because model stores and CI artifacts are mutable.
• Quantized or optimized ONNX artifacts need a link to the exact source model and optimization/quantization step, not only the final file name.
• Custom operators and runtime extensions should be treated like executable plugins and reviewed with the same provenance expectations as inference dependencies.
• A benign CPU fallback may be acceptable for development, but production should document whether fallback is allowed, monitored, and covered by performance/resource limits.

Remediation Quality

• Fix resolves the vulnerability
• Fix doesn't introduce new security issues
• Fix doesn't break functionality
• Issues found: Add ONNX artifact packaging gates for external data, conversion/opset provenance, execution-provider allowlists/fallback behavior, custom operator provenance, and Not Evaluable outcomes when only the final .onnx file is available.

Comparison to Other Tools

Tool	Catches this?	Notes
ONNX checker	Partial	Validates schema/model consistency but does not prove supply-chain provenance or approved runtime policy.
Artifact checksum tooling	Partial	Helps only if it hashes the complete model bundle, including external tensor files and manifests.
ONNX Runtime	Partial	Validates ONNX conformance and executes through configured providers; review must still inspect untrusted model handling and provider policy.

Overall Assessment

Strengths: Strong existing coverage for model provenance, checksums, unsafe pickle/torch loading, model cards, training pipeline integrity, registry access control, and inference dependency CVEs.

Needs improvement: ONNX-specific packaging and runtime behavior should be explicit so a safe-looking .onnx export is not accepted without external-data, conversion, and execution-provider evidence.

Priority recommendations:

1. Add an ONNX external-data and bundle integrity evidence gate.
2. Require conversion/opset/source-model provenance and parity validation for converted ONNX artifacts.
3. Add runtime provider and fallback evidence fields, including provider order, custom ops, fallback mode, and Not Evaluable guidance.

Sources Checked

• ONNX External Data Security: https://onnx.ai/onnx/repo-docs/Security.html
• ONNX External Data: https://onnx.ai/onnx/repo-docs/ExternalData.html
• ONNX Runtime documentation: https://onnxruntime.ai/docs/
• ONNX Runtime Execution Providers: https://onnxruntime.ai/docs/execution-providers/

Bounty Info

• I have read and agree to the CONTRIBUTING.md bounty terms
• Preferred payment method: GitHub Sponsors, or payment details can be provided privately after maintainer acceptance

[MAUROCERON]

Implemented in PR #1387: #1387