Enable Robust Symlink Support for External Data (HF Cache Support)

[tianleiwu]

Summary

This PR enables ONNX Runtime to correctly load models with external data when they are stored in symlinked directory structures. This is a common scenario for models cached by the Hugging Face Hub, where both the model file and data files are symlinks pointing into a flat blobs/ directory.

The Problem

Previously, ONNX Runtime's external data path validation would only check if the resolved data path was under the logical directory of the model. In symlinked structures (like Hugging Face's snapshots/ and blobs/ layout), the resolved data path often sits in a different physical directory than the logical model path, leading to a "path escapes model directory" error even when the data is safely associated with the model.

The Fix

I have updated ValidateExternalDataPath to implement a dual-check mechanism:

1. Logical Check: Verify if the resolved data path is under the provided base_dir (the directory where the model was loaded from).
2. Physical Check: If the logical check fails, verify if the resolved data path is under the parent directory of the real/canonical model path.

This approach ensures that models can load external data from both their logical siblings and their physical siblings while maintaining security constraints.

Changes

• Core Logic:
  • onnxruntime/core/framework/tensorprotoutils.cc/h: Updated ValidateExternalDataPath to accept model_path and perform the dual-check.
  • onnxruntime/core/graph/graph.cc: Updated call site in ConvertInitializersIntoOrtValues to pass the model_path.
• Cleanup:
  • Removed unused ValidateExternalDataPath from provider_api.h, provider_interfaces.h, and provider_bridge_ort.cc. These were redundant as path validation is handled by core during session initialization. Provider has no need to validate it.
• Testing:
  • Added onnxruntime/test/python/onnxruntime_test_python_symlink_data.py to simulate the Hugging Face Hub symlink structure and verify the fix.

[Copilot]

Pull request overview

This PR updates ONNX Runtime’s external data path validation to properly handle models stored in symlinked directory layouts (notably Hugging Face Hub cache layouts), while keeping path-escape protections in place.

Changes:

• Extend utils::ValidateExternalDataPath to optionally validate against the canonical (real) model directory when the logical base_dir check fails.
• Update Graph::ConvertInitializersIntoOrtValues to pass model_path into ValidateExternalDataPath.
• Remove provider-bridge exposure of ValidateExternalDataPath and add a Python symlink-based regression test.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
onnxruntime/core/framework/tensorprotoutils.h	Extends ValidateExternalDataPath API to accept optional model_path for canonical validation.
onnxruntime/core/framework/tensorprotoutils.cc	Implements dual logical+canonical directory validation for external data paths.
onnxruntime/core/graph/graph.cc	Passes model_path into external data validation for initializer loading.
onnxruntime/core/session/provider_bridge_ort.cc	Removes provider-host bridge method for external data path validation.
onnxruntime/core/providers/shared_library/provider_interfaces.h	Removes Utils__ValidateExternalDataPath from the ProviderHost interface.
onnxruntime/core/providers/shared_library/provider_api.h	Removes inline ValidateExternalDataPath wrapper from provider API.
onnxruntime/test/python/onnxruntime_test_python_symlink_data.py	Adds Python unittest coverage for HF-style symlink layouts and a negative escape case.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[github-actions]

You can commit the suggested changes from lintrunner.