VitisAI EP cache is reused across EP/driver version changes, contrary to AMD guidance

[KayMKM]

Problem

The VitisAI EP keeps a compilation cache on disk (default
C:\temp\%USERNAME%\vaip\.cache). AMD documents that this cache must not be
reused after the EP or the NPU driver is updated:

Cache directories generated by the Vitis AI Execution Provider should not be reused across different versions of the Vitis AI EP or across different version of the NPU drivers.

If using the VitisAI EP Cache the application should check the version of the Vitis AI EP and of the NPU drivers. If the application detects a version change, it should delete the cache, or create a new cache directory with a different name.

Source: https://ryzenai.docs.amd.com/en/latest/modelrun.html#vitisai-ep-cache

winml-cli performs no such check. The cache survives EP updates (including the
ones winml-cli itself installs through the WinML EP catalog at the start of a
run), so a user can silently keep using artifacts compiled by a previous EP or
driver version.

Why this matters

A stale cache does not merely risk undefined behaviour — it can hide real
failures. Observed on an AMD Ryzen AI 9 HX 370 with VitisAI EP 1.8.72.0, same
machine and same command, the only difference being cache state:

Cache state	winml perf -m facebook/convnext-tiny-224 --device npu --ep vitisai --precision w8a16
Stale entries from an earlier EP version	Completed, reported 21.98 ms avg
Emptied before the run	Crashed at session creation, exit 0xC0000409 (reproduced 2/2)

The "passing" run never exercised the compile path — it loaded an artifact built
by a different EP version. A user in this state gets results that do not reflect
the EP they actually have installed.

Current state

PR #1223 clears this cache, but only in scripts/e2e_eval/run_eval.py, which is
an internal evaluation harness. Users of the winml CLI are still exposed.

Proposed behaviour

Detect a VitisAI EP / NPU driver version change and act on it, per the AMD
guidance. Options, roughly in order of intrusiveness:

1. Warn. On session creation with the VitisAI EP, compare the current EP
   version against the version recorded when the cache was last written. On a
   mismatch, emit a warning naming the cache directory and how to clear it.
2. Version-scoped cache directory. Pass a cache_dir / cache_key provider
   option that includes the EP (and driver) version, so a version change
   naturally lands in a new directory and stale artifacts are never reused.
3. Clear automatically. Delete the stale cache on a detected version change,
   ideally behind an opt-out flag since recompilation is expensive (~3 minutes
   for a ConvNeXt-tiny on this machine).

Option 2 matches the "create a new cache directory with a different name"
wording in the AMD documentation and avoids deleting anything the user may still
want.

Implementation notes

The EP version is already available to winml-cli:

• src/winml/modelkit/session/ep_registry.py exposes a per-source version
  property and reads ep_metadata["version"] for the plugin version. The CLI
  already prints it when installing an EP (- Version: 1.8.72.0).
• The cache location is controlled by the cache_dir and cache_key provider
  options, which winml-cli sets in
  WinMLCompileConfig.for_vitisai (src/winml/modelkit/compiler/configs.py).
  Neither is currently set, so the EP default is used.

The NPU driver version is not currently read by winml-cli; if the initial
implementation only keys on the EP version, that should be called out so the
remaining gap is known.

[DingmaomaoBJTU]

Triage Summary

Priority: P2
Component: compiler / session (VitisAI EP cache management)
Classification: Bug — correctness (stale cache masks real failures)
Assigned to: @KayMKM

Triage Notes

• VitisAI EP compilation cache persists across EP/driver version changes, violating AMD guidance.
• Stale cache can silently produce results from a different EP version or hide crashes (observed 0xC0000409 on fresh cache vs passing on stale).
• Workaround exists: manually clear C:\temp\%USERNAME%\vaip\.cache.
• Recommended fix: Option 2 from the issue — version-scoped cache directory keyed on EP version in WinMLCompileConfig.for_vitisai. This matches AMD docs ("create a new cache directory with a different name") and avoids destructive deletion.
• NPU driver version gap should be documented as a known limitation if not addressed in v1.

Action: Implement version-scoped cache dir in compiler/configs.py; add integration test validating cache isolation across EP versions.