2/x: Introduce new ETCoreMLModelCache that adheres to cache protocol (#18682)

[metascroy]

Summary:

This is diff 2/9 to implement the new caching system for CoreML, following the design doc here: fbcode/executorch/backends/apple/coreml/docs/new_cache_design.md

[ gdoc version: https://docs.google.com/document/d/1aKwTNj-L3-sAyBetL92RwJdSsSwA4KhvAQ3FO1zbguE/edit?tab=t.0#heading=h.3ecu6rvzx1z3 ]

The design doc contains detailed plans for each diff in the stack, see https://fburl.com/gdoc/h4vincwz for what this diff does.

Reviewed By: digantdesai

Differential Revision: D91183831

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/18682

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures, 2 Cancelled Jobs, 27 Unrelated Failures

As of commit 63b8599 with merge base 28f3cf3 ():

NEW FAILURES - The following jobs have failed:

• pull / unittest-arm-backend-with-no-deps (test_pytest_models_tosa) / linux-job (gh)
  RuntimeError: Command docker exec -t 2eeba45c51bcf83ae95b9507047dd97fe245a4bbd9559086053117229ce294ae /exec failed with exit code 1
• trunk / test-arm-backend-ethos-u (test_memory_allocation) / linux-job (gh)
  RuntimeError: Command docker exec -t 1fc71af26c90d528f5ae8abc92e9c8ce893c28f699051336854e55c667e2649c /exec failed with exit code 92
• trunk / test-arm-ootb-linux (run_deit_e2e_ethos_u) / linux-job (gh)
  RuntimeError: Command docker exec -t 845b0d8514456137b64999007f639c4aa2c91c4cc52ec2d4869ca45f483b8015 /exec failed with exit code 92

CANCELLED JOBS - The following jobs were cancelled. Please retry:

• pull / unittest / windows / windows-job (gh)
• pull / unittest-buck / macos / macos-job (gh)
  ##[error]The operation was canceled.

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

• pull / test-llama-runner-linux (fp32, xnnpack+custom+qe, linux.arm64.2xlarge, executorch-ubuntu-22.04-gc... / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• pull / test-llama-runner-linux (fp32, xnnpack+custom+quantize_kv, linux.arm64.2xlarge, executorch-ubuntu... / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• pull / test-llama-runner-linux (fp32, xnnpack+quantize_kv, linux.arm64.2xlarge, executorch-ubuntu-22.04-... / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• pull / test-models-linux (add_mul, portable, linux.2xlarge) / linux-job (gh) (detected as infra flaky with no log or failing log classifier)
• pull / test-models-linux (mobilebert, portable, linux.2xlarge) / linux-job (gh) (detected as infra flaky with no log or failing log classifier)
• pull / test-models-linux (resnet50, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh) (detected as infra flaky with no log or failing log classifier)
• pull / test-models-linux-basic (mv3, xnnpack-quantization-delegation, cmake, linux.arm64.2xlarge, execut... / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• pull / test-models-linux-basic (vit, portable, cmake, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11... / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• pull / test-qnn-wheel-packages-linux (3.13) / linux-job (gh) (detected as infra flaky with no log or failing log classifier)
• pull / test-samsung-quantmodels-linux / linux-job (gh) (detected as infra flaky with no log or failing log classifier)
• pull / unittest-buck / linux / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• Test ARM Backend / test-arm / test-backend-linux (arm_tosa_fp, operators) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• trunk / test-arm-backend-ethos-u (test_pytest_ops_ethos_u55) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• trunk / test-huggingface-transformers-xnnpack (smollm3-3b|xnnpack|--quantize) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• trunk / test-models-linux-aarch64 (emformer_join, portable, linux.arm64.2xlarge) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• trunk / test-models-linux-aarch64 (emformer_transcribe, xnnpack-quantization-delegation, linux.arm64.2xla... / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• trunk / test-models-linux-aarch64 (ic3, xnnpack-quantization-delegation, linux.arm64.2xlarge) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• trunk / test-models-linux-aarch64 (mobilebert, portable, linux.arm64.2xlarge) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• trunk / test-models-linux-aarch64 (mobilebert, xnnpack-quantization-delegation, linux.arm64.2xlarge) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• trunk / test-models-linux-aarch64 (mul, portable, linux.arm64.2xlarge) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• trunk / test-models-linux-aarch64 (mv3, xnnpack-quantization-delegation, linux.arm64.2xlarge) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• trunk / test-models-linux-aarch64 (resnet50, portable, linux.arm64.2xlarge) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• trunk / test-models-linux-aarch64 (resnet50, xnnpack-quantization-delegation, linux.arm64.2xlarge) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128
• trunk / test-models-linux-aarch64 (w2l, xnnpack-quantization-delegation, linux.arm64.2xlarge) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / unittest-editable / windows / windows-job (gh) (trunk failure)
  ##[error]The operation was canceled.
• trunk / test-models-linux-aarch64 (qwen2_5_1_5b, portable, linux.arm64.2xlarge) / linux-job (gh) (trunk failure)
• trunk / unittest-release / windows / windows-job (gh) (trunk failure)
  ##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[meta-codesync]

@metascroy has exported this pull request. If you are a Meta employee, you can view the originating Diff in D91183831.

[digantdesai]

Review automatically exported from Phabricator review in Meta.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.