Add .ptd support to portable executor runner

[larryliu0820]

This pull request enhances the executor_runner example by adding support for loading and using .ptd (portable tensor data) files. This enables the runner to ingest pre-serialized tensor data, improving flexibility for model input handling. The changes include updates to both build configuration and the main runner logic.

Support for .ptd file loading and usage:

• Added a new command-line flag data_path to specify the path to a .ptd data file in executor_runner.cpp and integrated logic to load this file and parse its contents using FlatTensorDataMap. [1] [2]
• Updated the runner to pass the loaded tensor data map to the model method loader, allowing methods to access pre-loaded input data.

Build and dependency updates:

• Included flat_tensor_data_map as a dependency in both the Bazel build targets and CMake build configuration to ensure the new functionality is available during compilation. [1] [2] [3]
• Added the necessary header include for flat_tensor_data_map in executor_runner.cpp and updated the relevant namespace usage. [1] [2]

Test Plan:

Tested with .pte and .ptd for CUDA backend:

python -m executorch.examples.cuda.scripts.export --model_name linear --output_dir ./

Make sure we have linear.pte and aoti_cuda_blob.ptd.

Build executor runner with the following options:

cmake -DCMAKE_BUILD_TYPE=Debug -DEXECUTORCH_BUILD_CUDA=ON -DEXECUTORCH_BUILD_EXTENSION_DATA_LOADER=ON -DEXECUTORCH_BUILD_EXTENSION_FLAT_TENSOR=ON -DEXECUTORCH_BUILD_EXTENSION_TENSOR=ON -S. -Bcmake-out

Then do:

cmake --build cmake-out -j8

Then we can run:

cmake-out/executor_runner --model_path linear.pte --ptd_path aoti_cuda_blob.ptd
I 00:00:00.000594 executorch:executor_runner.cpp:189] PTD file aoti_cuda_blob.ptd is loaded.
I 00:00:00.000671 executorch:executor_runner.cpp:199] PTD data map created with 1 keys.
I 00:00:00.000749 executorch:executor_runner.cpp:249] Model file linear.pte is loaded.
I 00:00:00.000758 executorch:executor_runner.cpp:258] Using method forward
I 00:00:00.000770 executorch:executor_runner.cpp:309] Setting up planned buffer 0, size 96.
I 00:00:00.002908 executorch:cuda_backend.cpp:140] Writing 394624 bytes to /tmp/linear_so_blob844427.so
I 00:00:00.324783 executorch:cuda_backend.cpp:174] container_handle = 0x26a71b0
I 00:00:00.324867 executorch:executor_runner.cpp:337] Method loaded.
I 00:00:00.325796 executorch:cuda_backend.cpp:249] Inputs copied to GPU
I 00:00:00.325829 executorch:cuda_backend.cpp:278] Outputs created on GPU
E 00:00:00.326623 executorch:memory.cpp:286] Cannot delete null tensor
I 00:00:00.326678 executorch:executor_runner.cpp:374] Model executed successfully 1 time(s) in 1.777041 ms.
I 00:00:00.326691 executorch:executor_runner.cpp:383] 1 outputs: 
OutputX 0: tensor(sizes=[3, 3], [-0.199237, 0.550725, 0.0830356, -0.199237, 0.550725, 0.0830356, -0.199237, 0.550725, 0.0830356])
E 00:00:00.328474 executorch:memory.cpp:299] Didn't find tensor 0x699a3d0

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/14833

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 5 New Failures

As of commit d017656 with merge base 0b748bf ():

NEW FAILURES - The following jobs have failed:

• Build Presets / windows (pybind) / build (gh)
• Build Windows Wheels / pytorch/executorch / build-wheel-py3_10-cpu (gh)
• Build Windows Wheels / pytorch/executorch / upload / upload-wheel-py3_10-cpu (gh)
  Unable to download artifact(s): Artifact not found for name: pytorch_executorch__3.10_cpu_x64
• pull / unittest / macos / macos-job (gh)
  export/tests/test_target_recipes.py::TestTargetRecipes::test_linear_model
• trunk / test-qnn-optimum-model (fp32, mobilevit_v2) / linux-job (gh)
  RuntimeError: Command docker exec -t 32d39e087c215dd17d446c8272c181fd9e76fc6f7b5199fda4d12dee71cf329a /exec failed with exit code 1

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[lucylq]

Thanks for adding this!

[meta-codesync]

@larryliu0820 has imported this pull request. If you are a Meta employee, you can view this in D84020938.