Introduce public MergedDataMap

[pytorchbot]

Summary:
Add public merged data map. Module can use this to resolve multiple named data maps.
Creating as a sep dependency rather than inside module/ so it can be used independently of module. (think there may be some other internal usages soon)

Add support for BUCK and CMake. On CMake side, have to add the dep for everyone who uses module ..

Reviewed By: swolchok

Differential Revision: D83527299

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/14899

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• [ROCm][CI] Machines under the label linux.rocm.gpu.2 are undergoing maintenance.

❌ 4 Cancelled Jobs

As of commit 7b497d5 with merge base e0dda90 ():

CANCELLED JOBS - The following jobs were cancelled. Please retry:

• pull / unittest / macos / macos-job (gh)
  ##[error]The operation was canceled.
• pull / unittest-editable / macos / macos-job (gh)
  ##[error]The operation was canceled.
• trunk / test-models-macos-cpu (ic4, xnnpack-quantization-delegation) / macos-job (gh)
• trunk / unittest-release / macos / macos-job (gh)
  ##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[GregoryComer]

Rebasing to see if it fixes build failures.

[GregoryComer]

@lucylq Do you have any recommendations on how to handle the build error? Is there a prerequisite pick that needs to be merged? Thanks.

+ cmake -DCMAKE_INSTALL_PREFIX=cmake-out -DCMAKE_BUILD_TYPE=Release -DEXECUTORCH_ENABLE_LOGGING=ON -DEXECUTORCH_BUILD_EXTENSION_MODULE=ON -DEXECUTORCH_BUILD_EXTENSION_DATA_LOADER=ON -DEXECUTORCH_BUILD_EXTENSION_FLAT_TENSOR=ON -DEXECUTORCH_BUILD_EXTENSION_LLM=ON -DEXECUTORCH_BUILD_EXTENSION_LLM_RUNNER=ON -DEXECUTORCH_BUILD_EXTENSION_TENSOR=ON -DEXECUTORCH_BUILD_KERNELS_LLM=ON -DEXECUTORCH_BUILD_KERNELS_OPTIMIZED=ON -DEXECUTORCH_BUILD_KERNELS_QUANTIZED=ON -DEXECUTORCH_BUILD_XNNPACK=ON -DEXECUTORCH_XNNPACK_SHARED_WORKSPACE=ON -Bcmake-out .
CMake Error at tools/cmake/common/preset.cmake:126 (message):
  Use of 'EXECUTORCH_BUILD_EXTENSION_MODULE' requires
  'EXECUTORCH_BUILD_EXTENSION_NAMED_DATA_MAP'
Call Stack (most recent call first):
  tools/cmake/preset/default.cmake:281 (check_required_options_on)
  CMakeLists.txt:87 (include)

[lucylq]

@GregoryComer taking a look, I think #14907 needs to be CP'd in as well. Though that seems to be different to test-models-linux-aarch64 ?

[GregoryComer]

@GregoryComer taking a look, I think #14907 needs to be CP'd in as well. Though that seems to be different to test-models-linux-aarch64 ?

Thanks. I've merged that pick and will rebase on top. The other failure looks like an infra issue (download failure), so hopefully it will go away on re-run. I'll keep an eye on it.

[lucylq]

@GregoryComer also this one for the llava failure, sorry about that. #14973

[lucylq]

Are the macos runner stuck? Seem to be queued for >3hrs

Cancelled and re-running.