Use NoThreadPoolGuard in executor_runner if -cpu_threads is set to 1

[swolchok]

Doing so removes some overhead and is what I think we should recommend to customers.

[swolchok]

Stack from ghstack (oldest at bottom):

• -> Use NoThreadPoolGuard in executor_runner if -cpu_threads is set to 1 #15264

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/15264

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• [ROCm][CI] Machines under the label linux.rocm.gpu.2 are undergoing maintenance.

❌ 2 New Failures, 3 Unrelated Failures

As of commit 8b2afb1 with merge base 2e8e579 ():

NEW FAILURES - The following jobs have failed:

• pull / unittest-arm-backend-with-no-fvp (test_pytest_models) / linux-job (gh)
  RuntimeError: Command docker exec -t b8bdc490bfddc52de34ce93f43a123a54d206ed1e5910e88438f1feb72547225 /exec failed with exit code 1
• pull / unittest-nxp-neutron / linux-job (gh)
  RuntimeError: Command docker exec -t 3e437d9ddd0dcb61ae02ad6514989f65fda93012c726d2856fdbe194f11c827f /exec failed with exit code 1

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / android / run-emulator (gh) (trunk failure)
  The process '/usr/bin/sh' failed with exit code 255
• pull / unittest / linux / linux-job (gh) (trunk failure)
• pull / unittest-editable / linux / linux-job (gh) (trunk failure)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[digantdesai]

After this, does the perf look same as ET_USE_THREADPOOL not defined? Curious if there is anything else...

[swolchok]

It won't look the same, because we still wrap the function in a runtime::FunctionRef so there's an extra layer of function call.

[swolchok]

unittest-arm-backend-with-no-fvp and unittest-nxp-neutron both flaky/broken since at least Friday, merging