Always lock workspace in buck builds (#19734)

[GregoryComer]

Summary:

In older versions of the XNNPACK backend, we always locked when accessing the XNNPACK workspace. This gives guarantees that we don't re-size the workspace, even when callers violate thread safety.

In D95475030, an OSS contributor disabled locking when workspace sharing is off. This is safe as long as the caller observes proper thread safety. However, we're seeing some crashes that look like they might be caused by the workspace resizing or freeing during inference.

This diff restores the locking for buck builds in the workspace sharing disabled case as a hardening measure. I've left it enabled in CMake.

Differential Revision: D106026285

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/19734

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 4 New Failures, 2 Unrelated Failures

As of commit 57d55be with merge base 3b5f777 ():

NEW FAILURES - The following jobs have failed:

• pull / unittest / linux / linux-job (gh)
  RuntimeError: Command docker exec -t a475db6ea0fbf96d7ce06ca16b2f8e4d8f1f0984143cf41bc79b2475c9732b57 /exec failed with exit code 8
• pull / unittest / windows / windows-job (gh)
  Process completed with exit code 8.
• pull / unittest-editable / linux / linux-job (gh)
  RuntimeError: Command docker exec -t 13bd65752d97159a1ecbc2441325d8bfff6756a617c7897e5652fe648becceb1 /exec failed with exit code 8
• pull / unittest-editable / windows / windows-job (gh)
  Process completed with exit code 8.

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / unittest / macos / macos-job (gh) (trunk failure)
  ##[error]The operation was canceled.
• pull / unittest-editable / macos / macos-job (gh) (trunk failure)
  ##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[meta-codesync]

@GregoryComer has exported this pull request. If you are a Meta employee, you can view the originating Diff in D106026285.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[digantdesai]

This diff restores the locking for buck builds in the workspace sharing disabled case as a hardening measure. I've left it enabled in CMake.

Curious why bother with the differentiation between buck and cmake?

[GregoryComer]

This diff restores the locking for buck builds in the workspace sharing disabled case as a hardening measure. I've left it enabled in CMake.

Curious why bother with the differentiation between buck and cmake?

An OSS contributor requested we get rid of the locks when not using workspace sharing. I don't think the overhead is high, but I did merge their PR to skip locking in this case. I don't love the fragmentation between buck/cmake, so I'm open to changing it.