Decouple most Caffe2 components from the build systems

[cyyever]

This PR decouples most Caffe2 components from CMake and Bazel build systems for later removal, following the recent attempts to totally remove Caffe2 which were inspired by #122527.

cc @gujinghui @PenghuiCheng @XiaobingSuper @jianyuh @jgong5 @mingfeima @sanchitintel @ashokei @jingxu10 @min-jean-cho @yanbing-j @Guobing-Chen @Xia-Weiwen @snadampal

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/125392

• 📄 Preview Python docs built from this PR
• 📄 Preview C++ docs built from this PR
• ❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

❌ 11 New Failures, 2 Unrelated Failures

As of commit 9098062 with merge base b37bef9 ():

NEW FAILURES - The following jobs have failed:

• macos-arm64-binary-conda / conda-py3_12-cpu-build (gh)
  Process completed with exit code 1.
• windows-binary-conda / conda-py3_10-cuda12_1-build (gh)
  No files were found with the provided path: C:\actions-runner\_work\_temp/artifacts. No artifacts will be uploaded.
• windows-binary-conda / conda-py3_11-cuda12_4-test (gh)
  Process completed with exit code 1.
• windows-binary-conda / conda-py3_12-cpu-build (gh)
  No files were found with the provided path: C:\actions-runner\_work\_temp/artifacts. No artifacts will be uploaded.
• windows-binary-conda / conda-py3_12-cuda11_8-test (gh)
  Process completed with exit code 1.
• windows-binary-conda / conda-py3_12-cuda12_1-test (gh)
  Process completed with exit code 1.
• windows-binary-conda / conda-py3_12-cuda12_4-test (gh)
  Process completed with exit code 1.
• windows-binary-conda / conda-py3_8-cuda11_8-test (gh)
  Process completed with exit code 1.
• windows-binary-conda / conda-py3_8-cuda12_4-test (gh)
  Process completed with exit code 1.
• windows-binary-conda / conda-py3_9-cuda11_8-test (gh)
  Process completed with exit code 1.
• windows-binary-conda / conda-py3_9-cuda12_1-test (gh)
  Process completed with exit code 1.

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

• windows-binary-conda / conda-py3_10-cuda11_8-test (gh) (similar failure)
  Process completed with exit code 1.
• windows-binary-conda / conda-py3_11-cuda11_8-test (gh) (similar failure)
  Process completed with exit code 1.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[kit1980]

@cyyever please make all changes outside of caffe2 in a separate PR. Otherwise it's a huge pain to land internally.

[cyyever]

@pytorchbot label ciflow/binaries

[r-barnes]

@cyyever - To clarify @kit1980's point: there's a copy of caffe2 we wish to maintain internally. When you delete files here our sync process tries to delete them internally. We can choose not to sync, but if you've also modified files in the same PR then we have to do manual labor to split apart the PR.

I'm trying to get the failed tests to load so I can inspect them.

[r-barnes]

@cyyever - Thanks for this. It looks as though one of the failing tests is due to a missing nvrtc, which doesn't seem related. Please take a peek to make sure everything is in order or rerun tests. Otherwise, merge when ready.

[cyyever]

@r-barnes I think everything works fine except the two unrelated tests. Would you merge it or me?

[cyyever]

@pytorchmergebot merge -i

[cyyever]

@pytorchbot rebase

[pytorchmergebot]

@pytorchbot started a rebase job onto refs/remotes/origin/viable/strict. Check the current status here

[pytorchmergebot]

Successfully rebased caffe2_observer onto refs/remotes/origin/viable/strict, please pull locally before adding more changes (for example, via git checkout caffe2_observer && git pull --rebase)

[r-barnes]

@pytorchbot rebase

[pytorchmergebot]

@pytorchbot started a rebase job onto refs/remotes/origin/viable/strict. Check the current status here

[pytorchmergebot]

Successfully rebased caffe2_observer onto refs/remotes/origin/viable/strict, please pull locally before adding more changes (for example, via git checkout caffe2_observer && git pull --rebase)

[r-barnes]

@pytorchbot rebase viable/strict

[pytorch-bot]

❌ 🤖 pytorchbot command failed:

@pytorchbot: error: unrecognized arguments: viable/strict

usage: @pytorchbot [-h] {merge,revert,rebase,label,drci,cherry-pick,close} ...

Try @pytorchbot --help for more info.

[r-barnes]

@pytorchbot --help

[pytorch-bot]

PyTorchBot Help

usage: @pytorchbot [-h] {merge,revert,rebase,label,drci,cherry-pick,close} ...

In order to invoke the bot on your PR, include a line that starts with
@pytorchbot anywhere in a comment. That line will form the command; no
multi-line commands are allowed. Some commands may be used on issues as specified below.

Example:
    Some extra context, blah blah, wow this PR looks awesome

    @pytorchbot merge

optional arguments:
  -h, --help            Show this help message and exit.

command:
  {merge,revert,rebase,label,drci,cherry-pick,close}
    merge               Merge a PR
    revert              Revert a PR
    rebase              Rebase a PR
    label               Add label to a PR
    drci                Update Dr. CI
    cherry-pick         Cherry pick a PR onto a release branch
    close               Close a PR

Merge

usage: @pytorchbot merge [-f MESSAGE | -i] [-ic] [-r [{viable/strict,main}]]

Merge an accepted PR, subject to the rules in .github/merge_rules.json.
By default, this will wait for all required checks (lint, pull) to succeed before merging.

optional arguments:
  -f MESSAGE, --force MESSAGE
                        Merge without checking anything. This requires a reason for auditting purpose, for example:
                        @pytorchbot merge -f 'Minor update to fix lint. Expecting all PR tests to pass'
                        
                        Please use `-f` as last resort, prefer `--ignore-current` to continue the merge ignoring current failures. This will allow currently pending tests to finish and report signal before the merge.
  -i, --ignore-current  Merge while ignoring the currently failing jobs.  Behaves like -f if there are no pending jobs.
  -ic                   Old flag for --ignore-current. Deprecated in favor of -i.
  -r [{viable/strict,main}], --rebase [{viable/strict,main}]
                        Rebase the PR to re run checks before merging.  Accepts viable/strict or main as branch options and will default to viable/strict if not specified.

Revert

usage: @pytorchbot revert -m MESSAGE -c
                          {nosignal,ignoredsignal,landrace,weird,ghfirst}

Revert a merged PR. This requires that you are a Meta employee.

Example:
  @pytorchbot revert -m="This is breaking tests on trunk. hud.pytorch.org/" -c=nosignal

optional arguments:
  -m MESSAGE, --message MESSAGE
                        The reason you are reverting, will be put in the commit message. Must be longer than 3 words.
  -c {nosignal,ignoredsignal,landrace,weird,ghfirst}, --classification {nosignal,ignoredsignal,landrace,weird,ghfirst}
                        A machine-friendly classification of the revert reason.

Rebase

usage: @pytorchbot rebase [-s | -b BRANCH]

Rebase a PR. Rebasing defaults to the stable viable/strict branch of pytorch.
Repeat contributor may use this command to rebase their PR.

optional arguments:
  -s, --stable          [DEPRECATED] Rebase onto viable/strict
  -b BRANCH, --branch BRANCH
                        Branch you would like to rebase to

Label

usage: @pytorchbot label labels [labels ...]

Adds label to a PR or Issue [Can be used on Issues]

positional arguments:
  labels  Labels to add to given Pull Request or Issue [Can be used on Issues]

Dr CI

usage: @pytorchbot drci 

Update Dr. CI. Updates the Dr. CI comment on the PR in case it's gotten out of sync with actual CI results.

cherry-pick

usage: @pytorchbot cherry-pick --onto ONTO [--fixes FIXES] -c
                               {regression,critical,fixnewfeature,docs,release}

Cherry pick a pull request onto a release branch for inclusion in a release

optional arguments:
  --onto ONTO           Branch you would like to cherry pick onto (Example: release/2.1)
  --fixes FIXES         Link to the issue that your PR fixes (Example: https://github.com/pytorch/pytorch/issues/110666)
  -c {regression,critical,fixnewfeature,docs,release}, --classification {regression,critical,fixnewfeature,docs,release}
                        A machine-friendly classification of the cherry-pick reason.

Close

usage: @pytorchbot close

Close a PR [Can be used on issues]

[r-barnes]

@pytorchbot rebase -s

[pytorchmergebot]

@pytorchbot started a rebase job onto refs/remotes/origin/viable/strict. Check the current status here

[pytorchmergebot]

Successfully rebased caffe2_observer onto refs/remotes/origin/viable/strict, please pull locally before adding more changes (for example, via git checkout caffe2_observer && git pull --rebase)

[r-barnes]

Landed with #125711