[AoTI Minifier] UX Improvement #143330

yushangdi · 2024-12-16T22:14:31Z

Summary:

When a user specify TORCHINDUCTOR_MAX_AUTOTUNE=1 env variable, we add config.max_autotune=True to the generated minifier_launcher
We should do this to other inductor configs as well in a followup Diff

Currently in dynamo and aoti minifier, if a config is overwritten by an env variable, the config will not show up in the config list in the minifier_launcher.py file. As a result, when running the minifier_launcher, they need to re-apply the same env variable.
This is:

not convenient for the users
if they copy-paste the minifier_launcher.py to us without including the env variable, we could be confused and not able to reproduce the error.

Underlying implementation change:

Add env_default parameter to codegen_config(). If set, configs overriden by the env are not considered default.

Test Plan:

 buck2 run 'fbcode//mode/dev-nosan' fbcode//caffe2/test:utils -- -r test_codegen_config

Differential Revision: D67299312

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @ColinPeppler @amjames @desertfire @chauhang @aakhundov

pytorch-bot · 2024-12-16T22:14:35Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/143330

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (10 Unrelated Failures)

As of commit 65f117b with merge base f4e9aeb ():

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

inductor / cuda12.4-py3.10-gcc9-sm86 / build (gh) (trunk failure)
/var/lib/jenkins/workspace/aten/src/ATen/../../../third_party/cutlass/include/cute/stride.hpp:446:63: error: parameter ‘seq_v’ set but not used [-Werror=unused-but-set-parameter]
inductor / unit-test / cuda12.4-py3.10-gcc9-sm86 / build (gh) (trunk failure)
/var/lib/jenkins/workspace/aten/src/ATen/../../../third_party/cutlass/include/cute/stride.hpp:446:63: error: parameter ‘seq_v’ set but not used [-Werror=unused-but-set-parameter]
inductor / unit-test / cuda12.4-py3.12-gcc9-sm86 / build (gh) (trunk failure)
/var/lib/jenkins/workspace/aten/src/ATen/../../../third_party/cutlass/include/cute/stride.hpp:446:63: error: parameter ‘seq_v’ set but not used [-Werror=unused-but-set-parameter]
inductor / unit-test / cuda12.4-py3.13-gcc9-sm86 / build (gh) (trunk failure)
/var/lib/jenkins/workspace/aten/src/ATen/../../../third_party/cutlass/include/cute/stride.hpp:446:63: error: parameter ‘seq_v’ set but not used [-Werror=unused-but-set-parameter]
pull / cuda12.4-py3.10-gcc9-sm75 / build (gh) (trunk failure)
/var/lib/jenkins/workspace/aten/src/ATen/../../../third_party/cutlass/include/cute/stride.hpp:446:63: error: parameter ‘seq_v’ set but not used [-Werror=unused-but-set-parameter]
pull / linux-focal-cuda12.4-py3.10-gcc9 / build (gh) (trunk failure)
/var/lib/jenkins/workspace/aten/src/ATen/../../../third_party/cutlass/include/cute/stride.hpp:446:63: error: parameter ‘seq_v’ set but not used [-Werror=unused-but-set-parameter]
pull / linux-focal-cuda12.4-py3.10-gcc9-sm89 / build (gh) (trunk failure)
/var/lib/jenkins/workspace/aten/src/ATen/../../../third_party/cutlass/include/cute/stride.hpp:446:63: error: parameter ‘seq_v’ set but not used [-Werror=unused-but-set-parameter]
trunk / cuda12.1-py3.10-gcc9-sm80 / build (gh) (trunk failure)
/var/lib/jenkins/workspace/aten/src/ATen/../../../third_party/cutlass/include/cute/stride.hpp:446:63: error: parameter ‘seq_v’ set but not used [-Werror=unused-but-set-parameter]
trunk / libtorch-linux-focal-cuda12.4-py3.10-gcc9-debug / build (gh) (trunk failure)
/var/lib/jenkins/workspace/aten/src/ATen/../../../third_party/cutlass/include/cute/stride.hpp:446:63: error: parameter ‘seq_v’ set but not used [-Werror=unused-but-set-parameter]
trunk / linux-focal-cuda12.4-py3.10-gcc9-no-ops / build (gh) (trunk failure)
/var/lib/jenkins/workspace/aten/src/ATen/../../../third_party/cutlass/include/cute/stride.hpp:446:63: error: parameter ‘seq_v’ set but not used [-Werror=unused-but-set-parameter]

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2024-12-16T22:14:54Z

This pull request was exported from Phabricator. Differential Revision: D67299312

desertfire · 2024-12-17T01:03:33Z

torch/utils/_config_module.py

            skip_default does two things. One if a key has not been modified
                it skips it. The other is it modified the logging behaviour
                to match what codegen already did for modified skipped keys
+            env_default: If set, configs overriden by the env are not considered default.


I think this should always be True?

I'll change it to always be True

desertfire · 2024-12-17T01:05:14Z

Adding more inductor reviewers

eellison

Can you reuse the existing config serialization from minifier? i would not expect this many changes

eellison · 2024-12-17T01:27:20Z

torch/_inductor/config.py


 # enable slow autotuning passes to select algorithms
-max_autotune = os.environ.get("TORCHINDUCTOR_MAX_AUTOTUNE") == "1"
+max_autotune = Config(False, env_name_default="TORCHINDUCTOR_MAX_AUTOTUNE")


Why do we need this ? the existing minifier serializes config settings without this change

@eellison

The problem is that the minifier cannot produce the same config as the run time config.

For example, if you run the script below like TORCHINDUCTOR_MAX_AUTOTUNE=1 python test.py, the configs that the minifier dumps doesn't have torch._inductor.config.max_autotune = True, even though you have env variable TORCHINDUCTOR_MAX_AUTOTUNE=1.

This is because the config module takes the os.environ.get("TORCHINDUCTOR_MAX_AUTOTUNE") == "1" as default and skipped this config, which is wrong (or at least not convenient for the minifier users). The true default value is False, and the user actually changed the value of this config with an env variable.

import torch from torch._inductor import config as inductor_config class Model(torch.nn.Module): def __init__(self): super().__init__() self.fc1 = torch.nn.Linear(10, 16) self.relu = torch.nn.ReLU() self.sigmoid = torch.nn.Sigmoid() def forward(self, x): x = self.fc1(x) x = self.relu(x) x = self.sigmoid(x) return x inductor_config.aot_inductor.dump_aoti_minifier = True torch._inductor.config.triton.inject_relu_bug_TESTING_ONLY = "compile_error" with torch.no_grad(): model = Model().to("cuda") example_inputs = (torch.randn(8, 10).to("cuda"),) ep = torch.export.export(model, example_inputs) package_path = torch._inductor.aoti_compile_and_package(ep) compiled_model = torch._inductor.aoti_load_package(package_path) result = compiled_model(*example_inputs)

Most of the changes here is to add a flag when dumping the configs, so we don't skip the configs that are changed by the setting the env variables.

If we're going to fix max-autotune, it would be great to do a complete fix, not just max autotune. we could do this less invasively by

Recording modified config values - either w/custom logic in setattr, or by saving config on instantiation and diffing with current config

Serialize modfiied config values

serialize any env vars with TORCHINDUCTOR

If we're going to fix max-autotune, it would be great to do a complete fix, not just max autotune. we could do this less invasively by

@eellison Yeah, I'm planning to fix all configs in a follow-up Diff if the approach in this Diff looks ok.

Recording modified config values - either w/custom logic in setattr, or by saving config on instantiation and diffing with current config

Serialize modfiied config values

serialize any env vars with TORCHINDUCTOR

I believe currently we are doing these already in torch/utils/_config_module.py?

I think we should be able to do the above without the max_autotune = Config(False, env_name_default="TORCHINDUCTOR_MAX_AUTOTUNE") changes.. But I guess this is the standard way, so fine with me..

I think we should be able to do the above without the max_autotune = Config(False, env_name_default="TORCHINDUCTOR_MAX_AUTOTUNE") changes.. But I guess this is the standard way, so fine with me..

@eellison yeah I agree, we could do it another way, but since there is the Config class already, I thought we might as well just use it.

It does work, but I would still prefer we do the general thing that covers all configs instead of just this specific one.

It does work, but I would still prefer we do the general thing that covers all configs instead of just this specific one.

@eellison How about I change all configs to use the Config class (like D67303965)? It's a lot of lines to change, but if we do the general thing we still need to touch all configs. (I can do that in a follow-up PR)

We won't be able to just leave the config like max_autotune = os.environ.get("TORCHINDUCTOR_MAX_AUTOTUNE") == "1" untouched.

If we want to record the default, we at least need to change that to two steps:

specify the default: max_autotune = False.

record the change: setattr(max_autotune, os.environ.get("TORCHINDUCTOR_MAX_AUTOTUNE")=="1").

jansel · 2024-12-17T04:44:07Z

torch/_dynamo/config.py

    if DEBUG_DIR_VAR_NAME in os.environ:
        return os.path.join(os.environ[DEBUG_DIR_VAR_NAME], "torch_compile_debug")
    elif is_fbcode():
+        fbcode_dir = os.path.join(os.path.expanduser("~"), "fbsource", "fbcode")


Is this path general? Should this be cwd or home? Putting it in the fbsource tree seems a bit weird.

It's intended to be in the fbcode. The generated TARGETS file and python files need to be in fbsource for them to be compiled by buck.

I guess I could also make it cwd. I'll just delete the elif is_fbcode() branch. The file written to /tmp cannot be compiled by buck.

jansel · 2024-12-17T04:45:37Z

torch/_inductor/config.py

+# def _set_config_from_env(config, env) -> None:
+#     """
+#     Set config to True if env is set to 1.
+#     We use the if statement to avoid calling setattr if env is not set.
+#     This helps config.codegen_config() to only generate configs that are not
+#     the default value.
+#     """
+#     if os.environ.get(env) == "1":
+#         setattr(sys.modules[__name__], config, True)
+
+# _set_config_from_env("max_autotune", "TORCHINDUCTOR_MAX_AUTOTUNE")


jansel · 2024-12-17T04:49:51Z

torch/utils/_config_module.py

        ignored_keys: Optional[List[str]] = None,
        ignored_prefixes: Optional[List[str]] = None,
        skip_default: bool = False,
+        env_default: bool = False,


Does anything break if we just turn this on always? If so let's just remove the option and make it default.

I think it's probably ok. I'll make it default then.

jansel · 2024-12-17T04:50:32Z

torch/_dynamo/repro/aoti.py


    subparsers = parser.add_subparsers(
-        dest="command", metavar="{run,minify,analyze}", required=True
+        dest="command", metavar="{run,minify}", required=True


Was this unused?

yeah, AoTI minifier doesn't have an analyze mode yet

Summary: - When a user specify `TORCHINDUCTOR_MAX_AUTOTUNE=1` env variable, we add `config.max_autotune=True` to the generated minifier_launcher - We should do this to other inductor configs as well in a followup Diff Currently in dynamo and aoti minifier, if a config is overwritten by an env variable, the config will not show up in the config list in the minifier_launcher.py file. As a result, when running the minifier_launcher, they need to re-apply the same env variable. This is: 1) not convenient for the users 2) if they copy-paste the minifier_launcher.py to us without including the env variable, we could be confused and not able to reproduce the error. Underlying implementation change: - Add `env_default` parameter to `codegen_config()`. If set, configs overriden by the env are not considered default. Test Plan: ``` buck2 run 'fbcode//mode/dev-nosan' fbcode//caffe2/test:utils -- -r test_codegen_config ``` Differential Revision: D67299312

facebook-github-bot · 2024-12-17T22:34:07Z

This pull request was exported from Phabricator. Differential Revision: D67299312

yushangdi · 2024-12-18T17:26:43Z

@jansel @desertfire @eellison Pin for review on the updated version of the PR. Thank you!

Summary: - When a user specify env variables that include "TORCH", "INDUCTOR", "DYNAMO" or "TRITON", we add those env variable definitions to the generated minifier_launcher Example env variables generated: ``` import os os.environ['TORCHINDUCTOR_MAX_AUTOTUNE'] = '1' os.environ['PYTORCH_DDP_USE_SIDE_STREAM'] = '0' os.environ['TRITON_CACHE_MANAGER'] = 'triton.runtime.cache:RemoteCacheManager' os.environ['TRITON_REMOTE_CACHE_BACKEND'] = 'triton.fb.fb_memcache:FbMemcacheRemoteKernelCache' os.environ['TRITON_LIBDEVICE_PATH'] = '/data/users/shangdiy/fbsource/buck-out/v2/gen/fbcode/585c17c94f036562/tutorials/pytorch/nntest/__torchtest__/torchtest#link-tree/triton/fb/libdevice.10.bc' os.environ['TRITON_PTXAS_PATH'] = '/data/users/shangdiy/fbsource/buck-out/v2/gen/fbcode/585c17c94f036562/tutorials/pytorch/nntest/__torchtest__/torchtest#link-tree/triton/fb/ptxas' os.environ['TRITON_LIBCUDA_PATH'] = '/usr/local/fbcode/platform010/lib' os.environ['TORCHINDUCTOR_CACHE_DIR'] = '/tmp/torchinductor_shangdiy' ``` Currently in dynamo and aoti minifier, if a config is overwritten by an env variable, the config will not show up in the config list in the minifier_launcher.py file. As a result, when running the minifier_launcher, they need to re-apply the same env variable. This is: 1) not convenient for the users 2) if they copy-paste the minifier_launcher.py to us without including the env variable, we could be confused and not able to reproduce the error. We also make the following change in `codegen_config()`: - Add `env_default` parameter to `codegen_config()`. If set, configs overriden by the env are not considered default. Test Plan: ``` buck2 run 'fbcode//mode/dev-nosan' fbcode//caffe2/test:utils -- -r test_codegen_config ``` Reviewed By: eellison Differential Revision: D67299312

facebook-github-bot · 2025-01-07T02:27:53Z

This pull request was exported from Phabricator. Differential Revision: D67299312

Summary: - When a user specify env variables that include "TORCH", "INDUCTOR", "DYNAMO" or "TRITON", we add those env variable definitions to the generated minifier_launcher Example env variables generated: ``` import os os.environ['TORCHINDUCTOR_MAX_AUTOTUNE'] = '1' os.environ['PYTORCH_DDP_USE_SIDE_STREAM'] = '0' os.environ['TRITON_CACHE_MANAGER'] = 'triton.runtime.cache:RemoteCacheManager' os.environ['TRITON_REMOTE_CACHE_BACKEND'] = 'triton.fb.fb_memcache:FbMemcacheRemoteKernelCache' os.environ['TRITON_LIBDEVICE_PATH'] = '/data/users/shangdiy/fbsource/buck-out/v2/gen/fbcode/585c17c94f036562/tutorials/pytorch/nntest/__torchtest__/torchtest#link-tree/triton/fb/libdevice.10.bc' os.environ['TRITON_PTXAS_PATH'] = '/data/users/shangdiy/fbsource/buck-out/v2/gen/fbcode/585c17c94f036562/tutorials/pytorch/nntest/__torchtest__/torchtest#link-tree/triton/fb/ptxas' os.environ['TRITON_LIBCUDA_PATH'] = '/usr/local/fbcode/platform010/lib' os.environ['TORCHINDUCTOR_CACHE_DIR'] = '/tmp/torchinductor_shangdiy' ``` Currently in dynamo and aoti minifier, if a config is overwritten by an env variable, the config will not show up in the config list in the minifier_launcher.py file. As a result, when running the minifier_launcher, they need to re-apply the same env variable. This is: 1) not convenient for the users 2) if they copy-paste the minifier_launcher.py to us without including the env variable, we could be confused and not able to reproduce the error. We also make the following change in `codegen_config()`: - Add `env_default` parameter to `codegen_config()`. If set, configs overriden by the env are not considered default. Test Plan: ``` buck2 run 'fbcode//mode/dev-nosan' fbcode//caffe2/test:utils -- -r test_codegen_config ``` Reviewed By: eellison Differential Revision: D67299312

facebook-github-bot · 2025-01-07T02:32:29Z

This pull request was exported from Phabricator. Differential Revision: D67299312

Summary: - When a user specify env variables that include "TORCH", "INDUCTOR", "DYNAMO" or "TRITON", we add those env variable definitions to the generated minifier_launcher Example env variables generated: ``` import os os.environ['TORCHINDUCTOR_MAX_AUTOTUNE'] = '1' os.environ['PYTORCH_DDP_USE_SIDE_STREAM'] = '0' os.environ['TRITON_CACHE_MANAGER'] = 'triton.runtime.cache:RemoteCacheManager' os.environ['TRITON_REMOTE_CACHE_BACKEND'] = 'triton.fb.fb_memcache:FbMemcacheRemoteKernelCache' os.environ['TRITON_LIBDEVICE_PATH'] = '/data/users/shangdiy/fbsource/buck-out/v2/gen/fbcode/585c17c94f036562/tutorials/pytorch/nntest/__torchtest__/torchtest#link-tree/triton/fb/libdevice.10.bc' os.environ['TRITON_PTXAS_PATH'] = '/data/users/shangdiy/fbsource/buck-out/v2/gen/fbcode/585c17c94f036562/tutorials/pytorch/nntest/__torchtest__/torchtest#link-tree/triton/fb/ptxas' os.environ['TRITON_LIBCUDA_PATH'] = '/usr/local/fbcode/platform010/lib' os.environ['TORCHINDUCTOR_CACHE_DIR'] = '/tmp/torchinductor_shangdiy' ``` Currently in dynamo and aoti minifier, if a config is overwritten by an env variable, the config will not show up in the config list in the minifier_launcher.py file. As a result, when running the minifier_launcher, they need to re-apply the same env variable. This is: 1) not convenient for the users 2) if they copy-paste the minifier_launcher.py to us without including the env variable, we could be confused and not able to reproduce the error. We also make the following change in `codegen_config()`: - Add `env_default` parameter to `codegen_config()`. If set, configs overriden by the env are not considered default. Test Plan: ``` buck2 run 'fbcode//mode/dev-nosan' fbcode//caffe2/test:utils -- -r test_codegen_config ``` Reviewed By: eellison Differential Revision: D67299312

facebook-github-bot · 2025-01-07T02:37:09Z

This pull request was exported from Phabricator. Differential Revision: D67299312

Summary: - When a user specify env variables that include "TORCH", "INDUCTOR", "DYNAMO" or "TRITON", we add those env variable definitions to the generated minifier_launcher Example env variables generated: ``` import os os.environ['TORCHINDUCTOR_MAX_AUTOTUNE'] = '1' os.environ['PYTORCH_DDP_USE_SIDE_STREAM'] = '0' os.environ['TRITON_CACHE_MANAGER'] = 'triton.runtime.cache:RemoteCacheManager' os.environ['TORCHINDUCTOR_CACHE_DIR'] = '/tmp/torchinductor_shangdiy' os.environ['TRITON_REMOTE_CACHE_BACKEND'] = 'triton.fb.fb_memcache:FbMemcacheRemoteKernelCache' ``` We skip some env vars like below, because they are generated per compile. ``` os.environ['TRITON_LIBDEVICE_PATH'] = '/data/users/shangdiy/fbsource/buck-out/v2/gen/fbcode/585c17c94f036562/tutorials/pytorch/nntest/__torchtest__/torchtest#link-tree/triton/fb/libdevice.10.bc' os.environ['TRITON_PTXAS_PATH'] = '/data/users/shangdiy/fbsource/buck-out/v2/gen/fbcode/585c17c94f036562/tutorials/pytorch/nntest/__torchtest__/torchtest#link-tree/triton/fb/ptxas' os.environ['TRITON_LIBCUDA_PATH'] = '/usr/local/fbcode/platform010/lib' ``` Currently in dynamo and aoti minifier, if a config is overwritten by an env variable, the config will not show up in the config list in the minifier_launcher.py file. As a result, when running the minifier_launcher, they need to re-apply the same env variable. This is: 1) not convenient for the users 2) if they copy-paste the minifier_launcher.py to us without including the env variable, we could be confused and not able to reproduce the error. We also make the following change in `codegen_config()`: - Add `env_default` parameter to `codegen_config()`. If set, configs overriden by the env are not considered default. Test Plan: ``` buck2 run 'fbcode//mode/dev-nosan' fbcode//caffe2/test:utils -- -r test_codegen_config ``` Reviewed By: eellison Differential Revision: D67299312

facebook-github-bot · 2025-01-07T02:44:17Z

This pull request was exported from Phabricator. Differential Revision: D67299312

eellison

🚀 🚀 🚀

facebook-github-bot · 2025-01-07T19:56:55Z

@pytorchbot merge

(Initiating merge automatically since Phabricator Diff has merged)

pytorchmergebot · 2025-01-07T19:58:39Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

facebook-github-bot · 2025-01-09T23:28:04Z

This pull request has been reverted by 66ce13b. To re-land this change, please open another pull request, assignthe same reviewers, fix the CI failures that caused the revert and make sure that the failing CI runs on the PR by applying the proper ciflow label (e.g., ciflow/trunk).

yushangdi · 2025-01-09T23:31:33Z

This pull request has been reverted by 66ce13b. To re-land this change, please open another pull request, assignthe same reviewers, fix the CI failures that caused the revert and make sure that the failing CI runs on the PR by applying the proper ciflow label (e.g., ciflow/trunk).

Note: this Diff is only partially reverted in [66ce13b]. The majority of the Diff is not reverted.

pytorch-bot bot added ciflow/inductor module: dynamo module: inductor labels Dec 16, 2024

facebook-github-bot added the fb-exported label Dec 16, 2024

yushangdi requested a review from desertfire December 16, 2024 22:18

yushangdi added the topic: not user facing topic category label Dec 16, 2024

desertfire reviewed Dec 17, 2024

View reviewed changes

desertfire requested review from eellison and jansel December 17, 2024 01:03

eellison reviewed Dec 17, 2024

View reviewed changes

jansel requested changes Dec 17, 2024

View reviewed changes

yushangdi force-pushed the export-D67299312 branch from d793d5b to 4bdfebf Compare December 17, 2024 22:33

yushangdi requested review from desertfire and jansel December 17, 2024 23:02

yushangdi requested a review from eellison December 18, 2024 17:27

jansel approved these changes Dec 19, 2024

View reviewed changes

pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Dec 19, 2024

yushangdi force-pushed the export-D67299312 branch from 4bdfebf to 4215699 Compare January 7, 2025 02:27

yushangdi force-pushed the export-D67299312 branch from 4215699 to 66dffef Compare January 7, 2025 02:32

yushangdi force-pushed the export-D67299312 branch from 66dffef to ff6427a Compare January 7, 2025 02:36

yushangdi force-pushed the export-D67299312 branch from ff6427a to 65f117b Compare January 7, 2025 02:44

eellison approved these changes Jan 7, 2025

View reviewed changes

pytorchmergebot added the merging label Jan 7, 2025

pytorchmergebot added the Merged label Jan 7, 2025

pytorchmergebot closed this in 72e8f34 Jan 7, 2025

pytorchmergebot removed the merging label Jan 7, 2025

facebook-github-bot added the Reverted label Jan 9, 2025

yushangdi removed the Reverted label Jan 9, 2025

[AoTI Minifier] UX Improvement #143330

[AoTI Minifier] UX Improvement #143330

Uh oh!

Conversation

yushangdi commented Dec 16, 2024 • edited by pytorch-bot bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Dec 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/143330

✅ You can merge normally! (10 Unrelated Failures)

Uh oh!

facebook-github-bot commented Dec 16, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

desertfire commented Dec 17, 2024

Uh oh!

eellison left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yushangdi Dec 17, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

eellison Dec 17, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

eellison Dec 17, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yushangdi Dec 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yushangdi Dec 17, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot commented Dec 17, 2024

Uh oh!

yushangdi commented Dec 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

facebook-github-bot commented Jan 7, 2025

Uh oh!

facebook-github-bot commented Jan 7, 2025

Uh oh!

facebook-github-bot commented Jan 7, 2025

Uh oh!

facebook-github-bot commented Jan 7, 2025

yushangdi commented Dec 16, 2024 •

edited by pytorch-bot bot

Loading

pytorch-bot bot commented Dec 16, 2024 •

edited

Loading

yushangdi Dec 17, 2024 •

edited

Loading

eellison Dec 17, 2024 •

edited

Loading

eellison Dec 17, 2024 •

edited

Loading

yushangdi Dec 18, 2024 •

edited

Loading

yushangdi Dec 17, 2024 •

edited

Loading

yushangdi commented Dec 18, 2024 •

edited

Loading