Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix bug in upgrader generation in mobile #71578

Closed

Conversation

tugsbayasgalan
Copy link
Contributor

Summary: Use more robust way of extracting upgrader min and max versions

Test Plan: omgitsgreen

Differential Revision: D33690113

@pytorch-probot
Copy link

pytorch-probot bot commented Jan 20, 2022

CI Flow Status

⚛️ CI Flow

Ruleset - Version: v1
Ruleset - File: https://github.com/tugsbayasgalan/pytorch/blob/1e5e4431e503610031cdff36b37441235fb933a7/.github/generated-ciflow-ruleset.json
PR ciflow labels: ciflow/default

Workflows Labels (bold enabled) Status
Triggered Workflows
linux-binary-conda ciflow/binaries, ciflow/binaries_conda, ciflow/default ✅ triggered
linux-binary-libtorch-cxx11-abi ciflow/binaries, ciflow/binaries_libtorch, ciflow/default ✅ triggered
linux-binary-libtorch-pre-cxx11 ciflow/binaries, ciflow/binaries_libtorch, ciflow/default ✅ triggered
linux-binary-manywheel ciflow/binaries, ciflow/binaries_wheel, ciflow/default ✅ triggered
linux-bionic-py3.7-clang9 ciflow/all, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/noarch, ciflow/trunk, ciflow/xla ✅ triggered
linux-docs ciflow/all, ciflow/cpu, ciflow/default, ciflow/docs, ciflow/linux, ciflow/trunk ✅ triggered
linux-vulkan-bionic-py3.7-clang9 ciflow/all, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/trunk, ciflow/vulkan ✅ triggered
linux-xenial-cuda11.3-py3.7-gcc7 ciflow/all, ciflow/cuda, ciflow/default, ciflow/linux, ciflow/trunk ✅ triggered
linux-xenial-cuda11.3-py3.7-gcc7-bazel-test ciflow/all, ciflow/bazel, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/trunk ✅ triggered
linux-xenial-py3-clang5-mobile-build ciflow/all, ciflow/default, ciflow/linux, ciflow/mobile, ciflow/trunk ✅ triggered
linux-xenial-py3-clang5-mobile-custom-build-static ciflow/all, ciflow/default, ciflow/linux, ciflow/mobile, ciflow/trunk ✅ triggered
linux-xenial-py3.7-clang7-asan ciflow/all, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/sanitizers, ciflow/trunk ✅ triggered
linux-xenial-py3.7-clang7-onnx ciflow/all, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/onnx, ciflow/trunk ✅ triggered
linux-xenial-py3.7-gcc5.4 ciflow/all, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/trunk ✅ triggered
linux-xenial-py3.7-gcc7 ciflow/all, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/trunk ✅ triggered
linux-xenial-py3.7-gcc7-no-ops ciflow/all, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/trunk ✅ triggered
pytorch-linux-xenial-py3-clang5-android-ndk-r19c-gradle-custom-build-single ciflow/all, ciflow/android, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/trunk ✅ triggered
pytorch-linux-xenial-py3-clang5-android-ndk-r19c-gradle-custom-build-single-full-jit ciflow/all, ciflow/android, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/trunk ✅ triggered
win-vs2019-cpu-py3 ciflow/all, ciflow/cpu, ciflow/default, ciflow/trunk, ciflow/win ✅ triggered
win-vs2019-cuda11.3-py3 ciflow/all, ciflow/cuda, ciflow/default, ciflow/trunk, ciflow/win ✅ triggered
Skipped Workflows
caffe2-linux-xenial-py3.7-gcc5.4 ciflow/all, ciflow/cpu, ciflow/linux, ciflow/trunk 🚫 skipped
docker-builds ciflow/all, ciflow/trunk 🚫 skipped
ios-12-5-1-arm64 ciflow/all, ciflow/ios, ciflow/macos, ciflow/trunk 🚫 skipped
ios-12-5-1-arm64-coreml ciflow/all, ciflow/ios, ciflow/macos, ciflow/trunk 🚫 skipped
ios-12-5-1-arm64-custom-ops ciflow/all, ciflow/ios, ciflow/macos, ciflow/trunk 🚫 skipped
ios-12-5-1-arm64-full-jit ciflow/all, ciflow/ios, ciflow/macos, ciflow/trunk 🚫 skipped
ios-12-5-1-arm64-metal ciflow/all, ciflow/ios, ciflow/macos, ciflow/trunk 🚫 skipped
ios-12-5-1-x86-64 ciflow/all, ciflow/ios, ciflow/macos, ciflow/trunk 🚫 skipped
ios-12-5-1-x86-64-coreml ciflow/all, ciflow/ios, ciflow/macos, ciflow/trunk 🚫 skipped
ios-12-5-1-x86-64-full-jit ciflow/all, ciflow/ios, ciflow/macos, ciflow/trunk 🚫 skipped
libtorch-linux-xenial-cuda10.2-py3.7-gcc7 ciflow/all, ciflow/cuda, ciflow/libtorch, ciflow/linux, ciflow/trunk 🚫 skipped
libtorch-linux-xenial-cuda11.3-py3.7-gcc7 ciflow/all, ciflow/cuda, ciflow/libtorch, ciflow/linux, ciflow/trunk 🚫 skipped
linux-bionic-cuda10.2-py3.9-gcc7 ciflow/all, ciflow/cuda, ciflow/linux, ciflow/slow, ciflow/trunk 🚫 skipped
linux-bionic-rocm4.5-py3.7 ciflow/all, ciflow/linux, ciflow/rocm, ciflow/trunk 🚫 skipped
linux-docs-push ciflow/all, ciflow/cpu, ciflow/linux, ciflow/scheduled 🚫 skipped
linux-xenial-cuda11.3-py3.7-gcc7-no-ops ciflow/all, ciflow/cuda, ciflow/linux, ciflow/trunk 🚫 skipped
macos-10-15-py3-arm64 ciflow/all, ciflow/macos, ciflow/trunk 🚫 skipped
macos-10-15-py3-lite-interpreter-x86-64 ciflow/all, ciflow/macos, ciflow/trunk 🚫 skipped
macos-11-py3-x86-64 ciflow/all, ciflow/macos, ciflow/trunk 🚫 skipped
parallelnative-linux-xenial-py3.7-gcc5.4 ciflow/all, ciflow/cpu, ciflow/linux, ciflow/trunk 🚫 skipped
periodic-libtorch-linux-bionic-cuda11.5-py3.7-gcc7 ciflow/all, ciflow/cuda, ciflow/libtorch, ciflow/linux, ciflow/scheduled 🚫 skipped
periodic-libtorch-linux-xenial-cuda11.1-py3.7-gcc7 ciflow/all, ciflow/cuda, ciflow/libtorch, ciflow/linux, ciflow/scheduled 🚫 skipped
periodic-linux-bionic-cuda11.5-py3.7-gcc7 ciflow/all, ciflow/cuda, ciflow/linux, ciflow/scheduled 🚫 skipped
periodic-linux-xenial-cuda10.2-py3-gcc7-slow-gradcheck ciflow/all, ciflow/cuda, ciflow/linux, ciflow/scheduled, ciflow/slow, ciflow/slow-gradcheck 🚫 skipped
periodic-linux-xenial-cuda11.1-py3.7-gcc7-debug ciflow/all, ciflow/cuda, ciflow/linux, ciflow/scheduled 🚫 skipped
periodic-win-vs2019-cuda11.1-py3 ciflow/all, ciflow/cuda, ciflow/scheduled, ciflow/win 🚫 skipped
periodic-win-vs2019-cuda11.5-py3 ciflow/all, ciflow/cuda, ciflow/scheduled, ciflow/win 🚫 skipped
pytorch-linux-xenial-py3-clang5-android-ndk-r19c-build ciflow/all, ciflow/android, ciflow/cpu, ciflow/linux, ciflow/trunk 🚫 skipped

You can add a comment to the PR and tag @pytorchbot with the following commands:
# ciflow rerun, "ciflow/default" will always be added automatically
@pytorchbot ciflow rerun

# ciflow rerun with additional labels "-l <ciflow/label_name>", which is equivalent to adding these labels manually and trigger the rerun
@pytorchbot ciflow rerun -l ciflow/scheduled -l ciflow/slow

For more information, please take a look at the CI Flow Wiki.

@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Jan 20, 2022

🔗 Helpful links

💊 CI failures summary and remediations

As of commit b3990b1 (more details on the Dr. CI page):


  • 1/1 failures introduced in this PR

🕵️ 1 new failure recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

See GitHub Actions build linux-bionic-py3.7-clang9 / test (noarch, 1, 1, linux.2xlarge) (1/1)

Step: "Test" (full log | diagnosis details | 🔁 rerun)

2022-01-28T07:08:59.0065877Z FAIL [0.001s]: tes...d_nan_error_meta_float64 (__main__.TestLinalgMETA)
2022-01-28T07:08:59.0063261Z     return fn(slf, *args, **kwargs)
2022-01-28T07:08:59.0063666Z   File "/opt/conda/lib/python3.7/site-packages/torch/testing/_internal/common_device_type.py", line 788, in dep_fn
2022-01-28T07:08:59.0063934Z     return fn(slf, *args, **kwargs)
2022-01-28T07:08:59.0064210Z   File "test_linalg.py", line 2628, in test_svd_nan_error
2022-01-28T07:08:59.0064391Z     svd(a)
2022-01-28T07:08:59.0064727Z   File "/opt/conda/lib/python3.7/site-packages/torch/testing/_internal/common_utils.py", line 1639, in __exit__
2022-01-28T07:08:59.0065068Z     return super().__exit__(exc_type, exc_value, tb)
2022-01-28T07:08:59.0065297Z AssertionError: _LinAlgError not raised
2022-01-28T07:08:59.0065428Z 
2022-01-28T07:08:59.0065593Z ======================================================================
2022-01-28T07:08:59.0065877Z FAIL [0.001s]: test_svd_nan_error_meta_float64 (__main__.TestLinalgMETA)
2022-01-28T07:08:59.0066217Z ----------------------------------------------------------------------
2022-01-28T07:08:59.0066510Z Traceback (most recent call last):
2022-01-28T07:08:59.0066881Z   File "/opt/conda/lib/python3.7/site-packages/torch/testing/_internal/common_utils.py", line 1753, in wrapper
2022-01-28T07:08:59.0067135Z     method(*args, **kwargs)
2022-01-28T07:08:59.0067568Z   File "/opt/conda/lib/python3.7/site-packages/torch/testing/_internal/common_device_type.py", line 376, in instantiated_test
2022-01-28T07:08:59.0067868Z     result = test(self, **param_kwargs)
2022-01-28T07:08:59.0068290Z   File "/opt/conda/lib/python3.7/site-packages/torch/testing/_internal/common_device_type.py", line 788, in dep_fn
2022-01-28T07:08:59.0068563Z     return fn(slf, *args, **kwargs)
2022-01-28T07:08:59.0068978Z   File "/opt/conda/lib/python3.7/site-packages/torch/testing/_internal/common_device_type.py", line 788, in dep_fn
2022-01-28T07:08:59.0069315Z     return fn(slf, *args, **kwargs)

This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

@facebook-github-bot facebook-github-bot added oncall: jit Add this issue/PR to JIT oncall triage queue fb-exported labels Jan 20, 2022
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D33690113

Copy link
Contributor

@cccclai cccclai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Please also check CI. Thanks!

Comment on lines 4 to 6
namespace c10 {
TypePtr parseType(const std::string& pythonStr);
} // namespace c10
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like these lines were missing in the codegen, could you help with adding them?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I meant adding them as part of gen_mobile_upgraders.py

Comment on lines +10 to +12
#include <ATen/core/ivalue.h>
#include <caffe2/serialize/versions.h>
#include <torch/csrc/jit/mobile/type_parser.h>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These headers may not be needed, if the previous version work.

def _get_operator_version_map() -> Dict[str, List[_UpgraderEntry]]: ...

def _get_upgrader_ranges(name: str) -> List[_UpgraderRange]: ...
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for creating an api for it! Very neat

upgrader_info = list(filter(lambda token: token != "", upgrader_entry.upgrader_name.split('_')))
upgrader_min_version = upgrader_info[2]
upgrader_max_version = upgrader_info[3]
upgrader_ranges = torch._C._get_upgrader_ranges(op_name)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Neat. Thank!

@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D33690113

@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D33690113

@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D33690113

@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D33690113

@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D33690113

@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D33690113

@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D33690113

@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D33690113

@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D33690113

@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D33690113

@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D33690113

@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D33690113

@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D33690113

Summary:
Pull Request resolved: pytorch#71578

Use more robust way of extracting upgrader min and max versions

Test Plan: omgitsgreen

Reviewed By: cccclai

Differential Revision: D33690113

fbshipit-source-id: 5c1f7b90bcd5b57f9984a99ff8f7a7aad1a3c7d8
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D33690113

facebook-github-bot pushed a commit that referenced this pull request Jan 28, 2022
Summary:
Pull Request resolved: #71578

Use more robust way of extracting upgrader min and max versions

Test Plan: omgitsgreen

Reviewed By: cccclai

Differential Revision: D33690113

fbshipit-source-id: 79a964acb26d7ca1354e104710a285b8da3f46d1
pytorchmergebot pushed a commit that referenced this pull request Jan 28, 2022
Summary:
Pull Request resolved: #71578

Use more robust way of extracting upgrader min and max versions

Test Plan: omgitsgreen

Reviewed By: cccclai

Differential Revision: D33690113

fbshipit-source-id: 79a964acb26d7ca1354e104710a285b8da3f46d1
(cherry picked from commit 9e316ee)
cyyever pushed a commit to cyyever/pytorch_private that referenced this pull request Feb 3, 2022
Summary:
Pull Request resolved: pytorch/pytorch#71578

Use more robust way of extracting upgrader min and max versions

Test Plan: omgitsgreen

Reviewed By: cccclai

Differential Revision: D33690113

fbshipit-source-id: 79a964acb26d7ca1354e104710a285b8da3f46d1
(cherry picked from commit 9e316ee)
cyyever pushed a commit to cyyever/pytorch_private that referenced this pull request Feb 3, 2022
Summary:
Pull Request resolved: pytorch/pytorch#71578

Use more robust way of extracting upgrader min and max versions

Test Plan: omgitsgreen

Reviewed By: cccclai

Differential Revision: D33690113

fbshipit-source-id: 79a964acb26d7ca1354e104710a285b8da3f46d1
(cherry picked from commit 9e316ee)
cyyever pushed a commit to cyyever/pytorch_private that referenced this pull request Feb 9, 2022
Summary:
Pull Request resolved: pytorch/pytorch#71578

Use more robust way of extracting upgrader min and max versions

Test Plan: omgitsgreen

Reviewed By: cccclai

Differential Revision: D33690113

fbshipit-source-id: 79a964acb26d7ca1354e104710a285b8da3f46d1
(cherry picked from commit 9e316ee)
cyyever pushed a commit to cyyever/pytorch_private that referenced this pull request Feb 9, 2022
Summary:
Pull Request resolved: pytorch/pytorch#71578

Use more robust way of extracting upgrader min and max versions

Test Plan: omgitsgreen

Reviewed By: cccclai

Differential Revision: D33690113

fbshipit-source-id: 79a964acb26d7ca1354e104710a285b8da3f46d1
(cherry picked from commit 9e316ee)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla signed fb-exported oncall: jit Add this issue/PR to JIT oncall triage queue
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants