Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[jit][edge] Use dynamic type instead of union types for schema parsers. #70509

Closed
wants to merge 11 commits into from

Conversation

zhxchen17
Copy link
Contributor

@zhxchen17 zhxchen17 commented Dec 29, 2021

Stack from ghstack:

TypeFactory will construct DynamicType when building on Edge platforms. We use this facility to make FunctionSchema return DynamicType all the time for OptionalType. We don't explicitly use DynamicTypeFactory everywhere because that requires too many changes and will split the entire aten codebase.

Differential Revision: D33306737

TypeFactory will construct DynamicType when building on Edge platforms. We use this facility to make FunctionSchema return DynamicType all the time for OptionalType. We don't explicitly use DynamicTypeFactory everywhere because that requires too many changes and will split the entire aten codebase.

Differential Revision: [D33306737](https://our.internmc.facebook.com/intern/diff/D33306737/)

[ghstack-poisoned]
@pytorch-probot
Copy link

pytorch-probot bot commented Dec 29, 2021

CI Flow Status

⚛️ CI Flow

Ruleset - Version: v1
Ruleset - File: https://github.com/pytorch/pytorch/blob/295486973566d24399dbdfc7ccdb50b43fd0097b/.github/generated-ciflow-ruleset.json
PR ciflow labels: ciflow/default

Workflows Labels (bold enabled) Status
Triggered Workflows
linux-bionic-py3.7-clang9 ciflow/all, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/noarch, ciflow/trunk ✅ triggered
linux-docs ciflow/all, ciflow/cpu, ciflow/default, ciflow/docs, ciflow/linux, ciflow/trunk ✅ triggered
linux-vulkan-bionic-py3.7-clang9 ciflow/all, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/trunk, ciflow/vulkan ✅ triggered
linux-xenial-cuda11.3-py3.7-gcc7 ciflow/all, ciflow/cuda, ciflow/default, ciflow/linux, ciflow/trunk ✅ triggered
linux-xenial-cuda11.3-py3.7-gcc7-bazel-test ciflow/all, ciflow/bazel, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/trunk ✅ triggered
linux-xenial-py3-clang5-mobile-build ciflow/all, ciflow/default, ciflow/linux, ciflow/mobile, ciflow/trunk ✅ triggered
linux-xenial-py3-clang5-mobile-custom-build-static ciflow/all, ciflow/default, ciflow/linux, ciflow/mobile, ciflow/trunk ✅ triggered
linux-xenial-py3.7-clang7-asan ciflow/all, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/sanitizers, ciflow/trunk ✅ triggered
linux-xenial-py3.7-clang7-onnx ciflow/all, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/onnx, ciflow/trunk ✅ triggered
linux-xenial-py3.7-gcc5.4 ciflow/all, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/trunk ✅ triggered
linux-xenial-py3.7-gcc7 ciflow/all, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/trunk ✅ triggered
linux-xenial-py3.7-gcc7-no-ops ciflow/all, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/trunk ✅ triggered
pytorch-linux-xenial-py3-clang5-android-ndk-r19c-gradle-custom-build-single ciflow/all, ciflow/android, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/trunk ✅ triggered
pytorch-linux-xenial-py3-clang5-android-ndk-r19c-gradle-custom-build-single-full-jit ciflow/all, ciflow/android, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/trunk ✅ triggered
win-vs2019-cpu-py3 ciflow/all, ciflow/cpu, ciflow/default, ciflow/trunk, ciflow/win ✅ triggered
win-vs2019-cuda11.3-py3 ciflow/all, ciflow/cuda, ciflow/default, ciflow/trunk, ciflow/win ✅ triggered
Skipped Workflows
caffe2-linux-xenial-py3.7-gcc5.4 ciflow/all, ciflow/cpu, ciflow/linux, ciflow/trunk 🚫 skipped
docker-builds ciflow/all, ciflow/trunk 🚫 skipped
ios-12-5-1-arm64 ciflow/all, ciflow/ios, ciflow/macos, ciflow/trunk 🚫 skipped
ios-12-5-1-arm64-coreml ciflow/all, ciflow/ios, ciflow/macos, ciflow/trunk 🚫 skipped
ios-12-5-1-arm64-custom-ops ciflow/all, ciflow/ios, ciflow/macos, ciflow/trunk 🚫 skipped
ios-12-5-1-arm64-full-jit ciflow/all, ciflow/ios, ciflow/macos, ciflow/trunk 🚫 skipped
ios-12-5-1-arm64-metal ciflow/all, ciflow/ios, ciflow/macos, ciflow/trunk 🚫 skipped
ios-12-5-1-x86-64 ciflow/all, ciflow/ios, ciflow/macos, ciflow/trunk 🚫 skipped
ios-12-5-1-x86-64-coreml ciflow/all, ciflow/ios, ciflow/macos, ciflow/trunk 🚫 skipped
ios-12-5-1-x86-64-full-jit ciflow/all, ciflow/ios, ciflow/macos, ciflow/trunk 🚫 skipped
libtorch-linux-xenial-cuda10.2-py3.7-gcc7 ciflow/all, ciflow/cuda, ciflow/libtorch, ciflow/linux, ciflow/trunk 🚫 skipped
libtorch-linux-xenial-cuda11.3-py3.7-gcc7 ciflow/all, ciflow/cuda, ciflow/libtorch, ciflow/linux, ciflow/trunk 🚫 skipped
linux-binary-conda ciflow/binaries, ciflow/binaries/conda 🚫 skipped
linux-binary-libtorch-cxx11-abi ciflow/binaries, ciflow/binaries/libtorch 🚫 skipped
linux-binary-libtorch-pre-cxx11 ciflow/binaries, ciflow/binaries/libtorch 🚫 skipped
linux-binary-manywheel ciflow/binaries, ciflow/binaries/wheel 🚫 skipped
linux-bionic-cuda10.2-py3.9-gcc7 ciflow/all, ciflow/cuda, ciflow/linux, ciflow/slow, ciflow/trunk 🚫 skipped
linux-docs-push ciflow/all, ciflow/cpu, ciflow/linux, ciflow/scheduled 🚫 skipped
linux-xenial-cuda11.3-py3.7-gcc7-no-ops ciflow/all, ciflow/cuda, ciflow/linux, ciflow/trunk 🚫 skipped
macos-10-15-py3-arm64 ciflow/all, ciflow/macos, ciflow/trunk 🚫 skipped
macos-10-15-py3-lite-interpreter-x86-64 ciflow/all, ciflow/macos, ciflow/trunk 🚫 skipped
macos-11-py3-x86-64 ciflow/all, ciflow/macos, ciflow/trunk 🚫 skipped
parallelnative-linux-xenial-py3.7-gcc5.4 ciflow/all, ciflow/cpu, ciflow/linux, ciflow/trunk 🚫 skipped
periodic-libtorch-linux-bionic-cuda11.5-py3.7-gcc7 ciflow/all, ciflow/cuda, ciflow/libtorch, ciflow/linux, ciflow/scheduled 🚫 skipped
periodic-libtorch-linux-xenial-cuda11.1-py3.7-gcc7 ciflow/all, ciflow/cuda, ciflow/libtorch, ciflow/linux, ciflow/scheduled 🚫 skipped
periodic-linux-bionic-cuda11.5-py3.7-gcc7 ciflow/all, ciflow/cuda, ciflow/linux, ciflow/scheduled 🚫 skipped
periodic-linux-xenial-cuda10.2-py3-gcc7-slow-gradcheck ciflow/all, ciflow/cuda, ciflow/linux, ciflow/scheduled, ciflow/slow, ciflow/slow-gradcheck 🚫 skipped
periodic-linux-xenial-cuda11.1-py3.7-gcc7-debug ciflow/all, ciflow/cuda, ciflow/linux, ciflow/scheduled 🚫 skipped
periodic-win-vs2019-cuda11.1-py3 ciflow/all, ciflow/cuda, ciflow/scheduled, ciflow/win 🚫 skipped
periodic-win-vs2019-cuda11.5-py3 ciflow/all, ciflow/cuda, ciflow/scheduled, ciflow/win 🚫 skipped
pytorch-linux-xenial-py3-clang5-android-ndk-r19c-build ciflow/all, ciflow/android, ciflow/cpu, ciflow/linux, ciflow/trunk 🚫 skipped

You can add a comment to the PR and tag @pytorchbot with the following commands:
# ciflow rerun, "ciflow/default" will always be added automatically
@pytorchbot ciflow rerun

# ciflow rerun with additional labels "-l <ciflow/label_name>", which is equivalent to adding these labels manually and trigger the rerun
@pytorchbot ciflow rerun -l ciflow/scheduled -l ciflow/slow

For more information, please take a look at the CI Flow Wiki.

@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Dec 29, 2021

🔗 Helpful links

💊 CI failures summary and remediations

As of commit 2954869 (more details on the Dr. CI page):


  • 3/3 failures introduced in this PR

🕵️ 2 new failures recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

See GitHub Actions build linux-docs / build-docs (cpp) (1/2)

Step: "Unknown" (full log | diagnosis details | 🔁 rerun)

2022-01-11T00:48:25.1796016Z error: could not l...modules/third_party/zstd/config: Permission denied
2022-01-11T00:48:25.1656750Z http.https://github.com/.extraheader
2022-01-11T00:48:25.1666992Z error: could not lock config file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config: Permission denied
2022-01-11T00:48:25.1681075Z Entering 'third_party/tensorpipe/third_party/pybind11'
2022-01-11T00:48:25.1699024Z http.https://github.com/.extraheader
2022-01-11T00:48:25.1709269Z error: could not lock config file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config: Permission denied
2022-01-11T00:48:25.1721357Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang'
2022-01-11T00:48:25.1740083Z http.https://github.com/.extraheader
2022-01-11T00:48:25.1750641Z error: could not lock config file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config: Permission denied
2022-01-11T00:48:25.1767622Z Entering 'third_party/zstd'
2022-01-11T00:48:25.1786005Z http.https://github.com/.extraheader
2022-01-11T00:48:25.1796016Z error: could not lock config file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/zstd/config: Permission denied
2022-01-11T00:48:25.1861300Z Cleaning up orphan processes
2022-01-11T00:48:25.2094447Z Terminate orphan process: pid (3895) (docker)

See GitHub Actions build linux-docs / build-docs (python) (2/2)

Step: "Build python docs" (full log | diagnosis details | 🔁 rerun)

2022-01-11T00:48:08.7796082Z error: could not l...modules/third_party/zstd/config: Permission denied
2022-01-11T00:48:08.7655280Z http.https://github.com/.extraheader
2022-01-11T00:48:08.7665479Z error: could not lock config file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config: Permission denied
2022-01-11T00:48:08.7679552Z Entering 'third_party/tensorpipe/third_party/pybind11'
2022-01-11T00:48:08.7697269Z http.https://github.com/.extraheader
2022-01-11T00:48:08.7708193Z error: could not lock config file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config: Permission denied
2022-01-11T00:48:08.7721337Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang'
2022-01-11T00:48:08.7739071Z http.https://github.com/.extraheader
2022-01-11T00:48:08.7750287Z error: could not lock config file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config: Permission denied
2022-01-11T00:48:08.7766623Z Entering 'third_party/zstd'
2022-01-11T00:48:08.7784779Z http.https://github.com/.extraheader
2022-01-11T00:48:08.7796082Z error: could not lock config file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/zstd/config: Permission denied
2022-01-11T00:48:08.7859818Z Cleaning up orphan processes


1 failure not recognized by patterns:

Job Step Action
GitHub Actions linux-xenial-py3.7-gcc5.4 / test (docs_test, 1, 1, linux.2xlarge) Test 🔁 rerun

This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

…hema parsers."

TypeFactory will construct DynamicType when building on Edge platforms. We use this facility to make FunctionSchema return DynamicType all the time for OptionalType. We don't explicitly use DynamicTypeFactory everywhere because that requires too many changes and will split the entire aten codebase.

Differential Revision: [D33306737](https://our.internmc.facebook.com/intern/diff/D33306737/)

[ghstack-poisoned]
zhxchen17 added a commit that referenced this pull request Dec 29, 2021
Pull Request resolved: #70509

TypeFactory will construct DynamicType when building on Edge platforms. We use this facility to make FunctionSchema return DynamicType all the time for OptionalType. We don't explicitly use DynamicTypeFactory everywhere because that requires too many changes and will split the entire aten codebase.
ghstack-source-id: 146337362

Differential Revision: [D33306737](https://our.internmc.facebook.com/intern/diff/D33306737/)
…hema parsers."

TypeFactory will construct DynamicType when building on Edge platforms. We use this facility to make FunctionSchema return DynamicType all the time for OptionalType. We don't explicitly use DynamicTypeFactory everywhere because that requires too many changes and will split the entire aten codebase.

Differential Revision: [D33306737](https://our.internmc.facebook.com/intern/diff/D33306737/)

[ghstack-poisoned]
zhxchen17 added a commit that referenced this pull request Dec 30, 2021
Pull Request resolved: #70509

TypeFactory will construct DynamicType when building on Edge platforms. We use this facility to make FunctionSchema return DynamicType all the time for OptionalType. We don't explicitly use DynamicTypeFactory everywhere because that requires too many changes and will split the entire aten codebase.
ghstack-source-id: 146351141

Differential Revision: [D33306737](https://our.internmc.facebook.com/intern/diff/D33306737/)
…hema parsers."

TypeFactory will construct DynamicType when building on Edge platforms. We use this facility to make FunctionSchema return DynamicType all the time for OptionalType. We don't explicitly use DynamicTypeFactory everywhere because that requires too many changes and will split the entire aten codebase.

Differential Revision: [D33306737](https://our.internmc.facebook.com/intern/diff/D33306737/)

[ghstack-poisoned]
zhxchen17 added a commit that referenced this pull request Jan 4, 2022
Pull Request resolved: #70509

TypeFactory will construct DynamicType when building on Edge platforms. We use this facility to make FunctionSchema return DynamicType all the time for OptionalType. We don't explicitly use DynamicTypeFactory everywhere because that requires too many changes and will split the entire aten codebase.
ghstack-source-id: 146487894

Differential Revision: [D33306737](https://our.internmc.facebook.com/intern/diff/D33306737/)
Copy link
Contributor

@iseeyuan iseeyuan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Could you add the build size result in the comment of this PR? Thanks!

…hema parsers."

TypeFactory will construct DynamicType when building on Edge platforms. We use this facility to make FunctionSchema return DynamicType all the time for OptionalType. We don't explicitly use DynamicTypeFactory everywhere because that requires too many changes and will split the entire aten codebase.

Differential Revision: [D33306737](https://our.internmc.facebook.com/intern/diff/D33306737/)

[ghstack-poisoned]
zhxchen17 added a commit that referenced this pull request Jan 5, 2022
Pull Request resolved: #70509

TypeFactory will construct DynamicType when building on Edge platforms. We use this facility to make FunctionSchema return DynamicType all the time for OptionalType. We don't explicitly use DynamicTypeFactory everywhere because that requires too many changes and will split the entire aten codebase.
ghstack-source-id: 146564654

Differential Revision: [D33306737](https://our.internmc.facebook.com/intern/diff/D33306737/)
…hema parsers."

TypeFactory will construct DynamicType when building on Edge platforms. We use this facility to make FunctionSchema return DynamicType all the time for OptionalType. We don't explicitly use DynamicTypeFactory everywhere because that requires too many changes and will split the entire aten codebase.

Differential Revision: [D33306737](https://our.internmc.facebook.com/intern/diff/D33306737/)

[ghstack-poisoned]
zhxchen17 added a commit that referenced this pull request Jan 7, 2022
Pull Request resolved: #70509

TypeFactory will construct DynamicType when building on Edge platforms. We use this facility to make FunctionSchema return DynamicType all the time for OptionalType. We don't explicitly use DynamicTypeFactory everywhere because that requires too many changes and will split the entire aten codebase.
ghstack-source-id: 146670519

Differential Revision: [D33306737](https://our.internmc.facebook.com/intern/diff/D33306737/)
…hema parsers."

TypeFactory will construct DynamicType when building on Edge platforms. We use this facility to make FunctionSchema return DynamicType all the time for OptionalType. We don't explicitly use DynamicTypeFactory everywhere because that requires too many changes and will split the entire aten codebase.

Differential Revision: [D33306737](https://our.internmc.facebook.com/intern/diff/D33306737/)

[ghstack-poisoned]
…hema parsers."

TypeFactory will construct DynamicType when building on Edge platforms. We use this facility to make FunctionSchema return DynamicType all the time for OptionalType. We don't explicitly use DynamicTypeFactory everywhere because that requires too many changes and will split the entire aten codebase.

Differential Revision: [D33306737](https://our.internmc.facebook.com/intern/diff/D33306737/)

[ghstack-poisoned]
zhxchen17 added a commit that referenced this pull request Jan 8, 2022
Pull Request resolved: #70509

TypeFactory will construct DynamicType when building on Edge platforms. We use this facility to make FunctionSchema return DynamicType all the time for OptionalType. We don't explicitly use DynamicTypeFactory everywhere because that requires too many changes and will split the entire aten codebase.
ghstack-source-id: 146727336

Differential Revision: [D33306737](https://our.internmc.facebook.com/intern/diff/D33306737/)
…hema parsers."

TypeFactory will construct DynamicType when building on Edge platforms. We use this facility to make FunctionSchema return DynamicType all the time for OptionalType. We don't explicitly use DynamicTypeFactory everywhere because that requires too many changes and will split the entire aten codebase.

Differential Revision: [D33306737](https://our.internmc.facebook.com/intern/diff/D33306737/)

[ghstack-poisoned]
…hema parsers."

TypeFactory will construct DynamicType when building on Edge platforms. We use this facility to make FunctionSchema return DynamicType all the time for OptionalType. We don't explicitly use DynamicTypeFactory everywhere because that requires too many changes and will split the entire aten codebase.

Differential Revision: [D33306737](https://our.internmc.facebook.com/intern/diff/D33306737/)

[ghstack-poisoned]
zhxchen17 added a commit that referenced this pull request Jan 8, 2022
Pull Request resolved: #70509

TypeFactory will construct DynamicType when building on Edge platforms. We use this facility to make FunctionSchema return DynamicType all the time for OptionalType. We don't explicitly use DynamicTypeFactory everywhere because that requires too many changes and will split the entire aten codebase.
ghstack-source-id: 146735644

Differential Revision: [D33306737](https://our.internmc.facebook.com/intern/diff/D33306737/)
…hema parsers."

TypeFactory will construct DynamicType when building on Edge platforms. We use this facility to make FunctionSchema return DynamicType all the time for OptionalType. We don't explicitly use DynamicTypeFactory everywhere because that requires too many changes and will split the entire aten codebase.

Differential Revision: [D33306737](https://our.internmc.facebook.com/intern/diff/D33306737/)

[ghstack-poisoned]
zhxchen17 added a commit that referenced this pull request Jan 11, 2022
Pull Request resolved: #70509

TypeFactory will construct DynamicType when building on Edge platforms. We use this facility to make FunctionSchema return DynamicType all the time for OptionalType. We don't explicitly use DynamicTypeFactory everywhere because that requires too many changes and will split the entire aten codebase.
ghstack-source-id: 146818621

Differential Revision: [D33306737](https://our.internmc.facebook.com/intern/diff/D33306737/)
@facebook-github-bot facebook-github-bot deleted the gh/zhxchen17/71/head branch January 15, 2022 15:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla signed oncall: jit Add this issue/PR to JIT oncall triage queue
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants