Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ONNX] Export split with list of sizes #33161

Closed
wants to merge 3 commits into from

Conversation

neginraoof
Copy link
Contributor

Exporting Split with a dynamic list of split_sizes is not supported.
This PR enables export using onnx SplitToSequence + SequenceAt

@facebook-github-bot facebook-github-bot added the oncall: jit Add this issue/PR to JIT oncall triage queue label Feb 10, 2020
@dr-ci
Copy link

dr-ci bot commented Feb 11, 2020

💊 CircleCI build failures summary and remediations

As of commit 6f8ba4f:

  • 1/4 broken upstream at merge base c917a24 since Feb 10

    Please rebase on the viable/strict branch (expand for instructions)

    Since your merge base is older than viable/strict, run these commands:

    git fetch origin viable/strict
    git rebase viable/strict
    

    Check out the recency history of this "viable master" tracking branch.

  • 1/4 failures introduced in this PR

  • 2/4 recognized as flaky ❄️

    • Re-run these jobs?

Detailed failure analysis

One may explore the probable reasons each build failed interactively on the Dr. CI website.

🕵️ 1 new failure recognized by patterns

The following build failures do not appear to be due to upstream breakage:

See CircleCI build pytorch_xla_linux_xenial_py3_6_clang7_build (1/1)

Step: "Build" (full log | pattern match details)

Feb 13 22:15:19 caused by: Connection refused (os error 111)
Feb 13 22:15:19 +++ eval 'extract_trap_cmd ' 
Feb 13 22:15:19 ++++ extract_trap_cmd 
Feb 13 22:15:19 ++++ printf '%s\n' '' 
Feb 13 22:15:19 +++ printf '%s\n' cleanup 
Feb 13 22:15:19 ++ trap -- ' 
Feb 13 22:15:19 cleanup' EXIT 
Feb 13 22:15:19 ++ which sccache 
Feb 13 22:15:19 ++ sccache --stop-server 
Feb 13 22:15:19 Stopping sccache server... 
Feb 13 22:15:19 error: couldn't connect to server 
Feb 13 22:15:19 caused by: Connection refused (os error 111) 
Feb 13 22:15:19 ++ true 
Feb 13 22:15:19 ++ rm /var/lib/jenkins/sccache_error.log 
Feb 13 22:15:19 rm: cannot remove '/var/lib/jenkins/sccache_error.log': No such file or directory 
Feb 13 22:15:19 ++ true 
Feb 13 22:15:19 ++ SCCACHE_ERROR_LOG=/var/lib/jenkins/sccache_error.log 
Feb 13 22:15:19 ++ SCCACHE_IDLE_TIMEOUT=1200 
Feb 13 22:15:19 ++ RUST_LOG=sccache::server=error 
Feb 13 22:15:19 ++ sccache --start-server 
Feb 13 22:15:19 Starting sccache server... 
Feb 13 22:15:19 ++ sccache --zero-stats 

❄️ 2 failures recognized as flaky

The following build failures have been detected as flaky and may not be your fault:

See CircleCI build binary_linux_libtorch_2_7m_cpu_devtoolset7_shared-with-deps_test (1/2)

Step: "Set Up CI Environment After attach_workspace" (full log | pattern match details) ❄️

E: Failed to fetch https://download.docker.com/linux/ubuntu/dists/xenial/stable/binary-amd64/Packages.bz2 Hash Sum mismatch
                                                              96% [39 Packages store 0 B]                             Get:53 http://archive.ubuntu.com/ubuntu xenial-updates/multiverse amd64 Packages [16.8 kB] 
                                                         96% [39 Packages store 0 B]                             Get:54 http://archive.ubuntu.com/ubuntu xenial-updates/multiverse Translation-en [8,468 B] 
                                                               96% [39 Packages store 0 B]                             Get:55 http://archive.ubuntu.com/ubuntu xenial-backports/main Sources [4,848 B] 
                                                        96% [39 Packages store 0 B]                             Get:56 http://archive.ubuntu.com/ubuntu xenial-backports/universe Sources [7,120 B] 
                                                        96% [39 Packages store 0 B]                             Get:57 http://archive.ubuntu.com/ubuntu xenial-backports/main amd64 Packages [7,280 B] 
96% [Waiting for headers] 96% [40 Translation-en store 0 B] [Waiting for headers]                                                         Get:58 http://archive.ubuntu.com/ubuntu xenial-backports/main Translation-en [4,456 B] 
 96% [40 Translation-en store 0 B] [Waiting for headers]                                                         Get:59 http://archive.ubuntu.com/ubuntu xenial-backports/universe amd64 Packages [8,064 B] 
 96% [40 Translation-en store 0 B] [Waiting for headers]                                                         Get:60 http://archive.ubuntu.com/ubuntu xenial-backports/universe Translation-en [4,328 B] 
100% [60 Translation-en store 0 B]                                4,807 kB/s 0s 100% [Working]                                                    4,807 kB/s 0s                                                                                 Fetched 28.9 MB in 6s (4,285 kB/s) 
Reading package lists... 99%  Reading package lists... Done  
E: Failed to fetch https://download.docker.com/linux/ubuntu/dists/xenial/stable/binary-amd64/Packages.bz2  Hash Sum mismatch 
E: Some index files failed to download. They have been ignored, or old ones used instead. 

See CircleCI build pytorch_linux_xenial_cuda10_1_cudnn7_py3_nogpu_test (2/2)

Step: "Set Up CI Environment After attach_workspace" (full log | pattern match details) ❄️

E: Failed to fetch https://download.docker.com/linux/ubuntu/dists/xenial/stable/binary-amd64/Packages.bz2 Hash Sum mismatch
                                                                  96% [39 Packages store 0 B]                             Get:53 http://archive.ubuntu.com/ubuntu xenial-updates/multiverse amd64 Packages [16.8 kB] 
                                                               96% [39 Packages store 0 B]                             Get:54 http://archive.ubuntu.com/ubuntu xenial-updates/multiverse Translation-en [8,468 B] 
                                                                     96% [39 Packages store 0 B]                             Get:55 http://archive.ubuntu.com/ubuntu xenial-backports/main Sources [4,848 B] 
96% [Waiting for headers] 96% [40 Translation-en store 0 B] [Waiting for headers]                                                         Get:56 http://archive.ubuntu.com/ubuntu xenial-backports/universe Sources [7,120 B] 
                                                                    96% [40 Translation-en store 0 B]                                   Get:57 http://archive.ubuntu.com/ubuntu xenial-backports/main amd64 Packages [7,280 B] 
                                                                     96% [40 Translation-en store 0 B]                                   Get:58 http://archive.ubuntu.com/ubuntu xenial-backports/main Translation-en [4,456 B] 
                                                                           96% [40 Translation-en store 0 B]                                   Get:59 http://archive.ubuntu.com/ubuntu xenial-backports/universe amd64 Packages [8,064 B] 
                                                                     96% [40 Translation-en store 0 B]                                   Get:60 http://archive.ubuntu.com/ubuntu xenial-backports/universe Translation-en [4,328 B] 
100% [60 Translation-en store 0 B]                                4,740 kB/s 0s 100% [Working]                                                    4,740 kB/s 0s                                                                                 Fetched 28.9 MB in 6s (4,677 kB/s) 
Reading package lists... 99%  Reading package lists... Done  
E: Failed to fetch https://download.docker.com/linux/ubuntu/dists/xenial/stable/binary-amd64/Packages.bz2  Hash Sum mismatch 
E: Some index files failed to download. They have been ignored, or old ones used instead. 

🚧 1 upstream failure recognized by patterns:

These builds matched patterns, but were probably caused by upstream breakages:


This comment was automatically generated by Dr. CI (expand for details).Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker.

This comment has been revised 9 times.

@cpuhrsch cpuhrsch requested a review from albanD February 11, 2020 01:38
@cpuhrsch cpuhrsch added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Feb 11, 2020
@cpuhrsch cpuhrsch removed the request for review from albanD February 11, 2020 01:38
@cpuhrsch cpuhrsch added the module: onnx Related to torch.onnx label Feb 11, 2020
@neginraoof neginraoof changed the title [ONNX] Support for split with list of sizes [ONNX] Export split with list of sizes Feb 11, 2020
@neginraoof
Copy link
Contributor Author

Failure is unrelated.
cc @BowenBao @lara-hdr for review. Thanks.

Copy link
Contributor

@facebook-github-bot facebook-github-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@houseroad has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Copy link
Collaborator

@BowenBao BowenBao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, some minor comments regarding comments.

torch/csrc/jit/passes/onnx/peephole.cpp Outdated Show resolved Hide resolved
@@ -661,6 +661,45 @@ static void fuseUnbindListUnpack(Block *b) {
}
}

// Traced Split with list of sizes is being converted to ONNX as SplitToSequence + SequenceAt.
// Example IR
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: we should add ONNX IR version check.

Copy link
Member

@houseroad houseroad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's land it now. But we should have a check on ONNX IR version. Old IR doesn't support sequence I think.

Copy link
Contributor

@facebook-github-bot facebook-github-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@houseroad has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

@houseroad merged this pull request in 9823662.

ttumiel pushed a commit to ttumiel/pytorch that referenced this pull request Mar 4, 2020
Summary:
Exporting Split with a dynamic list of split_sizes is not supported.
This PR enables export using onnx SplitToSequence + SequenceAt
Pull Request resolved: pytorch#33161

Reviewed By: hl475

Differential Revision: D19860152

Pulled By: houseroad

fbshipit-source-id: 300afedc22b01923efb23acd1a3627aa146bb251
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Merged module: onnx Related to torch.onnx oncall: jit Add this issue/PR to JIT oncall triage queue open source triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants