[WIP] Add simple `groupby` DataPipe #58150

VitalyFedyunin · 2021-05-12T15:51:40Z

Stack from ghstack:

[WIP] DataPipes Developer docs #59053 [WIP] DataPipes Developer docs
[WIP] Example of DataPipes and DataFrames integration #58597 [WIP] Example of DataPipes and DataFrames integration
[WIP] Add notebook with DataPipes API examples #58153 [WIP] Add notebook with DataPipes API examples
[WIP] Add simple fork DataPipe #58152 [WIP] Add simple fork DataPipe
[WIP] Decorate ZipDataPipe as zip #58151 [WIP] Decorate ZipDataPipe as zip
[WIP] Add simple groupby DataPipe #58150 [WIP] Add simple groupby DataPipe
[WIP] Make batch DataPipe sensitive to batch_level argument #58149 [WIP] Make batch DataPipe sensitive to batch_level argument
[WIP] Add simple unbatch DataPipe #58148 [WIP] Add simple unbatch DataPipe
[WIP] Make filter DataPipe sensitive to batch_level argument #58147 [WIP] Make filter DataPipe sensitive to batch_level argument
[WIP] Make shuffle DataPipe sensitive to batch_level argument #58146 [WIP] Make shuffle DataPipe sensitive to batch_level argument
[DataLoader] Rename transform DataPipe to legacy_transform #58145 [DataLoader] Rename transform DataPipe to legacy_transform

[ghstack-poisoned]

facebook-github-bot · 2021-05-12T15:51:59Z

💊 CI failures summary and remediations

As of commit 05af5c8 (more details on the Dr. CI page and at hud.pytorch.org/pr/58150):

16/16 failures possibly* introduced in this PR
- 1/16 non-scanned failure(s)

🕵️ 10 new failures recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

Lint / mypy (1/10)

Step: "Run mypy" (full log | diagnosis details | 🔁 rerun)

2021-06-24T18:40:48.2471266Z torch/utils/data/d...error: Too many arguments for "_apply" [call-arg]

2021-06-24T18:39:56.2281081Z   LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.8.10/x64/lib
2021-06-24T18:39:56.2281719Z ##[endgroup]
2021-06-24T18:39:56.2401503Z + STATUS=
2021-06-24T18:39:56.2403200Z + for CONFIG in mypy*.ini
2021-06-24T18:39:56.2404835Z + mypy --config=mypy-strict.ini
2021-06-24T18:40:13.8423656Z Success: no issues found in 144 source files
2021-06-24T18:40:14.7013911Z + for CONFIG in mypy*.ini
2021-06-24T18:40:14.7016327Z + mypy --config=mypy.ini
2021-06-24T18:40:48.2463761Z torch/utils/data/datapipes/iter/grouping.py:96:1: error: Name 'UnBatchIterDataPipe' already defined on line 26  [no-redef]
2021-06-24T18:40:48.2465423Z torch/utils/data/datapipes/iter/grouping.py:251:9: error: Need type annotation for 'buffer' (hint: "buffer: Dict[<type>, <type>] = ...")  [var-annotated]
2021-06-24T18:40:48.2471266Z torch/utils/data/datapipes/iter/selecting.py:61:24: error: Too many arguments for "_apply"  [call-arg]
2021-06-24T18:41:30.8586401Z Found 3 errors in 2 files (checked 1385 source files)
2021-06-24T18:41:32.8025842Z + STATUS=fail
2021-06-24T18:41:32.8026951Z + '[' -n fail ']'
2021-06-24T18:41:32.8028151Z + echo 'Please fix the above mypy warnings.'
2021-06-24T18:41:32.8029976Z + false
2021-06-24T18:41:32.8030553Z Please fix the above mypy warnings.
2021-06-24T18:41:32.8039359Z ##[error]Process completed with exit code 1.
2021-06-24T18:41:32.8139315Z Post job cleanup.
2021-06-24T18:41:32.9196062Z [command]/usr/bin/git version
2021-06-24T18:41:32.9289316Z git version 2.32.0

Linux CI (pytorch-linux-xenial-py3.6-gcc5.4) / build (2/10)

Step: "Build PyTorch" (full log | diagnosis details | 🔁 rerun)

2021-06-24T18:47:44.1246069Z rm: cannot remove ...kins/sccache_error.log': No such file or directory

2021-06-24T18:47:44.1207991Z ++++ extract_trap_cmd
2021-06-24T18:47:44.1208607Z ++++ printf '%s\n' ''
2021-06-24T18:47:44.1209043Z +++ printf '%s\n' cleanup
2021-06-24T18:47:44.1209623Z ++ trap -- '
2021-06-24T18:47:44.1210030Z cleanup' EXIT
2021-06-24T18:47:44.1211910Z ++ [[ pytorch-linux-xenial-py3.6-gcc5.4 != *pytorch-win-* ]]
2021-06-24T18:47:44.1212507Z ++ which sccache
2021-06-24T18:47:44.1220067Z ++ sccache --stop-server
2021-06-24T18:47:44.1240395Z ++ true
2021-06-24T18:47:44.1241071Z ++ rm /var/lib/jenkins/sccache_error.log
2021-06-24T18:47:44.1246069Z rm: cannot remove '/var/lib/jenkins/sccache_error.log': No such file or directory
2021-06-24T18:47:44.1247073Z ++ true
2021-06-24T18:47:44.1247886Z ++ [[ -n 1 ]]
2021-06-24T18:47:44.1248955Z ++ echo 'Skipping sccache server initialization, setting environment variables'
2021-06-24T18:47:44.1249778Z Skipping sccache server initialization, setting environment variables
2021-06-24T18:47:44.1250348Z ++ export SCCACHE_IDLE_TIMEOUT=1200
2021-06-24T18:47:44.1250722Z ++ SCCACHE_IDLE_TIMEOUT=1200
2021-06-24T18:47:44.1251191Z ++ export SCCACHE_ERROR_LOG=/var/lib/jenkins/sccache_error.log
2021-06-24T18:47:44.1251725Z ++ SCCACHE_ERROR_LOG=/var/lib/jenkins/sccache_error.log
2021-06-24T18:47:44.1252647Z ++ export RUST_LOG=sccache::server=error
2021-06-24T18:47:44.1253080Z ++ RUST_LOG=sccache::server=error

pytorch_linux_bionic_py3_8_gcc9_coverage_build (3/10)

Step: "Build" (full log | diagnosis details | 🔁 rerun)

Jun 24 19:00:42 rm: cannot remove '/var/lib/jenkins/sccache_error.log': No such file or directory

Jun 24 19:00:42 ++++ extract_trap_cmd
Jun 24 19:00:42 ++++ printf '%s\n' ''
Jun 24 19:00:42 +++ printf '%s\n' cleanup
Jun 24 19:00:42 ++ trap -- '
Jun 24 19:00:42 cleanup' EXIT
Jun 24 19:00:42 ++ [[ pytorch-linux-bionic-py3.8-gcc9-coverage-build != *pytorch-win-* ]]
Jun 24 19:00:42 ++ which sccache
Jun 24 19:00:42 ++ sccache --stop-server
Jun 24 19:00:42 ++ true
Jun 24 19:00:42 ++ rm /var/lib/jenkins/sccache_error.log
Jun 24 19:00:42 rm: cannot remove '/var/lib/jenkins/sccache_error.log': No such file or directory
Jun 24 19:00:42 ++ true
Jun 24 19:00:42 ++ [[ -n '' ]]
Jun 24 19:00:42 ++ [[ pytorch-linux-bionic-py3.8-gcc9-coverage-build == *rocm* ]]
Jun 24 19:00:42 ++ SCCACHE_ERROR_LOG=/var/lib/jenkins/sccache_error.log
Jun 24 19:00:42 ++ SCCACHE_IDLE_TIMEOUT=1200
Jun 24 19:00:42 ++ RUST_LOG=sccache::server=error
Jun 24 19:00:42 ++ sccache --start-server
Jun 24 19:00:42 sccache: Starting the server...
Jun 24 19:00:42 ++ sccache --zero-stats
Jun 24 19:00:42 Compile requests                      0

pytorch_linux_bionic_py3_6_clang9_noarch_build (4/10)

Step: "Build" (full log | diagnosis details | 🔁 rerun)

Jun 24 18:48:02 rm: cannot remove '/var/lib/jenkins/sccache_error.log': No such file or directory

Jun 24 18:48:02 ++++ extract_trap_cmd
Jun 24 18:48:02 ++++ printf '%s\n' ''
Jun 24 18:48:02 +++ printf '%s\n' cleanup
Jun 24 18:48:02 ++ trap -- '
Jun 24 18:48:02 cleanup' EXIT
Jun 24 18:48:02 ++ [[ pytorch-linux-bionic-py3.6-clang9-noarch-build != *pytorch-win-* ]]
Jun 24 18:48:02 ++ which sccache
Jun 24 18:48:02 ++ sccache --stop-server
Jun 24 18:48:02 ++ true
Jun 24 18:48:02 ++ rm /var/lib/jenkins/sccache_error.log
Jun 24 18:48:02 rm: cannot remove '/var/lib/jenkins/sccache_error.log': No such file or directory
Jun 24 18:48:02 ++ true
Jun 24 18:48:02 ++ [[ -n '' ]]
Jun 24 18:48:02 ++ [[ pytorch-linux-bionic-py3.6-clang9-noarch-build == *rocm* ]]
Jun 24 18:48:02 ++ SCCACHE_ERROR_LOG=/var/lib/jenkins/sccache_error.log
Jun 24 18:48:02 ++ SCCACHE_IDLE_TIMEOUT=1200
Jun 24 18:48:02 ++ RUST_LOG=sccache::server=error
Jun 24 18:48:02 ++ sccache --start-server
Jun 24 18:48:02 sccache: Starting the server...
Jun 24 18:48:02 ++ sccache --zero-stats
Jun 24 18:48:02 Compile requests                      0

pytorch_xla_linux_bionic_py3_6_clang9_build (5/10)

Step: "Build" (full log | diagnosis details | 🔁 rerun)

Jun 24 18:48:25 rm: cannot remove '/var/lib/jenkins/sccache_error.log': No such file or directory

Jun 24 18:48:25 ++++ extract_trap_cmd
Jun 24 18:48:25 ++++ printf '%s\n' ''
Jun 24 18:48:25 +++ printf '%s\n' cleanup
Jun 24 18:48:25 ++ trap -- '
Jun 24 18:48:25 cleanup' EXIT
Jun 24 18:48:25 ++ [[ pytorch-xla-linux-bionic-py3.6-clang9-build != *pytorch-win-* ]]
Jun 24 18:48:25 ++ which sccache
Jun 24 18:48:25 ++ sccache --stop-server
Jun 24 18:48:25 ++ true
Jun 24 18:48:25 ++ rm /var/lib/jenkins/sccache_error.log
Jun 24 18:48:25 rm: cannot remove '/var/lib/jenkins/sccache_error.log': No such file or directory
Jun 24 18:48:25 ++ true
Jun 24 18:48:25 ++ [[ -n '' ]]
Jun 24 18:48:25 ++ [[ pytorch-xla-linux-bionic-py3.6-clang9-build == *rocm* ]]
Jun 24 18:48:25 ++ SCCACHE_ERROR_LOG=/var/lib/jenkins/sccache_error.log
Jun 24 18:48:25 ++ SCCACHE_IDLE_TIMEOUT=1200
Jun 24 18:48:25 ++ RUST_LOG=sccache::server=error
Jun 24 18:48:25 ++ sccache --start-server
Jun 24 18:48:25 sccache: Starting the server...
Jun 24 18:48:25 ++ sccache --zero-stats
Jun 24 18:48:25 Compile requests                      0

pytorch_linux_bionic_cuda10_2_cudnn7_py3_9_gcc7_build (6/10)

Step: "Build" (full log | diagnosis details | 🔁 rerun)

Jun 24 18:52:17 rm: cannot remove '/var/lib/jenkins/sccache_error.log': No such file or directory

Jun 24 18:52:17 ++++ extract_trap_cmd
Jun 24 18:52:17 ++++ printf '%s\n' ''
Jun 24 18:52:17 +++ printf '%s\n' cleanup
Jun 24 18:52:17 ++ trap -- '
Jun 24 18:52:17 cleanup' EXIT
Jun 24 18:52:17 ++ [[ pytorch-linux-bionic-cuda10.2-cudnn7-py3.9-gcc7-build != *pytorch-win-* ]]
Jun 24 18:52:17 ++ which sccache
Jun 24 18:52:17 ++ sccache --stop-server
Jun 24 18:52:17 ++ true
Jun 24 18:52:17 ++ rm /var/lib/jenkins/sccache_error.log
Jun 24 18:52:17 rm: cannot remove '/var/lib/jenkins/sccache_error.log': No such file or directory
Jun 24 18:52:17 ++ true
Jun 24 18:52:17 ++ [[ -n '' ]]
Jun 24 18:52:17 ++ [[ pytorch-linux-bionic-cuda10.2-cudnn7-py3.9-gcc7-build == *rocm* ]]
Jun 24 18:52:17 ++ SCCACHE_ERROR_LOG=/var/lib/jenkins/sccache_error.log
Jun 24 18:52:17 ++ SCCACHE_IDLE_TIMEOUT=1200
Jun 24 18:52:17 ++ RUST_LOG=sccache::server=error
Jun 24 18:52:17 ++ sccache --start-server
Jun 24 18:52:17 sccache: Starting the server...
Jun 24 18:52:17 ++ sccache --zero-stats
Jun 24 18:52:17 Compile requests                      0

pytorch_linux_xenial_cuda11_1_cudnn8_py3_gcc7_build (7/10)

Step: "Build" (full log | diagnosis details | 🔁 rerun)

Jun 24 19:21:28 ERROR 2021-06-24T18:57:58Z: scc...eof ((socklen_t)))\n ^\n" }

Jun 24 19:21:28 ERROR 2021-06-24T18:57:52Z: sccache::server: Compilation failed: Output { status: ExitStatus(ExitStatus(256)), stdout: "", stderr: "conftest.c: In function \'main\':\nconftest.c:332:2: error: \'struct sockaddr\' has no member named \'sa_len\'\n x.sa_len = 0;\n  ^\n" }
Jun 24 19:21:28 
Jun 24 19:21:28 ERROR 2021-06-24T18:57:55Z: sccache::server: Compilation failed: Output { status: ExitStatus(ExitStatus(256)), stdout: "", stderr: "conftest.c: In function \'main\':\nconftest.c:366:10: error: \'RTLD_MEMBER\' undeclared (first use in this function); did you mean \'RTLD_NEXT\'?\n   (void) RTLD_MEMBER;\n          ^~~~~~~~~~~\n          RTLD_NEXT\nconftest.c:366:10: note: each undeclared identifier is reported only once for each function it appears in\n" }
Jun 24 19:21:28 
Jun 24 19:21:28 ERROR 2021-06-24T18:57:55Z: sccache::server: Compilation failed: Output { status: ExitStatus(ExitStatus(256)), stdout: "", stderr: "conftest.c:361:9: error: unknown type name \'not\'\n         not a universal capable compiler\n         ^~~\nconftest.c:361:15: error: expected \'=\', \',\', \';\', \'asm\' or \'__attribute__\' before \'universal\'\n         not a universal capable compiler\n               ^~~~~~~~~\nconftest.c:361:15: error: unknown type name \'universal\'\n" }
Jun 24 19:21:28 
Jun 24 19:21:28 ERROR 2021-06-24T18:57:55Z: sccache::server: Compilation failed: Output { status: ExitStatus(ExitStatus(256)), stdout: "", stderr: "conftest.c: In function \'main\':\nconftest.c:367:4: error: unknown type name \'not\'; did you mean \'ino_t\'?\n    not big endian\n    ^~~\n    ino_t\nconftest.c:367:12: error: expected \'=\', \',\', \';\', \'asm\' or \'__attribute__\' before \'endian\'\n    not big endian\n            ^~~~~~\n" }
Jun 24 19:21:28 
Jun 24 19:21:28 ERROR 2021-06-24T18:57:57Z: sccache::server: Compilation failed: Output { status: ExitStatus(ExitStatus(256)), stdout: "", stderr: "conftest.c: In function \'main\':\nconftest.c:378:4: error: \'struct stat\' has no member named \'st_mtimespec\'; did you mean \'st_mtim\'?\n st.st_mtimespec.tv_nsec = 1;\n    ^~~~~~~~~~~~\n    st_mtim\n" }
Jun 24 19:21:28 
Jun 24 19:21:28 ERROR 2021-06-24T18:57:58Z: sccache::server: Compilation failed: Output { status: ExitStatus(ExitStatus(256)), stdout: "", stderr: "conftest.c: In function \'main\':\nconftest.c:402:24: error: expected expression before \')\' token\n if (sizeof ((socklen_t)))\n                        ^\n" }
Jun 24 19:21:28 
Jun 24 19:21:28 =========== If your build fails, please take a look at the log above for possible reasons ===========
Jun 24 19:21:28 Compile requests                  10249
Jun 24 19:21:28 Compile requests executed          6144
Jun 24 19:21:28 Cache hits                         6069
Jun 24 19:21:28 Cache hits (C/C++)                 5504
Jun 24 19:21:28 Cache hits (CUDA)                   565
Jun 24 19:21:28 Cache misses                          7
Jun 24 19:21:28 Cache misses (C/C++)                  7
Jun 24 19:21:28 Cache timeouts                        0

pytorch_linux_xenial_py3_6_gcc5_4_build (8/10)

Step: "Build" (full log | diagnosis details | 🔁 rerun)

Jun 24 18:46:01 rm: cannot remove '/var/lib/jenkins/sccache_error.log': No such file or directory

Jun 24 18:46:01 ++++ extract_trap_cmd
Jun 24 18:46:01 ++++ printf '%s\n' ''
Jun 24 18:46:01 +++ printf '%s\n' cleanup
Jun 24 18:46:01 ++ trap -- '
Jun 24 18:46:01 cleanup' EXIT
Jun 24 18:46:01 ++ [[ pytorch-linux-xenial-py3.6-gcc5.4-build != *pytorch-win-* ]]
Jun 24 18:46:01 ++ which sccache
Jun 24 18:46:01 ++ sccache --stop-server
Jun 24 18:46:01 ++ true
Jun 24 18:46:01 ++ rm /var/lib/jenkins/sccache_error.log
Jun 24 18:46:01 rm: cannot remove '/var/lib/jenkins/sccache_error.log': No such file or directory
Jun 24 18:46:01 ++ true
Jun 24 18:46:01 ++ [[ -n '' ]]
Jun 24 18:46:01 ++ [[ pytorch-linux-xenial-py3.6-gcc5.4-build == *rocm* ]]
Jun 24 18:46:01 ++ SCCACHE_ERROR_LOG=/var/lib/jenkins/sccache_error.log
Jun 24 18:46:01 ++ SCCACHE_IDLE_TIMEOUT=1200
Jun 24 18:46:01 ++ RUST_LOG=sccache::server=error
Jun 24 18:46:01 ++ sccache --start-server
Jun 24 18:46:01 sccache: Starting the server...
Jun 24 18:46:02 ++ sccache --zero-stats
Jun 24 18:46:02 Compile requests                      0

pytorch_vulkan_linux_bionic_py3_6_clang9_build (9/10)

Step: "Build" (full log | diagnosis details | 🔁 rerun)

Jun 24 18:48:22 rm: cannot remove '/var/lib/jenkins/sccache_error.log': No such file or directory

Jun 24 18:48:22 ++++ extract_trap_cmd
Jun 24 18:48:22 ++++ printf '%s\n' ''
Jun 24 18:48:22 +++ printf '%s\n' cleanup
Jun 24 18:48:22 ++ trap -- '
Jun 24 18:48:22 cleanup' EXIT
Jun 24 18:48:22 ++ [[ pytorch-vulkan-linux-bionic-py3.6-clang9-build != *pytorch-win-* ]]
Jun 24 18:48:22 ++ which sccache
Jun 24 18:48:22 ++ sccache --stop-server
Jun 24 18:48:22 ++ true
Jun 24 18:48:22 ++ rm /var/lib/jenkins/sccache_error.log
Jun 24 18:48:22 rm: cannot remove '/var/lib/jenkins/sccache_error.log': No such file or directory
Jun 24 18:48:22 ++ true
Jun 24 18:48:22 ++ [[ -n '' ]]
Jun 24 18:48:22 ++ [[ pytorch-vulkan-linux-bionic-py3.6-clang9-build == *rocm* ]]
Jun 24 18:48:22 ++ SCCACHE_ERROR_LOG=/var/lib/jenkins/sccache_error.log
Jun 24 18:48:22 ++ SCCACHE_IDLE_TIMEOUT=1200
Jun 24 18:48:22 ++ RUST_LOG=sccache::server=error
Jun 24 18:48:22 ++ sccache --start-server
Jun 24 18:48:22 sccache: Starting the server...
Jun 24 18:48:22 ++ sccache --zero-stats
Jun 24 18:48:22 Compile requests                      0

pytorch_linux_xenial_py3_clang7_onnx_build (10/10)

Step: "Build" (full log | diagnosis details | 🔁 rerun)

Jun 24 18:49:35 rm: cannot remove '/var/lib/jenkins/sccache_error.log': No such file or directory

Jun 24 18:49:35 ++++ extract_trap_cmd
Jun 24 18:49:35 ++++ printf '%s\n' ''
Jun 24 18:49:35 +++ printf '%s\n' cleanup
Jun 24 18:49:35 ++ trap -- '
Jun 24 18:49:35 cleanup' EXIT
Jun 24 18:49:35 ++ [[ pytorch-linux-xenial-py3-clang7-onnx-build != *pytorch-win-* ]]
Jun 24 18:49:35 ++ which sccache
Jun 24 18:49:35 ++ sccache --stop-server
Jun 24 18:49:35 ++ true
Jun 24 18:49:35 ++ rm /var/lib/jenkins/sccache_error.log
Jun 24 18:49:35 rm: cannot remove '/var/lib/jenkins/sccache_error.log': No such file or directory
Jun 24 18:49:35 ++ true
Jun 24 18:49:35 ++ [[ -n '' ]]
Jun 24 18:49:35 ++ [[ pytorch-linux-xenial-py3-clang7-onnx-build == *rocm* ]]
Jun 24 18:49:35 ++ SCCACHE_ERROR_LOG=/var/lib/jenkins/sccache_error.log
Jun 24 18:49:35 ++ SCCACHE_IDLE_TIMEOUT=1200
Jun 24 18:49:35 ++ RUST_LOG=sccache::server=error
Jun 24 18:49:35 ++ sccache --start-server
Jun 24 18:49:35 sccache: Starting the server...
Jun 24 18:49:35 ++ sccache --zero-stats
Jun 24 18:49:35 Compile requests                      0

5 failures not recognized by patterns:

Job	Step	Action
^{Windows CI (pytorch-win-vs2019-cuda10-cudnn7-py3) / render_test_results}	^{Download PyTorch Test Reports}	🔁 rerun
^{Lint / quick-checks}	^{Ensure no trailing spaces}	🔁 rerun
^{Lint / flake8-py3}	^{Fail if there were any warnings}	🔁 rerun
^{pytorch_macos_10_13_py3_test}	^Test	🔁 rerun
^{pytorch_linux_xenial_py3_clang5_asan_test1}	^{Run tests}	🔁 rerun

ci.pytorch.org: 1 failed

Failed: pr/pytorch-linux-bionic-rocm4.2-py3.6

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

[ghstack-poisoned]

ghstack-source-id: 0e8eb66 Pull Request resolved: pytorch#58150 update gorupby

ejguan

Checking this PR to understand the drop_remaining and one nit:

torch/utils/data/datapipes/iter/grouping.py

[ghstack-poisoned]

[WIP] Add simple groupby DataPipe

1caa702

[ghstack-poisoned]

This was referenced May 12, 2021

[DataLoader] Rename transform DataPipe to legacy_transform #58145

Closed

[WIP] Make shuffle DataPipe sensitive to batch_level argument #58146

Closed

facebook-github-bot added the cla signed label May 12, 2021

This was referenced May 12, 2021

[WIP] Make filter DataPipe sensitive to batch_level argument #58147

Closed

[WIP] Add simple unbatch DataPipe #58148

Closed

This was referenced May 12, 2021

[WIP] Make batch DataPipe sensitive to batch_level argument #58149

Closed

[WIP] Decorate ZipDataPipe as zip #58151

Closed

[WIP] Add simple fork DataPipe #58152

Closed

[WIP] Add notebook with DataPipes API examples #58153

Closed

Update on "[WIP] Add simple groupby DataPipe"

b5192e3

[ghstack-poisoned]

VitalyFedyunin mentioned this pull request May 19, 2021

[WIP] Example of DataPipes and DataFrames integration #58597

Closed

dgl-intel pushed a commit to dgl-intel/pytorch that referenced this pull request May 19, 2021

[WIP] Add simple groupby DataPipe

d26c6c4

ghstack-source-id: 0e8eb66 Pull Request resolved: pytorch#58150 update gorupby

ejguan reviewed May 19, 2021

View reviewed changes

torch/utils/data/datapipes/iter/grouping.py Show resolved Hide resolved

VitalyFedyunin mentioned this pull request May 26, 2021

[WIP] DataPipes Developer docs #59053

Closed

VitalyFedyunin added 4 commits June 24, 2021 14:08

Update on "[WIP] Add simple groupby DataPipe"

c0f1550

[ghstack-poisoned]

Update on "[WIP] Add simple groupby DataPipe"

f15623e

[ghstack-poisoned]

Update on "[WIP] Add simple groupby DataPipe"

7c6d1fe

[ghstack-poisoned]

Update on "[WIP] Add simple groupby DataPipe"

05af5c8

[ghstack-poisoned]

VitalyFedyunin closed this Jun 30, 2021

facebook-github-bot deleted the gh/VitalyFedyunin/155/head branch July 31, 2021 14:17

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[WIP] Add simple `groupby` DataPipe #58150

[WIP] Add simple `groupby` DataPipe #58150

Uh oh!

VitalyFedyunin commented May 12, 2021 •

edited

Loading

Uh oh!

facebook-github-bot commented May 12, 2021 •

edited

Loading

Uh oh!

ejguan left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

[WIP] Add simple groupby DataPipe #58150

[WIP] Add simple groupby DataPipe #58150

Uh oh!

Conversation

VitalyFedyunin commented May 12, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

facebook-github-bot commented May 12, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💊 CI failures summary and remediations

🕵️ 10 new failures recognized by patterns

Lint / mypy (1/10)

Linux CI (pytorch-linux-xenial-py3.6-gcc5.4) / build (2/10)

pytorch_linux_bionic_py3_8_gcc9_coverage_build (3/10)

pytorch_linux_bionic_py3_6_clang9_noarch_build (4/10)

pytorch_xla_linux_bionic_py3_6_clang9_build (5/10)

pytorch_linux_bionic_cuda10_2_cudnn7_py3_9_gcc7_build (6/10)

pytorch_linux_xenial_cuda11_1_cudnn8_py3_gcc7_build (7/10)

pytorch_linux_xenial_py3_6_gcc5_4_build (8/10)

pytorch_vulkan_linux_bionic_py3_6_clang9_build (9/10)

pytorch_linux_xenial_py3_clang7_onnx_build (10/10)

5 failures not recognized by patterns:

ci.pytorch.org: 1 failed

Uh oh!

ejguan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

[WIP] Add simple `groupby` DataPipe #58150

[WIP] Add simple `groupby` DataPipe #58150

VitalyFedyunin commented May 12, 2021 •

edited

Loading

facebook-github-bot commented May 12, 2021 •

edited

Loading