Add an ability to run DALI without GPU #2165
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
CI MESSAGE: [1503498]: BUILD STARTED |
|
CI MESSAGE: [1503498]: BUILD FAILED |
|
CI MESSAGE: [1503505]: BUILD STARTED |
|
CI MESSAGE: [1503505]: BUILD FAILED |
|
CI MESSAGE: [1504379]: BUILD STARTED |
|
CI MESSAGE: [1504495]: BUILD STARTED |
|
CI MESSAGE: [1504379]: BUILD FAILED |
JanuszL
force-pushed
the
cpu_only_core
branch
2 times, most recently
from
254502a
to
046af83
Compare
|
CI MESSAGE: [1504495]: BUILD FAILED |
|
CI MESSAGE: [1505415]: BUILD STARTED |
|
CI MESSAGE: [1505415]: BUILD FAILED |
|
CI MESSAGE: [1505433]: BUILD STARTED |
|
CI MESSAGE: [1505433]: BUILD FAILED |
JanuszL
force-pushed
the
cpu_only_core
branch
2 times, most recently
from
74a4ffd
to
f8496f0
Compare
|
CI MESSAGE: [1506219]: BUILD STARTED |
JanuszL
force-pushed
the
cpu_only_core
branch
2 times, most recently
from
18f38e7
to
5747322
Compare
|
CI MESSAGE: [1506219]: BUILD FAILED |
|
CI MESSAGE: [1506588]: BUILD STARTED |
|
CI MESSAGE: [1506588]: BUILD FAILED |
|
CI MESSAGE: [1506588]: BUILD PASSED |
|
CI MESSAGE: [1577356]: BUILD STARTED |
|
CI MESSAGE: [1577356]: BUILD FAILED |
|
CI MESSAGE: [1577356]: BUILD PASSED |
klecki
approved these changes
Aug 28, 2020
klecki
reviewed
Aug 28, 2020
dali/pipeline/executor/executor.h
Outdated
| "or equal to CPU_ONLY_DEVICE_ID."); | ||
| DALI_ENFORCE(graph_->NumOp(OpType::GPU) == 0 && graph_->NumOp(OpType::MIXED) == 0, | ||
| "Cannot run a pipeline with Mixed/GPU ops in CPU-only mode. Please provide " | ||
| "valid device id or change the operators device."); |
There was a problem hiding this comment.
Small nitpick:
Suggested change
| "valid device id or change the operators device."); | |
| "valid device id or change the operators' device."); |
Signed-off-by: Janusz Lisiecki <jlisiecki@nvidia.com>
|
CI MESSAGE: [1580145]: BUILD STARTED |
|
CI MESSAGE: [1580145]: BUILD FAILED |
|
CI MESSAGE: [1580145]: BUILD PASSED |
klecki
added a commit
to klecki/DALI
that referenced
this pull request
Apr 15, 2022
Remove from TensorVector the AsTensorList method and constructor from external shared_ptr<TensorList> Both allowed to observe the internal state of TensorVector from the outside, breaking the encapsulation. Adjust few places that relied on the changes of internal state to be externally visible: * Initializing the data graph in workspaces. The TV -> TL conversion is done vie Mixed stage. * ArgumentInputs that are produced as TensorList need to be resynced by ShareData instead. This reverts some changes from NVIDIA#2165: - CPU stage cannot be used as direct outputs due to the TensorList/TensorVector mismatch, we can only share data downwards, but we cannot share and preemptivelly expect the allocation to be mirrored. - CPU-only stage still uses Mixed stage, but just with MakeContigous constrained to CPU outputs. Workspace initialization now takes the CPU_ONLY_DEVICE_ID into consideration and does not set the stream (resulting in has_stream() being false, which in turn keeps the AccessOrder in Mixed ops as HostOrder only). Just for the time of tests two optimizations with contiguous TensorVector -> TensorList were disabled. Signed-off-by: Krzysztof Lecki <klecki@nvidia.com>
18 tasks
klecki
added a commit
to klecki/DALI
that referenced
this pull request
Apr 19, 2022
Remove from TensorVector the AsTensorList method and constructor from external shared_ptr<TensorList> Both allowed to observe the internal state of TensorVector from the outside, breaking the encapsulation. Adjust few places that relied on the changes of internal state to be externally visible: * Initializing the data graph in workspaces. The TV -> TL conversion is done vie Mixed stage. * ArgumentInputs that are produced as TensorList need to be resynced by ShareData instead. This reverts some changes from NVIDIA#2165: - CPU stage cannot be used as direct outputs due to the TensorList/TensorVector mismatch, we can only share data downwards, but we cannot share and preemptivelly expect the allocation to be mirrored. - CPU-only stage still uses Mixed stage, but just with MakeContigous constrained to CPU outputs. Workspace initialization now takes the CPU_ONLY_DEVICE_ID into consideration and does not set the stream (resulting in has_stream() being false, which in turn keeps the AccessOrder in Mixed ops as HostOrder only). Just for the time of tests two optimizations with contiguous TensorVector -> TensorList were disabled. Signed-off-by: Krzysztof Lecki <klecki@nvidia.com>
klecki
added a commit
to klecki/DALI
that referenced
this pull request
May 23, 2022
Remove from TensorVector the AsTensorList method and constructor from external shared_ptr<TensorList> Both allowed to observe the internal state of TensorVector from the outside, breaking the encapsulation. Adjust few places that relied on the changes of internal state to be externally visible: * Initializing the data graph in workspaces. The TV -> TL conversion is done vie Mixed stage. * ArgumentInputs that are produced as TensorList need to be resynced by ShareData instead. This reverts some changes from NVIDIA#2165: - CPU stage cannot be used as direct outputs due to the TensorList/TensorVector mismatch, we can only share data downwards, but we cannot share and preemptivelly expect the allocation to be mirrored. - CPU-only stage still uses Mixed stage, but just with MakeContigous constrained to CPU outputs. Workspace initialization now takes the CPU_ONLY_DEVICE_ID into consideration and does not set the stream (resulting in has_stream() being false, which in turn keeps the AccessOrder in Mixed ops as HostOrder only). Just for the time of tests two optimizations with contiguous TensorVector -> TensorList were disabled. Signed-off-by: Krzysztof Lecki <klecki@nvidia.com>
klecki
added a commit
to klecki/DALI
that referenced
this pull request
May 24, 2022
Remove from TensorVector the AsTensorList method and constructor from external shared_ptr<TensorList> Both allowed to observe the internal state of TensorVector from the outside, breaking the encapsulation. Adjust few places that relied on the changes of internal state to be externally visible: * Initializing the data graph in workspaces. The TV -> TL conversion is done vie Mixed stage. * ArgumentInputs that are produced as TensorList need to be resynced by ShareData instead. This reverts some changes from NVIDIA#2165: - CPU stage cannot be used as direct outputs due to the TensorList/TensorVector mismatch, we can only share data downwards, but we cannot share and preemptivelly expect the allocation to be mirrored. - CPU-only stage still uses Mixed stage, but just with MakeContigous constrained to CPU outputs. Workspace initialization now takes the CPU_ONLY_DEVICE_ID into consideration and does not set the stream (resulting in has_stream() being false, which in turn keeps the AccessOrder in Mixed ops as HostOrder only). Just for the time of tests two optimizations with contiguous TensorVector -> TensorList were disabled. Signed-off-by: Krzysztof Lecki <klecki@nvidia.com>
klecki
added a commit
to klecki/DALI
that referenced
this pull request
May 25, 2022
Remove from TensorVector the AsTensorList method and constructor from external shared_ptr<TensorList> Both allowed to observe the internal state of TensorVector from the outside, breaking the encapsulation. Adjust few places that relied on the changes of internal state to be externally visible: * Initializing the data graph in workspaces. The TV -> TL conversion is done vie Mixed stage. * ArgumentInputs that are produced as TensorList need to be resynced by ShareData instead. This reverts some changes from NVIDIA#2165: - CPU stage cannot be used as direct outputs due to the TensorList/TensorVector mismatch, we can only share data downwards, but we cannot share and preemptivelly expect the allocation to be mirrored. - CPU-only stage still uses Mixed stage, but just with MakeContigous constrained to CPU outputs. Workspace initialization now takes the CPU_ONLY_DEVICE_ID into consideration and does not set the stream (resulting in has_stream() being false, which in turn keeps the AccessOrder in Mixed ops as HostOrder only). Just for the time of tests two optimizations with contiguous TensorVector -> TensorList were disabled. Signed-off-by: Krzysztof Lecki <klecki@nvidia.com>
klecki
added a commit
to klecki/DALI
that referenced
this pull request
May 25, 2022
Remove from TensorVector the AsTensorList method and constructor from external shared_ptr<TensorList> Both allowed to observe the internal state of TensorVector from the outside, breaking the encapsulation. Adjust few places that relied on the changes of internal state to be externally visible: * Initializing the data graph in workspaces. The TV -> TL conversion is done vie Mixed stage. * ArgumentInputs that are produced as TensorList need to be resynced by ShareData instead. This reverts some changes from NVIDIA#2165: - CPU stage cannot be used as direct outputs due to the TensorList/TensorVector mismatch, we can only share data downwards, but we cannot share and preemptivelly expect the allocation to be mirrored. - CPU-only stage still uses Mixed stage, but just with MakeContigous constrained to CPU outputs. Workspace initialization now takes the CPU_ONLY_DEVICE_ID into consideration and does not set the stream (resulting in has_stream() being false, which in turn keeps the AccessOrder in Mixed ops as HostOrder only). Just for the time of tests two optimizations with contiguous TensorVector -> TensorList were disabled. Signed-off-by: Krzysztof Lecki <klecki@nvidia.com>
klecki
added a commit
to klecki/DALI
that referenced
this pull request
May 26, 2022
Remove from TensorVector the AsTensorList method and constructor from external shared_ptr<TensorList> Both allowed to observe the internal state of TensorVector from the outside, breaking the encapsulation. Adjust few places that relied on the changes of internal state to be externally visible: * Initializing the data graph in workspaces. The TV -> TL conversion is done vie Mixed stage. * ArgumentInputs that are produced as TensorList need to be resynced by ShareData instead. This reverts some changes from NVIDIA#2165: - CPU stage cannot be used as direct outputs due to the TensorList/TensorVector mismatch, we can only share data downwards, but we cannot share and preemptivelly expect the allocation to be mirrored. - CPU-only stage still uses Mixed stage, but just with MakeContigous constrained to CPU outputs. Workspace initialization now takes the CPU_ONLY_DEVICE_ID into consideration and does not set the stream (resulting in has_stream() being false, which in turn keeps the AccessOrder in Mixed ops as HostOrder only). Just for the time of tests two optimizations with contiguous TensorVector -> TensorList were disabled. Signed-off-by: Krzysztof Lecki <klecki@nvidia.com>
klecki
added a commit
to klecki/DALI
that referenced
this pull request
Jun 22, 2022
Remove from TensorVector the AsTensorList method and constructor from external shared_ptr<TensorList> Both allowed to observe the internal state of TensorVector from the outside, breaking the encapsulation. Adjust few places that relied on the changes of internal state to be externally visible: * Initializing the data graph in workspaces. The TV -> TL conversion is done vie Mixed stage. * ArgumentInputs that are produced as TensorList need to be resynced by ShareData instead. This reverts some changes from NVIDIA#2165: - CPU stage cannot be used as direct outputs due to the TensorList/TensorVector mismatch, we can only share data downwards, but we cannot share and preemptivelly expect the allocation to be mirrored. - CPU-only stage still uses Mixed stage, but just with MakeContigous constrained to CPU outputs. Workspace initialization now takes the CPU_ONLY_DEVICE_ID into consideration and does not set the stream (resulting in has_stream() being false, which in turn keeps the AccessOrder in Mixed ops as HostOrder only). Just for the time of tests two optimizations with contiguous TensorVector -> TensorList were disabled. Signed-off-by: Krzysztof Lecki <klecki@nvidia.com>
klecki
added a commit
to klecki/DALI
that referenced
this pull request
Jun 27, 2022
Remove from TensorVector the AsTensorList method and constructor from external shared_ptr<TensorList> Both allowed to observe the internal state of TensorVector from the outside, breaking the encapsulation. Adjust few places that relied on the changes of internal state to be externally visible: * Initializing the data graph in workspaces. The TV -> TL conversion is done vie Mixed stage. * ArgumentInputs that are produced as TensorList need to be resynced by ShareData instead. This reverts some changes from NVIDIA#2165: - CPU stage cannot be used as direct outputs due to the TensorList/TensorVector mismatch, we can only share data downwards, but we cannot share and preemptivelly expect the allocation to be mirrored. - CPU-only stage still uses Mixed stage, but just with MakeContigous constrained to CPU outputs. Workspace initialization now takes the CPU_ONLY_DEVICE_ID into consideration and does not set the stream (resulting in has_stream() being false, which in turn keeps the AccessOrder in Mixed ops as HostOrder only). Just for the time of tests two optimizations with contiguous TensorVector -> TensorList were disabled. Signed-off-by: Krzysztof Lecki <klecki@nvidia.com>
klecki
added a commit
to klecki/DALI
that referenced
this pull request
Jul 13, 2022
Remove from TensorVector the AsTensorList method and constructor from external shared_ptr<TensorList> Both allowed to observe the internal state of TensorVector from the outside, breaking the encapsulation. Adjust few places that relied on the changes of internal state to be externally visible: * Initializing the data graph in workspaces. The TV -> TL conversion is done vie Mixed stage. * ArgumentInputs that are produced as TensorList need to be resynced by ShareData instead. This reverts some changes from NVIDIA#2165: - CPU stage cannot be used as direct outputs due to the TensorList/TensorVector mismatch, we can only share data downwards, but we cannot share and preemptivelly expect the allocation to be mirrored. - CPU-only stage still uses Mixed stage, but just with MakeContigous constrained to CPU outputs. Workspace initialization now takes the CPU_ONLY_DEVICE_ID into consideration and does not set the stream (resulting in has_stream() being false, which in turn keeps the AccessOrder in Mixed ops as HostOrder only). Memory is set to non pinned when CPU_ONLY_DEVICE_ID is detected. Eager mode optimizations with contiguous TensorVector -> TensorList was disabled - it waits for the rework of TensorVector replacint TensorList. TODO: This commit also removes the External Source optimization. Signed-off-by: Krzysztof Lecki <klecki@nvidia.com>
klecki
added a commit
to klecki/DALI
that referenced
this pull request
Jul 14, 2022
Remove from TensorVector the AsTensorList method and constructor from external shared_ptr<TensorList> Both allowed to observe the internal state of TensorVector from the outside, breaking the encapsulation. Adjust few places that relied on the changes of internal state to be externally visible: * Initializing the data graph in workspaces. The TV -> TL conversion is done vie Mixed stage. * ArgumentInputs that are produced as TensorList need to be resynced by ShareData instead. This reverts some changes from NVIDIA#2165: - CPU stage cannot be used as direct outputs due to the TensorList/TensorVector mismatch, we can only share data downwards, but we cannot share and preemptivelly expect the allocation to be mirrored. - CPU-only stage still uses Mixed stage, but just with MakeContigous constrained to CPU outputs. Workspace initialization now takes the CPU_ONLY_DEVICE_ID into consideration and does not set the stream (resulting in has_stream() being false, which in turn keeps the AccessOrder in Mixed ops as HostOrder only). Memory is set to non pinned when CPU_ONLY_DEVICE_ID is detected. Eager mode optimizations with contiguous TensorVector -> TensorList was disabled - it waits for the rework of TensorVector replacint TensorList. TODO: This commit also removes the External Source optimization. Signed-off-by: Krzysztof Lecki <klecki@nvidia.com>
klecki
added a commit
that referenced
this pull request
Jul 14, 2022
Remove from TensorVector the AsTensorList method and constructor from external shared_ptr<TensorList> Both allowed to observe the internal state of TensorVector from the outside, breaking the encapsulation. Adjust few places that relied on the changes of internal state to be externally visible: * Initializing the data graph in workspaces. The TV -> TL conversion is done vie Mixed stage. * ArgumentInputs that are produced as TensorList need to be resynced by ShareData instead. This reverts some changes from #2165: - CPU stage cannot be used as direct outputs due to the TensorList/TensorVector mismatch, we can only share data downwards, but we cannot share and preemptivelly expect the allocation to be mirrored. - CPU-only stage still uses Mixed stage, but just with MakeContigous constrained to CPU outputs. Workspace initialization now takes the CPU_ONLY_DEVICE_ID into consideration and does not set the stream (resulting in has_stream() being false, which in turn keeps the AccessOrder in Mixed ops as HostOrder only). Memory is set to non pinned when CPU_ONLY_DEVICE_ID is detected. Eager mode optimizations with contiguous TensorVector -> TensorList was disabled - it waits for the rework of TensorVector replacing TensorList. An escape hatch to access the shared_ptr of the allocation was ported to TensorVector from TensorList, to allow for ExternalSource to pass data without copy. Signed-off-by: Krzysztof Lecki <klecki@nvidia.com>
staniewzki
pushed a commit
to staniewzki/DALI
that referenced
this pull request
Jul 19, 2022
Remove from TensorVector the AsTensorList method and constructor from external shared_ptr<TensorList> Both allowed to observe the internal state of TensorVector from the outside, breaking the encapsulation. Adjust few places that relied on the changes of internal state to be externally visible: * Initializing the data graph in workspaces. The TV -> TL conversion is done vie Mixed stage. * ArgumentInputs that are produced as TensorList need to be resynced by ShareData instead. This reverts some changes from NVIDIA#2165: - CPU stage cannot be used as direct outputs due to the TensorList/TensorVector mismatch, we can only share data downwards, but we cannot share and preemptivelly expect the allocation to be mirrored. - CPU-only stage still uses Mixed stage, but just with MakeContigous constrained to CPU outputs. Workspace initialization now takes the CPU_ONLY_DEVICE_ID into consideration and does not set the stream (resulting in has_stream() being false, which in turn keeps the AccessOrder in Mixed ops as HostOrder only). Memory is set to non pinned when CPU_ONLY_DEVICE_ID is detected. Eager mode optimizations with contiguous TensorVector -> TensorList was disabled - it waits for the rework of TensorVector replacing TensorList. An escape hatch to access the shared_ptr of the allocation was ported to TensorVector from TensorList, to allow for ExternalSource to pass data without copy. Signed-off-by: Krzysztof Lecki <klecki@nvidia.com>
szkarpinski
pushed a commit
to szkarpinski/DALI
that referenced
this pull request
Jul 21, 2022
Remove from TensorVector the AsTensorList method and constructor from external shared_ptr<TensorList> Both allowed to observe the internal state of TensorVector from the outside, breaking the encapsulation. Adjust few places that relied on the changes of internal state to be externally visible: * Initializing the data graph in workspaces. The TV -> TL conversion is done vie Mixed stage. * ArgumentInputs that are produced as TensorList need to be resynced by ShareData instead. This reverts some changes from NVIDIA#2165: - CPU stage cannot be used as direct outputs due to the TensorList/TensorVector mismatch, we can only share data downwards, but we cannot share and preemptivelly expect the allocation to be mirrored. - CPU-only stage still uses Mixed stage, but just with MakeContigous constrained to CPU outputs. Workspace initialization now takes the CPU_ONLY_DEVICE_ID into consideration and does not set the stream (resulting in has_stream() being false, which in turn keeps the AccessOrder in Mixed ops as HostOrder only). Memory is set to non pinned when CPU_ONLY_DEVICE_ID is detected. Eager mode optimizations with contiguous TensorVector -> TensorList was disabled - it waits for the rework of TensorVector replacing TensorList. An escape hatch to access the shared_ptr of the allocation was ported to TensorVector from TensorList, to allow for ExternalSource to pass data without copy. Signed-off-by: Krzysztof Lecki <klecki@nvidia.com>
szkarpinski
pushed a commit
to szkarpinski/DALI
that referenced
this pull request
Jul 21, 2022
Remove from TensorVector the AsTensorList method and constructor from external shared_ptr<TensorList> Both allowed to observe the internal state of TensorVector from the outside, breaking the encapsulation. Adjust few places that relied on the changes of internal state to be externally visible: * Initializing the data graph in workspaces. The TV -> TL conversion is done vie Mixed stage. * ArgumentInputs that are produced as TensorList need to be resynced by ShareData instead. This reverts some changes from NVIDIA#2165: - CPU stage cannot be used as direct outputs due to the TensorList/TensorVector mismatch, we can only share data downwards, but we cannot share and preemptivelly expect the allocation to be mirrored. - CPU-only stage still uses Mixed stage, but just with MakeContigous constrained to CPU outputs. Workspace initialization now takes the CPU_ONLY_DEVICE_ID into consideration and does not set the stream (resulting in has_stream() being false, which in turn keeps the AccessOrder in Mixed ops as HostOrder only). Memory is set to non pinned when CPU_ONLY_DEVICE_ID is detected. Eager mode optimizations with contiguous TensorVector -> TensorList was disabled - it waits for the rework of TensorVector replacing TensorList. An escape hatch to access the shared_ptr of the allocation was ported to TensorVector from TensorList, to allow for ExternalSource to pass data without copy. Signed-off-by: Krzysztof Lecki <klecki@nvidia.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why we need this PR?
Pick one, remove the rest
What happened in this PR?
Fill relevant points, put NA otherwise. Replace anything inside []
Adds a special case for device_id that skips all CUDA related calls
Executor
A couple of operators
Some fixes are hacky, check if it can be improved
Added a test case for CPU only scenario
Updated pipeline description
JIRA TASK: [DALI-1491]