-
Notifications
You must be signed in to change notification settings - Fork 613
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add an ability to run DALI without GPU #2165
Conversation
CI MESSAGE: [1503498]: BUILD STARTED |
CI MESSAGE: [1503498]: BUILD FAILED |
CI MESSAGE: [1503505]: BUILD STARTED |
CI MESSAGE: [1503505]: BUILD FAILED |
CI MESSAGE: [1504379]: BUILD STARTED |
CI MESSAGE: [1504495]: BUILD STARTED |
CI MESSAGE: [1504379]: BUILD FAILED |
254502a
to
046af83
Compare
CI MESSAGE: [1504495]: BUILD FAILED |
CI MESSAGE: [1505415]: BUILD STARTED |
CI MESSAGE: [1505415]: BUILD FAILED |
CI MESSAGE: [1505433]: BUILD STARTED |
CI MESSAGE: [1505433]: BUILD FAILED |
74a4ffd
to
f8496f0
Compare
CI MESSAGE: [1506219]: BUILD STARTED |
18f38e7
to
5747322
Compare
CI MESSAGE: [1506219]: BUILD FAILED |
CI MESSAGE: [1506588]: BUILD STARTED |
CI MESSAGE: [1506588]: BUILD FAILED |
CI MESSAGE: [1506588]: BUILD PASSED |
CI MESSAGE: [1577356]: BUILD STARTED |
CI MESSAGE: [1577356]: BUILD FAILED |
CI MESSAGE: [1577356]: BUILD PASSED |
dali/pipeline/executor/executor.h
Outdated
"or equal to CPU_ONLY_DEVICE_ID."); | ||
DALI_ENFORCE(graph_->NumOp(OpType::GPU) == 0 && graph_->NumOp(OpType::MIXED) == 0, | ||
"Cannot run a pipeline with Mixed/GPU ops in CPU-only mode. Please provide " | ||
"valid device id or change the operators device."); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Small nitpick:
"valid device id or change the operators device."); | |
"valid device id or change the operators' device."); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
Signed-off-by: Janusz Lisiecki <jlisiecki@nvidia.com>
CI MESSAGE: [1580145]: BUILD STARTED |
CI MESSAGE: [1580145]: BUILD FAILED |
CI MESSAGE: [1580145]: BUILD PASSED |
Remove from TensorVector the AsTensorList method and constructor from external shared_ptr<TensorList> Both allowed to observe the internal state of TensorVector from the outside, breaking the encapsulation. Adjust few places that relied on the changes of internal state to be externally visible: * Initializing the data graph in workspaces. The TV -> TL conversion is done vie Mixed stage. * ArgumentInputs that are produced as TensorList need to be resynced by ShareData instead. This reverts some changes from NVIDIA#2165: - CPU stage cannot be used as direct outputs due to the TensorList/TensorVector mismatch, we can only share data downwards, but we cannot share and preemptivelly expect the allocation to be mirrored. - CPU-only stage still uses Mixed stage, but just with MakeContigous constrained to CPU outputs. Workspace initialization now takes the CPU_ONLY_DEVICE_ID into consideration and does not set the stream (resulting in has_stream() being false, which in turn keeps the AccessOrder in Mixed ops as HostOrder only). Just for the time of tests two optimizations with contiguous TensorVector -> TensorList were disabled. Signed-off-by: Krzysztof Lecki <klecki@nvidia.com>
Remove from TensorVector the AsTensorList method and constructor from external shared_ptr<TensorList> Both allowed to observe the internal state of TensorVector from the outside, breaking the encapsulation. Adjust few places that relied on the changes of internal state to be externally visible: * Initializing the data graph in workspaces. The TV -> TL conversion is done vie Mixed stage. * ArgumentInputs that are produced as TensorList need to be resynced by ShareData instead. This reverts some changes from NVIDIA#2165: - CPU stage cannot be used as direct outputs due to the TensorList/TensorVector mismatch, we can only share data downwards, but we cannot share and preemptivelly expect the allocation to be mirrored. - CPU-only stage still uses Mixed stage, but just with MakeContigous constrained to CPU outputs. Workspace initialization now takes the CPU_ONLY_DEVICE_ID into consideration and does not set the stream (resulting in has_stream() being false, which in turn keeps the AccessOrder in Mixed ops as HostOrder only). Just for the time of tests two optimizations with contiguous TensorVector -> TensorList were disabled. Signed-off-by: Krzysztof Lecki <klecki@nvidia.com>
Remove from TensorVector the AsTensorList method and constructor from external shared_ptr<TensorList> Both allowed to observe the internal state of TensorVector from the outside, breaking the encapsulation. Adjust few places that relied on the changes of internal state to be externally visible: * Initializing the data graph in workspaces. The TV -> TL conversion is done vie Mixed stage. * ArgumentInputs that are produced as TensorList need to be resynced by ShareData instead. This reverts some changes from NVIDIA#2165: - CPU stage cannot be used as direct outputs due to the TensorList/TensorVector mismatch, we can only share data downwards, but we cannot share and preemptivelly expect the allocation to be mirrored. - CPU-only stage still uses Mixed stage, but just with MakeContigous constrained to CPU outputs. Workspace initialization now takes the CPU_ONLY_DEVICE_ID into consideration and does not set the stream (resulting in has_stream() being false, which in turn keeps the AccessOrder in Mixed ops as HostOrder only). Just for the time of tests two optimizations with contiguous TensorVector -> TensorList were disabled. Signed-off-by: Krzysztof Lecki <klecki@nvidia.com>
Remove from TensorVector the AsTensorList method and constructor from external shared_ptr<TensorList> Both allowed to observe the internal state of TensorVector from the outside, breaking the encapsulation. Adjust few places that relied on the changes of internal state to be externally visible: * Initializing the data graph in workspaces. The TV -> TL conversion is done vie Mixed stage. * ArgumentInputs that are produced as TensorList need to be resynced by ShareData instead. This reverts some changes from NVIDIA#2165: - CPU stage cannot be used as direct outputs due to the TensorList/TensorVector mismatch, we can only share data downwards, but we cannot share and preemptivelly expect the allocation to be mirrored. - CPU-only stage still uses Mixed stage, but just with MakeContigous constrained to CPU outputs. Workspace initialization now takes the CPU_ONLY_DEVICE_ID into consideration and does not set the stream (resulting in has_stream() being false, which in turn keeps the AccessOrder in Mixed ops as HostOrder only). Just for the time of tests two optimizations with contiguous TensorVector -> TensorList were disabled. Signed-off-by: Krzysztof Lecki <klecki@nvidia.com>
Remove from TensorVector the AsTensorList method and constructor from external shared_ptr<TensorList> Both allowed to observe the internal state of TensorVector from the outside, breaking the encapsulation. Adjust few places that relied on the changes of internal state to be externally visible: * Initializing the data graph in workspaces. The TV -> TL conversion is done vie Mixed stage. * ArgumentInputs that are produced as TensorList need to be resynced by ShareData instead. This reverts some changes from NVIDIA#2165: - CPU stage cannot be used as direct outputs due to the TensorList/TensorVector mismatch, we can only share data downwards, but we cannot share and preemptivelly expect the allocation to be mirrored. - CPU-only stage still uses Mixed stage, but just with MakeContigous constrained to CPU outputs. Workspace initialization now takes the CPU_ONLY_DEVICE_ID into consideration and does not set the stream (resulting in has_stream() being false, which in turn keeps the AccessOrder in Mixed ops as HostOrder only). Just for the time of tests two optimizations with contiguous TensorVector -> TensorList were disabled. Signed-off-by: Krzysztof Lecki <klecki@nvidia.com>
Remove from TensorVector the AsTensorList method and constructor from external shared_ptr<TensorList> Both allowed to observe the internal state of TensorVector from the outside, breaking the encapsulation. Adjust few places that relied on the changes of internal state to be externally visible: * Initializing the data graph in workspaces. The TV -> TL conversion is done vie Mixed stage. * ArgumentInputs that are produced as TensorList need to be resynced by ShareData instead. This reverts some changes from NVIDIA#2165: - CPU stage cannot be used as direct outputs due to the TensorList/TensorVector mismatch, we can only share data downwards, but we cannot share and preemptivelly expect the allocation to be mirrored. - CPU-only stage still uses Mixed stage, but just with MakeContigous constrained to CPU outputs. Workspace initialization now takes the CPU_ONLY_DEVICE_ID into consideration and does not set the stream (resulting in has_stream() being false, which in turn keeps the AccessOrder in Mixed ops as HostOrder only). Just for the time of tests two optimizations with contiguous TensorVector -> TensorList were disabled. Signed-off-by: Krzysztof Lecki <klecki@nvidia.com>
Remove from TensorVector the AsTensorList method and constructor from external shared_ptr<TensorList> Both allowed to observe the internal state of TensorVector from the outside, breaking the encapsulation. Adjust few places that relied on the changes of internal state to be externally visible: * Initializing the data graph in workspaces. The TV -> TL conversion is done vie Mixed stage. * ArgumentInputs that are produced as TensorList need to be resynced by ShareData instead. This reverts some changes from NVIDIA#2165: - CPU stage cannot be used as direct outputs due to the TensorList/TensorVector mismatch, we can only share data downwards, but we cannot share and preemptivelly expect the allocation to be mirrored. - CPU-only stage still uses Mixed stage, but just with MakeContigous constrained to CPU outputs. Workspace initialization now takes the CPU_ONLY_DEVICE_ID into consideration and does not set the stream (resulting in has_stream() being false, which in turn keeps the AccessOrder in Mixed ops as HostOrder only). Just for the time of tests two optimizations with contiguous TensorVector -> TensorList were disabled. Signed-off-by: Krzysztof Lecki <klecki@nvidia.com>
Remove from TensorVector the AsTensorList method and constructor from external shared_ptr<TensorList> Both allowed to observe the internal state of TensorVector from the outside, breaking the encapsulation. Adjust few places that relied on the changes of internal state to be externally visible: * Initializing the data graph in workspaces. The TV -> TL conversion is done vie Mixed stage. * ArgumentInputs that are produced as TensorList need to be resynced by ShareData instead. This reverts some changes from NVIDIA#2165: - CPU stage cannot be used as direct outputs due to the TensorList/TensorVector mismatch, we can only share data downwards, but we cannot share and preemptivelly expect the allocation to be mirrored. - CPU-only stage still uses Mixed stage, but just with MakeContigous constrained to CPU outputs. Workspace initialization now takes the CPU_ONLY_DEVICE_ID into consideration and does not set the stream (resulting in has_stream() being false, which in turn keeps the AccessOrder in Mixed ops as HostOrder only). Just for the time of tests two optimizations with contiguous TensorVector -> TensorList were disabled. Signed-off-by: Krzysztof Lecki <klecki@nvidia.com>
Remove from TensorVector the AsTensorList method and constructor from external shared_ptr<TensorList> Both allowed to observe the internal state of TensorVector from the outside, breaking the encapsulation. Adjust few places that relied on the changes of internal state to be externally visible: * Initializing the data graph in workspaces. The TV -> TL conversion is done vie Mixed stage. * ArgumentInputs that are produced as TensorList need to be resynced by ShareData instead. This reverts some changes from NVIDIA#2165: - CPU stage cannot be used as direct outputs due to the TensorList/TensorVector mismatch, we can only share data downwards, but we cannot share and preemptivelly expect the allocation to be mirrored. - CPU-only stage still uses Mixed stage, but just with MakeContigous constrained to CPU outputs. Workspace initialization now takes the CPU_ONLY_DEVICE_ID into consideration and does not set the stream (resulting in has_stream() being false, which in turn keeps the AccessOrder in Mixed ops as HostOrder only). Just for the time of tests two optimizations with contiguous TensorVector -> TensorList were disabled. Signed-off-by: Krzysztof Lecki <klecki@nvidia.com>
Remove from TensorVector the AsTensorList method and constructor from external shared_ptr<TensorList> Both allowed to observe the internal state of TensorVector from the outside, breaking the encapsulation. Adjust few places that relied on the changes of internal state to be externally visible: * Initializing the data graph in workspaces. The TV -> TL conversion is done vie Mixed stage. * ArgumentInputs that are produced as TensorList need to be resynced by ShareData instead. This reverts some changes from NVIDIA#2165: - CPU stage cannot be used as direct outputs due to the TensorList/TensorVector mismatch, we can only share data downwards, but we cannot share and preemptivelly expect the allocation to be mirrored. - CPU-only stage still uses Mixed stage, but just with MakeContigous constrained to CPU outputs. Workspace initialization now takes the CPU_ONLY_DEVICE_ID into consideration and does not set the stream (resulting in has_stream() being false, which in turn keeps the AccessOrder in Mixed ops as HostOrder only). Memory is set to non pinned when CPU_ONLY_DEVICE_ID is detected. Eager mode optimizations with contiguous TensorVector -> TensorList was disabled - it waits for the rework of TensorVector replacint TensorList. TODO: This commit also removes the External Source optimization. Signed-off-by: Krzysztof Lecki <klecki@nvidia.com>
Remove from TensorVector the AsTensorList method and constructor from external shared_ptr<TensorList> Both allowed to observe the internal state of TensorVector from the outside, breaking the encapsulation. Adjust few places that relied on the changes of internal state to be externally visible: * Initializing the data graph in workspaces. The TV -> TL conversion is done vie Mixed stage. * ArgumentInputs that are produced as TensorList need to be resynced by ShareData instead. This reverts some changes from NVIDIA#2165: - CPU stage cannot be used as direct outputs due to the TensorList/TensorVector mismatch, we can only share data downwards, but we cannot share and preemptivelly expect the allocation to be mirrored. - CPU-only stage still uses Mixed stage, but just with MakeContigous constrained to CPU outputs. Workspace initialization now takes the CPU_ONLY_DEVICE_ID into consideration and does not set the stream (resulting in has_stream() being false, which in turn keeps the AccessOrder in Mixed ops as HostOrder only). Memory is set to non pinned when CPU_ONLY_DEVICE_ID is detected. Eager mode optimizations with contiguous TensorVector -> TensorList was disabled - it waits for the rework of TensorVector replacint TensorList. TODO: This commit also removes the External Source optimization. Signed-off-by: Krzysztof Lecki <klecki@nvidia.com>
Remove from TensorVector the AsTensorList method and constructor from external shared_ptr<TensorList> Both allowed to observe the internal state of TensorVector from the outside, breaking the encapsulation. Adjust few places that relied on the changes of internal state to be externally visible: * Initializing the data graph in workspaces. The TV -> TL conversion is done vie Mixed stage. * ArgumentInputs that are produced as TensorList need to be resynced by ShareData instead. This reverts some changes from #2165: - CPU stage cannot be used as direct outputs due to the TensorList/TensorVector mismatch, we can only share data downwards, but we cannot share and preemptivelly expect the allocation to be mirrored. - CPU-only stage still uses Mixed stage, but just with MakeContigous constrained to CPU outputs. Workspace initialization now takes the CPU_ONLY_DEVICE_ID into consideration and does not set the stream (resulting in has_stream() being false, which in turn keeps the AccessOrder in Mixed ops as HostOrder only). Memory is set to non pinned when CPU_ONLY_DEVICE_ID is detected. Eager mode optimizations with contiguous TensorVector -> TensorList was disabled - it waits for the rework of TensorVector replacing TensorList. An escape hatch to access the shared_ptr of the allocation was ported to TensorVector from TensorList, to allow for ExternalSource to pass data without copy. Signed-off-by: Krzysztof Lecki <klecki@nvidia.com>
Remove from TensorVector the AsTensorList method and constructor from external shared_ptr<TensorList> Both allowed to observe the internal state of TensorVector from the outside, breaking the encapsulation. Adjust few places that relied on the changes of internal state to be externally visible: * Initializing the data graph in workspaces. The TV -> TL conversion is done vie Mixed stage. * ArgumentInputs that are produced as TensorList need to be resynced by ShareData instead. This reverts some changes from NVIDIA#2165: - CPU stage cannot be used as direct outputs due to the TensorList/TensorVector mismatch, we can only share data downwards, but we cannot share and preemptivelly expect the allocation to be mirrored. - CPU-only stage still uses Mixed stage, but just with MakeContigous constrained to CPU outputs. Workspace initialization now takes the CPU_ONLY_DEVICE_ID into consideration and does not set the stream (resulting in has_stream() being false, which in turn keeps the AccessOrder in Mixed ops as HostOrder only). Memory is set to non pinned when CPU_ONLY_DEVICE_ID is detected. Eager mode optimizations with contiguous TensorVector -> TensorList was disabled - it waits for the rework of TensorVector replacing TensorList. An escape hatch to access the shared_ptr of the allocation was ported to TensorVector from TensorList, to allow for ExternalSource to pass data without copy. Signed-off-by: Krzysztof Lecki <klecki@nvidia.com>
Remove from TensorVector the AsTensorList method and constructor from external shared_ptr<TensorList> Both allowed to observe the internal state of TensorVector from the outside, breaking the encapsulation. Adjust few places that relied on the changes of internal state to be externally visible: * Initializing the data graph in workspaces. The TV -> TL conversion is done vie Mixed stage. * ArgumentInputs that are produced as TensorList need to be resynced by ShareData instead. This reverts some changes from NVIDIA#2165: - CPU stage cannot be used as direct outputs due to the TensorList/TensorVector mismatch, we can only share data downwards, but we cannot share and preemptivelly expect the allocation to be mirrored. - CPU-only stage still uses Mixed stage, but just with MakeContigous constrained to CPU outputs. Workspace initialization now takes the CPU_ONLY_DEVICE_ID into consideration and does not set the stream (resulting in has_stream() being false, which in turn keeps the AccessOrder in Mixed ops as HostOrder only). Memory is set to non pinned when CPU_ONLY_DEVICE_ID is detected. Eager mode optimizations with contiguous TensorVector -> TensorList was disabled - it waits for the rework of TensorVector replacing TensorList. An escape hatch to access the shared_ptr of the allocation was ported to TensorVector from TensorList, to allow for ExternalSource to pass data without copy. Signed-off-by: Krzysztof Lecki <klecki@nvidia.com>
Remove from TensorVector the AsTensorList method and constructor from external shared_ptr<TensorList> Both allowed to observe the internal state of TensorVector from the outside, breaking the encapsulation. Adjust few places that relied on the changes of internal state to be externally visible: * Initializing the data graph in workspaces. The TV -> TL conversion is done vie Mixed stage. * ArgumentInputs that are produced as TensorList need to be resynced by ShareData instead. This reverts some changes from NVIDIA#2165: - CPU stage cannot be used as direct outputs due to the TensorList/TensorVector mismatch, we can only share data downwards, but we cannot share and preemptivelly expect the allocation to be mirrored. - CPU-only stage still uses Mixed stage, but just with MakeContigous constrained to CPU outputs. Workspace initialization now takes the CPU_ONLY_DEVICE_ID into consideration and does not set the stream (resulting in has_stream() being false, which in turn keeps the AccessOrder in Mixed ops as HostOrder only). Memory is set to non pinned when CPU_ONLY_DEVICE_ID is detected. Eager mode optimizations with contiguous TensorVector -> TensorList was disabled - it waits for the rework of TensorVector replacing TensorList. An escape hatch to access the shared_ptr of the allocation was ported to TensorVector from TensorList, to allow for ExternalSource to pass data without copy. Signed-off-by: Krzysztof Lecki <klecki@nvidia.com>
Why we need this PR?
Pick one, remove the rest
What happened in this PR?
Fill relevant points, put NA otherwise. Replace anything inside []
Adds a special case for device_id that skips all CUDA related calls
Executor
A couple of operators
Some fixes are hacky, check if it can be improved
Added a test case for CPU only scenario
Updated pipeline description
JIRA TASK: [DALI-1491]