Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add TF Dataset GPU #1354

Merged
merged 11 commits into from
Oct 14, 2019
Merged

Add TF Dataset GPU #1354

merged 11 commits into from
Oct 14, 2019

Conversation

awolant
Copy link
Contributor

@awolant awolant commented Oct 8, 2019 •
edited
Loading

Signed-off-by: Albert Wolant awolant@nvidia.com

Why we need this PR?

  • It adds support for single GPU in TensorFlwo Dataset

What happened in this PR?

  • Explain solution of the problem, new feature added.
    Add GPU support for TF Dataset
  • What is most important part that reviewers should focus on?
    Changes in DALIDatasetOp
  • Was this PR tested? How?
    Added test for GPU and L1 test using other device_id

JIRA TASK: [DALI-730]

@awolant
Add TF Dataset GPU
2898da1 
Signed-off-by: Albert Wolant <awolant@nvidia.com>
@dali-automaton
Copy link
Collaborator

CI MESSAGE: [935571]: BUILD STARTED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [935571]: BUILD FAILED

@awolant
Refactor dataset options
57d191d 
Signed-off-by: Albert Wolant <awolant@nvidia.com>
@dali-automaton
Copy link
Collaborator

CI MESSAGE: [937057]: BUILD STARTED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [937057]: BUILD FAILED

@awolant
Fix for TF 1.13
d8b004f 
Signed-off-by: Albert Wolant <awolant@nvidia.com>
@dali-automaton
Copy link
Collaborator

CI MESSAGE: [937161]: BUILD STARTED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [937161]: BUILD FAILED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [937233]: BUILD STARTED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [937233]: BUILD FAILED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [937233]: BUILD PASSED

@awolant
Copy link
Contributor Author

awolant commented Oct 9, 2019

!build

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [937422]: BUILD STARTED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [937422]: BUILD PASSED

@awolant
Fix and refactor test
cbc29f4 
Signed-off-by: Albert Wolant <awolant@nvidia.com>
@awolant
Add stream support
eaddb53 
Signed-off-by: Albert Wolant <awolant@nvidia.com>
@awolant
Add device selection
1b3433e 
Signed-off-by: Albert Wolant <awolant@nvidia.com>
@awolant
Add tf dataset multigpu L1 test
6921c86 
Signed-off-by: Albert Wolant <awolant@nvidia.com>
@awolant awolant changed the title [WIP] Add TF Dataset GPU Add TF Dataset GPU Oct 11, 2019
@dali-automaton
Copy link
Collaborator

CI MESSAGE: [940877]: BUILD STARTED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [940882]: BUILD STARTED

awolant
awolant commented Oct 11, 2019
View reviewed changes
dali/test/python/test_dali_tf_dataset.py
device = "cpu",
dtype = types.FLOAT16)
device = device,
dtype = types.INT16)
Copy link
Contributor Author

@awolant awolant Oct 11, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't support FLOAT16 in the Cast on the GPU, so I've changed it to INT16 to have the same test both for the CPU and the GPU.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll fix the Cast operator in a separate PR coming soon

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [940877]: BUILD PASSED

@awolant
Fix test path
df0662b 
Signed-off-by: Albert Wolant <awolant@nvidia.com>
@dali-automaton
Copy link
Collaborator

CI MESSAGE: [940882]: BUILD PASSED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [941114]: BUILD STARTED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [941214]: BUILD STARTED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [941114]: BUILD FAILED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [941214]: BUILD PASSED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [941114]: BUILD PASSED

@NVIDIA NVIDIA deleted a comment from dali-automaton Oct 14, 2019
@dali-automaton
Copy link
Collaborator

CI MESSAGE: [943551]: BUILD STARTED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [943551]: BUILD PASSED

@awolant awolant merged commit 8c577e1 into NVIDIA:master Oct 14, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jantonguirao jantonguirao jantonguirao approved these changes

@JanuszL JanuszL JanuszL approved these changes

@klecki klecki Awaiting requested review from klecki

@mzient mzient Awaiting requested review from mzient

@szalpal szalpal Awaiting requested review from szalpal

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@awolant @dali-automaton @jantonguirao @JanuszL