Add custom dataset and dataloader tutorial for C++ #841
Conversation
✅ Deploy Preview for pytorch-tutorials-preview ready!
To edit notification comments on pull requests, go to your Netlify site settings. |
|
@glaringlee - can you review this tutorial and suggest any updates? |
There was a problem hiding this comment.
@dhpollack Thanks for contributing this tutorial. This generally looks good to me. I slightly commented it.
This tutorial have 2 major parts, one is how to create a customized dataset/dataloader workflow, the other one is how to build it with libtorch. Can you put two reference link into this tutorial as well? We have a code example for creating dataset/dataloader and we have a documentation for build libtorch with customized code, but we have no tutorial to put all of these together, I think this tutorial is a good place to have all of these references.
code example: https://github.com/pytorch/examples/blob/master/cpp/custom-dataset/custom-dataset.cpp
libtorch build document: https://pytorch.org/cppdocs/installing.html
|
There are some CI tests failures, but it seems not related to this PR. Let's see whether CI tests is recovered after this PR is updated. |
I was struggling with creating a custom dataset and dataloader for the C++ frontend partially because I couldn't find a good minimal example and ran into a few gotchas when trying to model something after the MNIST dataset. This tutorial might help others avoid the pitfalls that I ran into. Signed-off-by: David Pollack <david@da3.net>
dhpollack
force-pushed
the
dhp/cpp_dataset_tutorial
branch
from
06a6126 to
5255c0a
Compare
|
@glaringlee @jspisak I need to test this with the newest version of pytorch as I created this over a year ago and haven't tried it out with the newest versions of libtorch. I am on vacation this week, but I can try it this weekend / Monday when I get back. |
|
@glaringlee I tested this with libtorch 1.8 and gcc 10.2 and it works on my system. Good to go unless you have some other comments. |
|
@dhpollack This is good to me, there is a CI failure, we will merge this after clear the CI failure. |
|
@brianjo |
Yea, librosa is an audio module. Probably related to one of the audio tutorials. Has nothing to do with this PR. |
|
Rebasing this some fixes I added to master. |
|
@guyang3532 - looks like some Windows failures are occuring - can you take a look? |
I think this just needs a rebase. I did that. Should fix the issues. |
|
Sorry, looks like there is a Windows issue on this one.
…________________________________
From: Joseph Spisak ***@***.***>
Sent: Sunday, March 21, 2021 12:06:53 PM
To: pytorch/tutorials ***@***.***>
Cc: Brian Johnson ***@***.***>; Assign ***@***.***>
Subject: Re: [pytorch/tutorials] Add custom dataset and dataloader tutorial for C++ (#841)
@guyang3532<https://github.com/guyang3532> - looks like some Windows failures are occuring - can you take a look?
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub<#841 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AALOH5KBN5BVB2RCNHNALFDTEYKR3ANCNFSM4KOFJDJQ>.
|
|
stale |
|
how to build a custom dataset and dataloader of an object detection based task(that is to say each image has different numbers of anchor. When iterating dataloader, batch cannot easily stack) |
I was struggling with creating a custom dataset and dataloader for the
C++ frontend partially because I couldn't find a good minimal example
and ran into a few gotchas when trying to model something after the
MNIST dataset. This tutorial might help others avoid the pitfalls that
I ran into.
Signed-off-by: David Pollack david@da3.net