Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
Add DataLoader class: a way to lazy-load iterable datasets. #47
Using generators lazily and for instance overloading the iter and len methods. To be used with Streaming Pipelines.
Question: could they be without length / infinite?
I'd also like to think about how we could have things that enable to duplicate the data (e.g.: introduce local shuffle (with window size) or the concept of epoch loops to train NNs.