Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Webdataset reader operator implementation #3306

Merged
merged 45 commits into from Sep 16, 2021
Merged

Webdataset reader operator implementation #3306

merged 45 commits into from Sep 16, 2021

Commits on Sep 15, 2021

  1. Implementation of the reader 

    Signed-off-by: Bartłomiej Cieślar <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 15, 2021
    Copy the full SHA
    f786de6 View commit details
    Browse the repository at this point in the history

  2. Implementation skeleton for wds loader 

    Signed-off-by: Bartłomiej Cieślar <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 15, 2021
    Copy the full SHA
    dbc044c View commit details
    Browse the repository at this point in the history

  3. Partial implementation of webdataset loader 

    Signed-off-by: Bartłomiej Cieślar <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 15, 2021
    Copy the full SHA
    6c82500 View commit details
    Browse the repository at this point in the history

  4. Further implementation of the webdataset loader 

    Signed-off-by: Bartłomiej Cieślar <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 15, 2021
    Copy the full SHA
    fb8ca11 View commit details
    Browse the repository at this point in the history

  5. Initial refactoring of the way indexing works 

    Signed-off-by: Bartłomiej Cieślar <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 15, 2021
    Copy the full SHA
    2b888ee View commit details
    Browse the repository at this point in the history

  6. Changed the formatting of the index file 

    Signed-off-by: Bartłomiej Cieślar <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 15, 2021
    Copy the full SHA
    97707ee View commit details
    Browse the repository at this point in the history

  7. removed boost and added sample filtering 

    Signed-off-by: Bartłomiej Cieślar <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 15, 2021
    Copy the full SHA
    453d280 View commit details
    Browse the repository at this point in the history

  8. readsample implementation for wds loader 

    Signed-off-by: Bartłomiej Cieślar <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 15, 2021
    Copy the full SHA
    90cd41e View commit details
    Browse the repository at this point in the history

  9. initial bug fixes 

    Signed-off-by: Bartłomiej Cieślar <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 15, 2021
    Copy the full SHA
    c05395a View commit details
    Browse the repository at this point in the history

  10. Implementation of the reader 

    Signed-off-by: Bartłomiej Cieślar <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 15, 2021
    Copy the full SHA
    4dc239e View commit details
    Browse the repository at this point in the history

  11. Implementation skeleton for wds loader 

    Signed-off-by: Bartłomiej Cieślar <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 15, 2021
    Copy the full SHA
    4b4c4f5 View commit details
    Browse the repository at this point in the history

  12. Partial implementation of webdataset loader 

    Signed-off-by: Bartłomiej Cieślar <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 15, 2021
    Copy the full SHA
    45c3c80 View commit details
    Browse the repository at this point in the history

  13. Further implementation of the webdataset loader 

    Signed-off-by: Bartłomiej Cieślar <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 15, 2021
    Copy the full SHA
    9b91661 View commit details
    Browse the repository at this point in the history

  14. Initial refactoring of the way indexing works 

    Signed-off-by: Bartłomiej Cieślar <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 15, 2021
    Copy the full SHA
    697070f View commit details
    Browse the repository at this point in the history

  15. Changed the formatting of the index file 

    Signed-off-by: Bartłomiej Cieślar <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 15, 2021
    Copy the full SHA
    1c0651c View commit details
    Browse the repository at this point in the history

  16. removed boost and added sample filtering 

    Signed-off-by: Bartłomiej Cieślar <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 15, 2021
    Copy the full SHA
    981d16d View commit details
    Browse the repository at this point in the history

  17. readsample implementation for wds loader 

    Signed-off-by: Bartłomiej Cieślar <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 15, 2021
    Copy the full SHA
    791fbe7 View commit details
    Browse the repository at this point in the history

  18. initial bug fixes 

    Signed-off-by: Bartłomiej Cieślar <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 15, 2021
    Copy the full SHA
    268265a View commit details
    Browse the repository at this point in the history

  19. Further implementation of the tests for Webdataset Reader 

    Signed-off-by: barci2 <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 15, 2021
    Copy the full SHA
    a714690 View commit details
    Browse the repository at this point in the history

  20. Split tests into several files 

    Signed-off-by: barci2 <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 15, 2021
    Copy the full SHA
    2d2836e View commit details
    Browse the repository at this point in the history

  21. bug fixes and further tests 

    Signed-off-by: Bartłomiej Cieślar <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 15, 2021
    Copy the full SHA
    a8a2df8 View commit details
    Browse the repository at this point in the history

  22. almost final tests 

    Signed-off-by: Bartłomiej Cieślar <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 15, 2021
    Copy the full SHA
    b016bd1 View commit details
    Browse the repository at this point in the history

  23. final fixes to the tests 

    Signed-off-by: Bartłomiej Cieślar <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 15, 2021
    Copy the full SHA
    30b0749 View commit details
    Browse the repository at this point in the history

  24. linter fixes 

    Signed-off-by: Bartłomiej Cieślar <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 15, 2021
    Copy the full SHA
    38731b4 View commit details
    Browse the repository at this point in the history

  25. bug fixes 

    Signed-off-by: Bartłomiej Cieślar <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 15, 2021
    Copy the full SHA
    fad0f83 View commit details
    Browse the repository at this point in the history

  26. removed debug messages 

    Signed-off-by: Bartłomiej Cieślar <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 15, 2021
    Copy the full SHA
    48255be View commit details
    Browse the repository at this point in the history

  27. fixed lgtm.com errors 

    Signed-off-by: Bartłomiej Cieślar <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 15, 2021
    Copy the full SHA
    f8b3997 View commit details
    Browse the repository at this point in the history

  28. upgraded the cross-check test for wds reader 

    Signed-off-by: Bartłomiej Cieślar <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 15, 2021
    Copy the full SHA
    358eec9 View commit details
    Browse the repository at this point in the history

  29. formatting fix 

    Signed-off-by: Bartłomiej Cieślar <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 15, 2021
    Copy the full SHA
    cb57098 View commit details
    Browse the repository at this point in the history

  30. slightly improved documentation for the wds2idx.py tool 

    Signed-off-by: Bartłomiej Cieślar <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 15, 2021
    Copy the full SHA
    8e00216 View commit details
    Browse the repository at this point in the history

  31. refactors advised in PR 

    Signed-off-by: Bartłomiej Cieślar <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 15, 2021
    Copy the full SHA
    9f5d8b9 View commit details
    Browse the repository at this point in the history

  32. docstrings for the WDS reader 

    Signed-off-by: Bartłomiej Cieślar <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 15, 2021
    Copy the full SHA
    6dcaffd View commit details
    Browse the repository at this point in the history

  33. error throwing tests 

    Signed-off-by: Bartłomiej Cieślar <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 15, 2021
    Copy the full SHA
    28ca3be View commit details
    Browse the repository at this point in the history

  34. next batch of reviews ready 

    Signed-off-by: Bartłomiej Cieślar <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 15, 2021
    Copy the full SHA
    48edd03 View commit details
    Browse the repository at this point in the history

  35. rework of WDS reader to parse component to output matching in Prepare… 

    …Metadata

Signed-off-by: Bartłomiej Cieślar <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 15, 2021
    Copy the full SHA
    e8c6b65 View commit details
    Browse the repository at this point in the history

  36. Review suggestions except for dostrings 

    Signed-off-by: Bartłomiej Cieślar <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 15, 2021
    Copy the full SHA
    25c0359 View commit details
    Browse the repository at this point in the history

  37. Next review suggestions 

    Signed-off-by: Bartłomiej Cieślar <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 15, 2021
    Copy the full SHA
    26c0e93 View commit details
    Browse the repository at this point in the history

  38. Next review suggestions 

    Signed-off-by: Bartłomiej Cieślar <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 15, 2021
    Copy the full SHA
    36c4982 View commit details
    Browse the repository at this point in the history

  39. Hopefully final round of review changes 

    Signed-off-by: Bartłomiej Cieślar <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 15, 2021
    Copy the full SHA
    5a5643a View commit details
    Browse the repository at this point in the history

  40. Michał Zientkiewicz's suggestions to dosctrings 

    Signed-off-by: Bartłomiej Cieślar <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 15, 2021
    Copy the full SHA
    d71948d View commit details
    Browse the repository at this point in the history

  41. Final review suggestions 

    Signed-off-by: Bartłomiej Cieślar <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 15, 2021
    Copy the full SHA
    373ab5a View commit details
    Browse the repository at this point in the history

  42. Bug fixes 

    Signed-off-by: Bartłomiej Cieślar <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 15, 2021
    Copy the full SHA
    bd3116c View commit details
    Browse the repository at this point in the history

  43. Bug fixes 

    Signed-off-by: Bartłomiej Cieślar <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 15, 2021
    Copy the full SHA
    2bdbf12 View commit details
    Browse the repository at this point in the history

  44. Bug fixes 

    Signed-off-by: Bartłomiej Cieślar <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 15, 2021
    Copy the full SHA
    78d5cd1 View commit details
    Browse the repository at this point in the history

Commits on Sep 16, 2021

  1. updated dali extra hash 

    Signed-off-by: Bartłomiej Cieślar <bcieslar2001@gmail.com>
    @barci2
    barci2 committed Sep 16, 2021
    Copy the full SHA
    fccad59 View commit details
    Browse the repository at this point in the history