refactor: shared get_dataloader between train and evaluate#19
refactor: shared get_dataloader between train and evaluate#19ziv-lazarov-nagish merged 3 commits intonagishfrom
Conversation
…valuate - Move get_dataloader from train.py to datasets/common.py with explicit args parameter (no global state dependency) - evaluate.py uses shared get_dataloader instead of manual build_datasets + DataLoader - Fix duplicate manifest in train.py (train + val collected same data)
AmitMY
left a comment
There was a problem hiding this comment.
actually, shouldn't some of it go into main?
it will, i was planning to cherry-pick the commits from |
AmitMY
left a comment
There was a problem hiding this comment.
my approach would have been to update main, and then nagish
but you can also update nagish then ask claude to make a PR to main selecting only the non-spepcif nagish changes
that was my intention since i thought most of our efforts would be in the |
Summary
get_dataloaderfromtrain.pytodatasets/common.pywith explicitargsparameterevaluate.pyuses sharedget_dataloaderinstead of manualbuild_datasets+DataLoadernagishbranchChanged files
datasets/common.py— addget_dataloader()with**augment_overridessupporttrain.py— remove localget_dataloader, import from commonevaluate.py— useget_dataloaderinstead of manual constructiondatasets/__init__.py— exportget_dataloader.github/workflows/test.yaml/lint.yaml— addnagishto PR branch triggersTest plan
ruff check .passespytestpasses (61 tests)