Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable using a generator as data loader #1011

Open
BenjaminBossan opened this issue Aug 17, 2023 · 0 comments
Open

Enable using a generator as data loader #1011

BenjaminBossan opened this issue Aug 17, 2023 · 0 comments

Comments

@BenjaminBossan
Copy link
Collaborator

As discussed with @ottonemo

In #835, we made a change that results in the data loader being only initialized once per fit call, not once per iteration (i.e. epoch) as previously. Although we considered backwards compatibility, we missed one use case, namely passing a generator to iterator_train and iterator_valid (or, more precisely, a generator factory).

Now, if a user tries this, the training will run fine for the first epoch but afterwards the generator will be exhausted, resulting in the fit loop running 0 times, which creates strange results.

This issue should be addressed by two follow up tasks:

  1. Re-enable passing a generator (factory) to iterator_train and iterator_valid. This can be achieved by checking the type and, if a types.GeneratorType is encountered, to wrap it into a class that makes it an iterator.
  2. Place a guard inside the fit loop that detects if it is called 0 times. I would imagine this is never desired and should raise an error, or at the very least give a warning. As is, there is no error and no warning, but the results are very strange in the print log.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant