Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Loader.__len__ is not a pure function #163

Closed
dpaleka opened this issue Feb 17, 2022 · 2 comments
Closed

Loader.__len__ is not a pure function #163

dpaleka opened this issue Feb 17, 2022 · 2 comments
Assignees
Labels
Milestone

Comments

@dpaleka
Copy link

dpaleka commented Feb 17, 2022

Counterintuitively, calling len() on a ffcv.loader.Loader instance modifies its internal state:

ffcv/ffcv/loader/loader.py

Lines 253 to 258 in bfd9b3d

def __len__(self):
next_order = self.next_traversal_order()
if self.drop_last:
return len(next_order) // self.batch_size
else:
return int(np.ceil(len(next_order) / self.batch_size))

This causes the bug in #140 when paired with OrderOption.QUASI_RANDOM.
I suppose the culprit is this line:

random.shuffle(page_to_samples_array[page_ix, :page_size])

@GuillaumeLeclerc
Copy link
Collaborator

Thanks @dpaleka It should be an easy fix :)

GuillaumeLeclerc added a commit that referenced this issue Feb 25, 2022
@GuillaumeLeclerc
Copy link
Collaborator

Thanks for the report, it will be in the next release

netw0rkf10w pushed a commit to netw0rkf10w/ffcv that referenced this issue Nov 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants