Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

len of iterator incorrect for dynamic batching #171

Closed
LMescheder opened this issue Nov 9, 2017 · 0 comments · Fixed by #192
Closed

len of iterator incorrect for dynamic batching #171

LMescheder opened this issue Nov 9, 2017 · 0 comments · Fixed by #192
Labels

Comments

@LMescheder
Copy link

The __len__ method of Iterator (defined here) returns a wrong result for dynamic batching (i.e. if batch_size_fn is not None).

For example if we set batch_size_fn to

lambda x, n, b: b + len(x.text)

there might be more or fewer batches in the dataset than the __len__ outputs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants