Skip to content
This repository has been archived by the owner on Dec 16, 2022. It is now read-only.

switch for DataIterators whether smaller batches should be skipped #3140

Merged
merged 5 commits into from
Aug 15, 2019

Conversation

mnschmit
Copy link
Contributor

fixes #3118

@mnschmit mnschmit changed the title switch for BucketIterator whether smaller batches should be skipped switch for DataIterators whether smaller batches should be skipped Aug 12, 2019
Copy link
Contributor

@DeNeutoy DeNeutoy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great thanks!

Please could you add a test for your changes here:

https://github.com/allenai/allennlp/blob/master/allennlp/tests/data/iterators/bucket_iterator_test.py

A test that demonstrates that the iterator does not pick up batches < the batch size would be fine.

@mnschmit
Copy link
Contributor Author

Yes, of course, I added them. Thanks for the hint!

@DeNeutoy
Copy link
Contributor

Great, thanks a lot!

@DeNeutoy DeNeutoy merged commit 770791a into allenai:master Aug 15, 2019
reiyw pushed a commit to reiyw/allennlp that referenced this pull request Nov 12, 2019
…llenai#3140)

* added a switch to bucket_iterator whether batches smaller than batch_size should be skipped

* added documentation for the new switch

* added the switch also to `HomogeneousBatchIterator`

* added unit tests for bucket and homogeneous iterator
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Feature request: Switch for DataIterators whether batches smaller than batch_size should be skipped
2 participants