FIx dataset loading when there are multiple valid subsets #835
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
When we have multiple valid subsets, say `valid', `valid1` and `valid2`, if `combine=True` holds, when loading `valid` subset, it will try to locate and load `valid`, `valid1`, `valid2`... and then combine them into one dataset. Set combine to False solves this issue.
Contributor
|
Thank you for your pull request and welcome to our community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. In order for us to review and merge your code, please sign up at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need the corporate CLA signed. If you have received this in error or have any questions, please contact us at cla@fb.com. Thanks! |
Contributor
|
Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Facebook open source project. Thanks! |
Contributor
facebook-github-bot
left a comment
facebook-github-bot
left a comment
There was a problem hiding this comment.
@myleott is landing this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
|
Good catch, thanks! |
Contributor
yzpang
pushed a commit
to yzpang/gold-off-policy-text-gen-iclr21
that referenced
this pull request
Feb 19, 2021
Summary: When we have multiple valid subsets, say `valid`, `valid1` and `valid2`, if `combine=True` holds, when loading `valid` subset, it will try to locate and load `valid`, `valid1`, `valid2`... and then combine them into one dataset. Set `combine` to `False` solves this issue. In my experiment, I have 3 valid subsets with 3000, 5000 and 8701 examples, with argument `--valid-subset valid,valid1,valid2`, the log is as follows: ``` ...... | ./mix_data/bin valid src-trg 3000 examples | ./mix_data/bin valid1 src-trg 5000 examples | ./mix_data/bin valid2 src-trg 7801 examples | ./mix_data/bin valid1 src-trg 5000 examples | ./mix_data/bin valid2 src-trg 7801 examples ...... ``` As shown above, `valid1` and `valid2` subsets are incorrectly loaded twice. Pull Request resolved: facebookresearch/fairseq#835 Differential Revision: D16006343 Pulled By: myleott fbshipit-source-id: ece7fee3a00f97a6b3409defbf7f7ffaf0a54fdc
Harleen8118
pushed a commit
to Harleen8118/IBERT
that referenced
this pull request
Jun 26, 2025
Summary: When we have multiple valid subsets, say `valid`, `valid1` and `valid2`, if `combine=True` holds, when loading `valid` subset, it will try to locate and load `valid`, `valid1`, `valid2`... and then combine them into one dataset. Set `combine` to `False` solves this issue. In my experiment, I have 3 valid subsets with 3000, 5000 and 8701 examples, with argument `--valid-subset valid,valid1,valid2`, the log is as follows: ``` ...... | ./mix_data/bin valid src-trg 3000 examples | ./mix_data/bin valid1 src-trg 5000 examples | ./mix_data/bin valid2 src-trg 7801 examples | ./mix_data/bin valid1 src-trg 5000 examples | ./mix_data/bin valid2 src-trg 7801 examples ...... ``` As shown above, `valid1` and `valid2` subsets are incorrectly loaded twice. Pull Request resolved: facebookresearch/fairseq#835 Differential Revision: D16006343 Pulled By: myleott fbshipit-source-id: ece7fee3a00f97a6b3409defbf7f7ffaf0a54fdc
caltia
pushed a commit
to caltia/fairseq
that referenced
this pull request
Jul 8, 2025
Summary: When we have multiple valid subsets, say `valid`, `valid1` and `valid2`, if `combine=True` holds, when loading `valid` subset, it will try to locate and load `valid`, `valid1`, `valid2`... and then combine them into one dataset. Set `combine` to `False` solves this issue. In my experiment, I have 3 valid subsets with 3000, 5000 and 8701 examples, with argument `--valid-subset valid,valid1,valid2`, the log is as follows: ``` ...... | ./mix_data/bin valid src-trg 3000 examples | ./mix_data/bin valid1 src-trg 5000 examples | ./mix_data/bin valid2 src-trg 7801 examples | ./mix_data/bin valid1 src-trg 5000 examples | ./mix_data/bin valid2 src-trg 7801 examples ...... ``` As shown above, `valid1` and `valid2` subsets are incorrectly loaded twice. Pull Request resolved: facebookresearch/fairseq#835 Differential Revision: D16006343 Pulled By: myleott fbshipit-source-id: ece7fee3a00f97a6b3409defbf7f7ffaf0a54fdc
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When we have multiple valid subsets, say
valid,valid1andvalid2, ifcombine=Trueholds, when loadingvalidsubset, it will try to locate and loadvalid,valid1,valid2... and then combine them into one dataset. SetcombinetoFalsesolves this issue.In my experiment, I have 3 valid subsets with 3000, 5000 and 8701 examples, with argument
--valid-subset valid,valid1,valid2, the log is as follows:As shown above,
valid1andvalid2subsets are incorrectly loaded twice.