Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Changed the keep_zip default to False to comply with StreamingDataset #150

Merged
merged 2 commits into from
May 17, 2023
Merged

Changed the keep_zip default to False to comply with StreamingDataset #150

merged 2 commits into from
May 17, 2023

Conversation

karan6181
Copy link
Contributor

Description

  • Changed the keep_zip default to False to comply with StreamingDataset.

@samhavens
Copy link
Contributor

This is what Streaming requires, so I'm a little surprised that this hasn't caused problems already.

@karan6181
Copy link
Contributor Author

This is what Streaming requires, so I'm a little surprised that this hasn't caused problems already.

The finetuning script (class StreamingFinetuningDataset) was having an issue, not the end-to-end training (class StreamingTextDataset)! So that's the reason why this is working for certain use-case.

@karan6181 karan6181 merged commit afeb7a6 into mosaicml:main May 17, 2023
6 checks passed
@karan6181 karan6181 deleted the fix_keep_zip branch May 17, 2023 00:26
bmosaicml pushed a commit that referenced this pull request Jun 6, 2023
@abhi-mosaic
Fix YAML referencing to allow better CLI overrides (#150)
5e5e643
bmosaicml pushed a commit that referenced this pull request Jun 6, 2023
@karan6181
Changed the keep_zip default to False to comply with StreamingDataset (…
054ad57 
…#150)

* Changed the keep_zip default to False to comply with StreamingDataset

* keep_zip only accept boolean in StreamingDataset
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@samhavens samhavens samhavens approved these changes

@abhi-mosaic abhi-mosaic Awaiting requested review from abhi-mosaic

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@karan6181 @samhavens