Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add TSSB-3M dataset #2693

Merged
merged 4 commits into from May 11, 2023
Merged

add TSSB-3M dataset #2693

merged 4 commits into from May 11, 2023

Conversation

zirui
Copy link
Contributor

@zirui zirui commented Apr 18, 2023

add the TSSM-3M code bugs dataset
issue: #1395

sedthh
sedthh requested changes Apr 18, 2023
View reviewed changes
Copy link
Collaborator

@sedthh sedthh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Had a few questions, awesome job nevertheless! This will make a great addition to the instruciton datasets.

data/datasets/TSSB-3M/generate_dataset.py Show resolved Hide resolved
data/datasets/TSSB-3M/README.md Show resolved Hide resolved
data/datasets/TSSB-3M/generate_dataset.py Outdated Show resolved Hide resolved
1. optimize regex for removing issue id in commit mesasges 2. randoml…
763a3f4 
…y generate synonymous instructions 3. filter invalid commit mesage
@github-actions
Copy link

github-actions bot commented May 2, 2023

❌ pre-commit failed.
Please run pre-commit run --all-files locally and commit the changes.
Find more information in the repository's CONTRIBUTING.md

olliestanley
olliestanley requested changes May 2, 2023
View reviewed changes
Copy link
Collaborator

@olliestanley olliestanley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The .tsv data file will need removing before merge

@zirui
Copy link
Contributor Author

zirui commented May 2, 2023

The .tsv data file will need removing before merge
The. tsv data file (generated through statistics and review) is read in the script to filter commit messages

@olliestanley
Copy link
Collaborator

The. tsv data file (generated through statistics and review) is read in the script to filter commit messages

I would advocate having it in a HF repo and downloading it as part of the script. But I guess it is reasonably small so maybe not a big problem

olliestanley
olliestanley approved these changes May 2, 2023
View reviewed changes
Copy link
Collaborator

@olliestanley olliestanley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

0x22almostEvil
0x22almostEvil suggested changes May 8, 2023
View reviewed changes
data/datasets/__init__.py Outdated Show resolved Hide resolved
@sedthh sedthh merged commit ea6af41 into LAION-AI:main May 11, 2023
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@olliestanley olliestanley olliestanley approved these changes

@sedthh sedthh sedthh approved these changes

@0x22almostEvil 0x22almostEvil 0x22almostEvil approved these changes

@Vechtomov Vechtomov Awaiting requested review from Vechtomov Vechtomov is a code owner

@bitplane bitplane Awaiting requested review from bitplane bitplane is a code owner

@huu4ontocord huu4ontocord Awaiting requested review from huu4ontocord

Assignees
No one assigned
Labels
data
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
5 participants
@zirui @olliestanley @sedthh @0x22almostEvil @andreaskoepf