Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add KsponSpeech recipe #1353

Merged
merged 6 commits into from
Jun 12, 2024

Conversation

whsqkaak
Copy link
Contributor

KsponSpeech is a large-scale spontaneous speech corpus of Korean.
This corpus contains 969 hours of open-domain dialog utterances,
spoken by about 2,000 native Korean speakers in a clean environment.

All data were constructed by recording the dialogue of two people
freely conversing on a variety of topics and manually transcribing the utterances.

The transcription provides a dual transcription consisting of orthography and pronunciation,
and disfluency tags for spontaneity of speech, such as filler words, repeated words, and word fragments.

The original audio data has a pcm extension.
During preprocessing, it is converted into a file in the wav extension and saved anew.

KsponSpeech is publicly available on an open data hub site of the Korea government.
The dataset must be downloaded manually.

For more details, please visit:

Dataset: https://aihub.or.kr/aihubdata/data/view.do?currMenu=115&topMenu=100&aihubDataSe=realm&dataSetSn=123
Paper: https://www.mdpi.com/2076-3417/10/19/6936

Copy link
Collaborator

@pzelasko pzelasko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, great work! I left 2 comments.

)


def normalize(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we make normalization optional (and enabled by default) via a parameter in prepare_ksponspeech? you can see other recipes, some of them have a string option making it possible to choose different flavors of normalization (we can have "default" and "none" here)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we make normalization optional (and enabled by default) via a parameter in prepare_ksponspeech? you can see other recipes, some of them have a string option making it possible to choose different flavors of normalization (we can have "default" and "none" here)

Fix it in e7dc868

return manifests


def pcm_to_wav(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we convert to FLAC instead? 2x storage space savings

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we convert to FLAC instead? 2x storage space savings

Fix it in 36aa16d

@whsqkaak
Copy link
Contributor Author

Thanks, great work! I left 2 comments.

Thanks for your feedback! I'll fix the proposal quickly.

@pzelasko pzelasko enabled auto-merge (squash) June 12, 2024 13:49
@pzelasko
Copy link
Collaborator

Thanks! LGTM

@pzelasko pzelasko merged commit f9fb181 into lhotse-speech:master Jun 12, 2024
9 checks passed
@whsqkaak whsqkaak deleted the feature/ksponspeech-recipe branch June 13, 2024 01:23
@pzelasko pzelasko modified the milestone: v1.24.0 Jun 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants