This repository has been archived by the owner on Mar 8, 2023. It is now read-only.
Lablita importer - improvement #109
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hi,
following discussion on #90 and reference on #107
This could be the implementation for LABLITA Corpus Importer.
It has been tested on Windows, python 3.6.5
Test1: (param filter_italian_enabled=False)
Execution time: ~5,14h
Total Clips imported: 34288 mp3 (547M)
Single Clip Duration: smoothly between less than 1 second to 53 seconds.
Total duration: 23h.21m.30sec
Test2:
Test with the parameter filter_italian_enabled=True has not been performed
Transcripts are saved in a csv file.
Future Note:
This importer could be first of a series of scripts for the aggregation of various italian corpora audio+transcription produced in the academic research and free available.
The characterizing aspect of the new aggregate dataset is the orientation towards the training in ASR models.
To achieve this we have to implement an audio segmentation of long files into small clips with their own transcription #107
It is useful in ASR training since the small audio segments take much lesser total time as to using full audio at one.