Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update CommonVoice transformer recipes (code from Samsung AI Center Cambridge) #2465

Merged
merged 55 commits into from
Apr 18, 2024

Conversation

TParcollet
Copy link
Collaborator

@TParcollet TParcollet commented Mar 19, 2024

Much like our Transducer recipes (problem addressed in #2433) our transformer recipes on CommonVoice are old AF. I updated everything here. I also removed the per language yaml as only the output vocabulary and number of epochs must be changed.

Summary of changes:

  • Switch from Transformer to Conformer
  • Adam to AdamW
  • Add dynamic batching
  • Change in scheduler
  • Removal of the two-stage training

We will need someone to retrain the models ..... at least english / french / something else?

To do

  • Test a few things around label smoothing to see if it's necessary.
  • Finish training of the full English CV set.

@TParcollet TParcollet added refactor Edit code without changing functionality work in progress Not ready for merge labels Mar 19, 2024
Titouan Parcollet/Embedded AI /SRUK/Engineer/Samsung Electronics added 3 commits March 20, 2024 13:06
@TParcollet TParcollet added ready to review Waiting on reviewer to provide feedback and removed work in progress Not ready for merge labels Mar 20, 2024
@TParcollet TParcollet changed the title Update CommonVoice transformer recipes Update CommonVoice transformer recipes (code from Samsung AI Center Cambridge) Apr 12, 2024
@Adel-Moumen
Copy link
Collaborator

Hi,

LGTM. I uploaded the models on the dropbox. There will be shortly a follow up with the EN based model. All the recipe tests passed.

Test

python -c 'from tests.utils.recipe_tests import run_recipe_tests; print("TEST FAILED!") if not(run_recipe_tests(filters_fields=["Hparam_file"], filters=[["recipes/CommonVoice/ASR/transformer/hparams/conformer_large.yaml"]], do_checks=False, run_opts="--device=cuda")) else print("TEST PASSED")'
(1/1) Running test for CommonVoice_row_18...
        ... 151.56s
TEST PASSED

@Adel-Moumen Adel-Moumen merged commit f7ede69 into speechbrain:develop Apr 18, 2024
4 checks passed
fpaissan added a commit to fpaissan/speechbrain that referenced this pull request May 2, 2024
* Skip lazy imports when the caller is inspect.py

This avoids having certain inspect functions import our lazy modules when we don't want them to. `getframeinfo` in particular appears to do it, and this gets called by PyTorch at some point. IPython might also be doing it but autocomplete still seems to work.

This does not appear to break anything. Added test for hyperpyyaml to ensure we're not breaking that.

* SSL_Semantic_Token _ new PR (speechbrain#2509)

* remove unnecassry  files and move to dasb

* remove extra recepie from test

* update ljspeech qunatization recepie

* add discrete_ssl and remove extra files

* fix precommit

* update kmeans and add tokeizer for postprocessing

* fix precommit

* Update discrete_ssl.py

* fix clone warning

---------

Co-authored-by: Mirco Ravanelli <mirco.ravanelli@gmail.com>

* _ensure_module Raises docstring

* Expose `ensure_module` so that docs get generated for it

This is already an internal class anyway, and this is safe to call.

* Update actions/setup-python

* Use `uv` in test CI + merge some dep installs

The consequence is faster dependency installation. Merging some of the dependency installs helps avoid some packages being reinstalled from one line to the next. Additionally, CPU versions are specified when relevant, to avoid downloading CUDA stuff the CI can't use anyway.

* Use `uv` in doc CI + merge some dep installs

Similar rationale as for the test CI

* Parallelize doc generation with Sphinx

This does not affect the entire doc generation process but should allow some minor multithreading even with the 2-core CI workers.

* Enable `uv` caching on the test CI

* Enable `uv` caching on the docs CI

* CTC-only training recipes for LibriSpeech (code from Samsung AI Cambridge) (speechbrain#2290)

CTC-only pre-training of conformer and branchformer.

---------

Co-authored-by: Shucong Zhang/Embedded AI /SRUK/Engineer/Samsung Electronics <s1.zhang@sruk-ccn4.eu.corp.samsungelectronics.net>
Co-authored-by: Adel Moumen <adelmoumen.pro@gmail.com>
Co-authored-by: Adel Moumen <88119391+Adel-Moumen@users.noreply.github.com>
Co-authored-by: Parcollet Titouan <titouan.parcollet@univ-avignon.fr>

* Update CommonVoice transformer recipes (code from Samsung AI Center Cambridge) (speechbrain#2465)

* Update CV transformer recipes to match latest results with conformer.

---------

Co-authored-by: Titouan Parcollet/Embedded AI /SRUK/Engineer/Samsung Electronics <t.parcollet@sruk-ccn4.eu.corp.samsungelectronics.net>
Co-authored-by: Mirco Ravanelli <mirco.ravanelli@gmail.com>
Co-authored-by: Adel Moumen <adelmoumen.pro@gmail.com>

* Whisper improvements: flash attention, KV caching, lang_id, translation, training... (speechbrain#2450)

Whisper improvements:
- flash attention
- kv caching
- lang identifaction
- translation
- finetuning amelioration 
... and more ...

* Update README.md

* precommit

* update zed download link (speechbrain#2514)

* `RelPosEncXL` refactor and precision fixes (speechbrain#2498)

* Add `RelPosEncXL.make_pe`, rework precision handling

* Rework RelPosEncXL output dtype selection

* Fix in-place input normalization when using `sentence`/`speaker` norm (speechbrain#2504)

* fix LOCAL_RANK to be RANK in if_main_process (speechbrain#2506)

* Fix Separation and Enhancement recipes behavior when NaN encountered (speechbrain#2524)

* Fix Separation and Enhancement recipes behavior when NaN encountered

* Formatting using precommit hooks

* Lock torch version in requirements.txt (speechbrain#2528)

* Fix compatibility for torchaudio versions without `.io` (speechbrain#2532)

This avoids having the Python interpreter attempt to resolve the type annotation directly.

* fix docstrings

* consistency tests - classification

* consistency tests - classification

* consistency tests - interpret

* default to no wham

* fix after tests pass

* fix after tests pass

* tests after that

* fix consistency

---------

Co-authored-by: asu <sdelang@sdelang.fr>
Co-authored-by: Pooneh Mousavi <moosavi.pooneh@gmail.com>
Co-authored-by: Mirco Ravanelli <mirco.ravanelli@gmail.com>
Co-authored-by: shucongzhang <104781888+shucongzhang@users.noreply.github.com>
Co-authored-by: Shucong Zhang/Embedded AI /SRUK/Engineer/Samsung Electronics <s1.zhang@sruk-ccn4.eu.corp.samsungelectronics.net>
Co-authored-by: Adel Moumen <adelmoumen.pro@gmail.com>
Co-authored-by: Adel Moumen <88119391+Adel-Moumen@users.noreply.github.com>
Co-authored-by: Parcollet Titouan <titouan.parcollet@univ-avignon.fr>
Co-authored-by: Parcollet Titouan <parcollet.titouan@gmail.com>
Co-authored-by: Titouan Parcollet/Embedded AI /SRUK/Engineer/Samsung Electronics <t.parcollet@sruk-ccn4.eu.corp.samsungelectronics.net>
Co-authored-by: Yingzhi WANG <41187612+BenoitWang@users.noreply.github.com>
Co-authored-by: Peter Plantinga <plantinga.peter@protonmail.com>
Co-authored-by: Séverin <123748182+SevKod@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready to review Waiting on reviewer to provide feedback refactor Edit code without changing functionality
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants