I want to finetune pre-trained citrinet for 8Khz data set. What steps should I follow?

[Enescigdem]

I want to finetune pre-trained citrinet-1024 for my 8Khz data set. What steps should I follow?
I did these steps

1. tokenizer traininig with my train data
   But then
2. asr_model = nemo_asr.models.EncDecCTCModelBPE.from_pretrained(model_name="stt_en_citrinet_1024") does not see my config file and gives error
3. then i created the citrinet model from scratch using config_bpe.yaml file,it gives very high WERs.

What should i do for this task ?

[titu1994]

There are quite a few ASR tutorials that cover fine tuning, could you please upsample all of your 8 kHz data to 16 kHz and train following the steps laid out on the fine tuning tutorial -

ASR finetuning - https://github.com/NVIDIA/NeMo/blob/main/tutorials/asr/10_ASR_CTC_Language_Finetuning.ipynb

Recommendations for 8 kHz data - https://github.com/NVIDIA/NeMo/blob/main/tutorials/asr/ASR_for_telephony_speech.ipynb

[Enescigdem]

I saw these tutorials.What is the exact model name of pre-trained citrinet on English data and how can I initialize it?

first_asr_model = nemo_asr.models.ASRModel.from_pretrained("stt_en_citrinet_1024") gives me
ValueError: Invalid Datatype for loaders: NoneType and parameters are different my config file

[titu1994]

Here is the info about all the models we support and how to instantiate them - https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/main/asr/results.html

[titu1994]

Can you post the full error stack ?

[Enescigdem]

first_asr_model = nemo_asr.models.ASRModel.from_pretrained("stt_en_citrinet_1024"
trainer.fit(first_asr_model)
`
Traceback (most recent call last):
File "script.py", line 29, in
trainer.fit(first_asr_model)
File "path/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 499, in fit
self.dispatch()
File "path/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 546, in dispatch
self.accelerator.start_training(self)
File "path/lib/python3.7/site-packages/pytorch_lightning/accelerators/accelerator.py", line 73, in start_training
self.training_type_plugin.start_training(trainer)
File "path/lib/python3.7/site-packages/pytorch_lightning/plugins/training_type/training_type_plugin.py", line 114, in start_training
self._results = trainer.run_train()
File "/path/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 620, in run_train
self.train_loop.reset_train_val_dataloaders(model)
File "path/lib/python3.7/site-packages/pytorch_lightning/trainer/training_loop.py", line 218, in reset_train_val_dataloaders
self.trainer.reset_train_dataloader(model)
File "path/lib/python3.7/site-packages/pytorch_lightning/trainer/data_loading.py", line 222, in reset_train_dataloader
self.train_dataloader = CombinedLoader(self.train_dataloader, self._multiple_trainloader_mode)
File "path/lib/python3.7/site-packages/pytorch_lightning/trainer/supporters.py", line 365, in init
self._wrap_loaders_max_size_cycle()
File "path/lib/python3.7/site-packages/pytorch_lightning/trainer/supporters.py", line 402, in _wrap_loaders_max_size_cycle
raise ValueError(f'Invalid Datatype for loaders: {type(self.loaders).name}')
ValueError: Invalid Datatype for loaders: NoneType

[titu1994]

The training data loaders have not been instantiated. Please refer to the above notebooks to see all of the necessary steps to prepare the model after instantiation and before calling trainer.fit()

[Enescigdem]

I modified the data paths and tokenizer. Also, the sample rate is modified according to these tutorials. Before loading from the pre-trained checkpoint, this error did not exist. So please tell me what am I missed?

I can not change the configuration of the pretrained model.
It gives this output.

[NeMo W 2021-06-23 16:39:19 modelPT:138] If you intend to do training or fine-tuning, please call the ModelPT.setup_training_data() method and provide a valid configuration file to setup the train data loader.
Train config :
manifest_filepath: null
sample_rate: 16000
batch_size: 32
trim_silence: true
max_duration: 16.7
shuffle: true
is_tarred: false
tarred_audio_filepaths: null
use_start_end_token: false
@titu1994

[titu1994]

Setting the paths is insufficient - you need to call the appropriate methods setup_*_data and change_vocabulary(). The notebook shows these steps, please follow them exactly in your own script and it will work after that.

[Enescigdem]

It works now but I did not use the from_pretrained method I restored the pre-trained model from Nemo model then started to train.