How can I find Kaldi directory?

[Kik099]

In the section "Instalation" it is asked to put the Kaldi directory. How do I is location?

[freewym]

You can simply cd espresso/tools, and then run make. It will automatically download kaldi in the the current directory and compile; or if you have kaldi in another location, make a symbolic link to that location in espresso/tools

[Kik099]

Thanks for the answer. I already did that, but now this appears:

[freewym]

Are you trying to run a recipe in espresso? if yes, cd into a recipe directory e.g., cd examples/asr_wsj, and then ./run.sh <args>

[Kik099]

I was just trying to run the train.py from the first folder

[Kik099]

I think I already know why it is. When I did the make, this error appeared. And I had already done the pip install of the packages

[freewym]

These are Kaldi installation issues. Some dependencies are missing. You can install most of them by sudo apt-get install <package name>. As to MKL, you can find the installation script in https://github.com/kaldi-asr/kaldi/blob/master/tools/extras/install_mkl.sh

[Kik099]

you are right. I t was missing the sox. I already installed the sox and know this appears when i do the make

[Kik099]

I think the error above is resolved. Know says that sph2pipe program doesn't exist when I run:
!cd espresso/examples/asr_wsj; ./run.sh

Error:
Could not find (or execute) the sph2pipe program at /root/../content/kaldi/tools/tools/sph2pipe_v2.5/sph2pipe

[freewym]

Please follow steps in kaldi/tools/INSTALL

[Kik099]

I followed the steps and this appeared:
./run.sh: line 45: ./utils/parse_options.sh: No such file or directory

[freewym]

espresso/tools/kaldi should be present there. If not, please follow the last step of the installation part of README in Espresso:
cd espresso/tools; make KALDI=<path/to/a/compiled/kaldi/directory>

[Kik099]

I did that and know it appears this error:
Stage 0: Data Preparation
ln: failed to create symbolic link 'links/??-?.?': File exists
ln: failed to create symbolic link 'links/??-??.?': File exists
wsj_data_prep.sh: Spot check of command line arguments failed
Command line arguments must be absolute pathnames to WSJ directories
with names like 11-13.1.
Note: if you have old-style WSJ distribution,
local/cstr_wsj_data_prep.sh may work instead, see run.sh for example.

Like you said in another question I deleted the folders and rerun the script. But the same error shows up again

[freewym]

did you specify wsj0 and wsj1 in run.sh correctly?

[Kik099]

No, can you explain me what I need to do?

[freewym]

You need to specify the paths to WSJ corpus for data preparation. WSJ is not a publicly available dataset so if you don't have access to it, I would suggest another recipe LIbriSpeech (https://github.com/freewym/espresso/blob/master/examples/asr_librispeech/run_torchaudio.sh). It is much larger than WSJ though (960 hours vs 80 hours).

[Kik099]

i didn't know that. I am not using that dataset because it is not downloading the dataset. It appears this error:
Stage 0: Data Downloading
Usage: local/download_and_untar.sh [--remove-archive]
e.g.: local/download_and_untar.sh /export/a15/vpanayotov/data www.openslr.org/resources/11 dev-clean
With --remove-archive it will remove the archive after successfully un-tarring it.
can be one of: dev-clean, test-clean, dev-other, test-other,
train-clean-100, train-clean-360, train-other-500.
local/download_and_untar.sh: no such directory www.openslr.org/resources/12

[freewym]

you need to specify data in run.sh to store the downloaded files

[Kik099]

thanks mate I am almost there. In the stage 2 appears me this error:

Stage 2: Feature Generation
steps/make_fbank_pitch.sh --cmd queue.pl --mem 10G --nj 32 --write_utt2num_frames true data/dev_clean exp/make_fbank/dev_clean fbank
utils/validate_data_dir.sh: Successfully validated data-directory data/dev_clean
steps/make_fbank_pitch.sh: [info]: no segments file exists: assuming wav.scp indexed by utterance.
queue.pl: Error submitting jobs to queue (return status was 32512)
queue log file is exp/make_fbank/dev_clean/q/make_fbank_pitch_dev_clean.log, command was qsub -v PATH -cwd -S /bin/bash -j y -l arch=64 -o exp/make_fbank/dev_clean/q/make_fbank_pitch_dev_clean.log -l mem_free=10G,ram_free=10G -t 1:32 /content/espresso/examples/asr_librispeech/exp/make_fbank/dev_clean/q/make_fbank_pitch_dev_clean.sh >>exp/make_fbank/dev_clean/q/make_fbank_pitch_dev_clean.log 2>&1
Output of qsub was: sh: 1: qsub: not found

[freewym]

please replace all the appearances of "queue.pl" in examples/asr_librispeech/cmd.sh with "run.pl". Sorry This recipe assumes people are already familiar with Kaldi's setup. You can use run_torchaudio.sh instead and set download=true inside that. run_torchaudio.sh does not depend on Kaldi.

[Kik099]

I just need her in one more thing. In the run.sh I can choose which files I want to download. It is possible to do the same in run_torchaudio.sh?

[freewym]

You can. Just modify the SPLIT list at https://github.com/freewym/espresso/blob/master/examples/asr_librispeech/local/prepare_librispeech.py#L30

[Kik099]

Sorry mate. In the line 121 of your run_torchaudio.sh it is missing this ".txt".

Your line is this: cut -f 2- -d" " $data_dir/${train_set}/text> $data_dir/lang/input
And your line should be this: cut -f 2- -d" " $data_dir/${train_set}/text.txt > $data_dir/lang/input

[freewym]

you are right. I pushed a fix.

[Kik099]

I just have a question. The model used in the dataset "asr_librispeech" is the Subword LM model, am I right?

[freewym]

Yes, it's sentencepiece LM

[Kik099]

'thanks, I am new in this area. Can you explain me what is happening in each stage of the "run_torcahudio.py"? I just want to know the basics and if possible could you tell me which model/Decoder/Encoder is running in each stage ?

[freewym]

At beginning of each stage there is an echo command printing out a description of that stage.

[Kik099]

sorry but in the stage 5 appears this warning:

/usr/local/lib/python3.7/dist-packages/torch/utils/data/dataloader.py:477: UserWarning: This DataLoader will create 8 worker processes in total. Our suggested max number of worker in current system is 2, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
cpuset_checked))
[2021-05-05 16:16:38,463][fairseq.trainer][INFO] - begin training epoch 1

[freewym]

You can change the the value of the argument dataset.num_workers in the config file https://github.com/freewym/espresso/blob/master/examples/asr_librispeech/config/lstm_librispeech.yaml#L27

[Kik099]

Sorry but I want to use a dataset that I created. For that what I need to have in the folder of the dataset?

I just want to use dev_clean from librispeech. I don't need test_clean and the others ones

[freewym]

For each dataset ,you only need to prepare wav.txt, wav2num_frames and text.txt, and then wrap them into a json file as shown in stage 2 of run_torchaudio.sh

[Kik099]

thanks, I just have another question. When I was running the run_torchaudio with all datasets apart from train_360 and train 500 one error appeared. This is in the stage 5:

[2021-05-22 17:08:15,184][fairseq_cli.train][INFO] - task: LanguageModelingForASRTask
[2021-05-22 17:08:15,184][fairseq_cli.train][INFO] - model: LSTMLanguageModelEspresso
[2021-05-22 17:08:15,184][fairseq_cli.train][INFO] - criterion: CrossEntropyCriterion
[2021-05-22 17:08:15,185][fairseq_cli.train][INFO] - num. shared model params: 24,508,000 (num. trained: 24,508,000)
[2021-05-22 17:08:15,185][fairseq_cli.train][INFO] - num. expert model params: 0 (num. trained: 0)
[2021-05-22 17:08:15,189][fairseq.data.data_utils][INFO] - loaded 5,567 examples from: /home/kkiko56/espresso/examples/asr_librispeech/data/lm_text/valid
Traceback (most recent call last):
File "/opt/conda/lib/python3.7/site-packages/hydra/_internal/utils.py", line 198, in run_and_report
return func()
File "/opt/conda/lib/python3.7/site-packages/hydra/_internal/utils.py", line 350, in
overrides=args.overrides,
File "/opt/conda/lib/python3.7/site-packages/hydra/_internal/hydra.py", line 112, in run
configure_logging=with_log_configuration,
File "/opt/conda/lib/python3.7/site-packages/hydra/core/utils.py", line 127, in run_job
ret.return_value = task_function(task_cfg)
File "/home/kkiko56/espresso/fairseq_cli/hydra_train.py", line 45, in hydra_main
distributed_utils.call_main(cfg, pre_main)
File "/home/kkiko56/espresso/fairseq/distributed/utils.py", line 369, in call_main
main(cfg, **kwargs)
File "/home/kkiko56/espresso/fairseq_cli/train.py", line 121, in main
task.load_dataset(valid_sub_split, combine=False, epoch=1)
File "/home/kkiko56/espresso/fairseq/tasks/language_modeling.py", line 237, in load_dataset
plasma_path=self.args.plasma_path,
File "/home/kkiko56/espresso/fairseq/data/token_block_dataset.py", line 61, in init
sizes, break_mode, document_sep_len, block_size
File "/home/kkiko56/espresso/fairseq/data/token_block_dataset.py", line 87, in _build_slice_indices
from fairseq.data.token_block_utils_fast import (
File "fairseq/data/token_block_utils_fast.pyx", line 1, in init fairseq.data.token_block_utils_fast
ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 88 from C header, got 80 from PyObject
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/conda/bin/fairseq-hydra-train", line 33, in
sys.exit(load_entry_point('fairseq', 'console_scripts', 'fairseq-hydra-train')())
File "/home/kkiko56/espresso/fairseq_cli/hydra_train.py", line 91, in cli_main
hydra_main()
File "/opt/conda/lib/python3.7/site-packages/hydra/main.py", line 37, in decorated_main
strict=strict,

[freewym]

Maybe the prepared data is messed up. Prepare the correct data by running stage 4 may help

[Kik099]

when i run the run_torchaudio the results we get are the table of were from librispeech on the article, am i correct?

[freewym]

The results in the paper is obtained from run.sh. The major difference from run_torchaudio.sh is that in run.sh it also extracta pitch features so the input dim is 83. It didn't affect the finalvresults much though. So I would say they are very close (the difference is ~0.1 in my experiment)

[Kik099]

Does the WER appears in the different systems?
For example in the article it appears the eos threshold, the improvements coverage, the lm subsets, etc...

[freewym]

The improved coverage has been removed. The others remain in the current recipes.