Why normalize target data ?

[Amforever]

As in the model.py file like: "/project/03-asvspoof-mega/lfcc-lcnn-lstmsum-p2s/01/model.py ", I don't understand why use def normalize_target(self, y): in line #272, I thought the target data was the label just 0 or 1.
Alos don't understand the difference of "# case 2, loss is defined independent of pt_model" and "# case 1, pt_model.loss is available" in line #112 and line #120 of nn_manager.py file.

[TonyWangX]

Ah, sorry for the confusion.

My poor documentation ...

---

On 1st question

I don't understand why use def normalize_target(self, y): in line #272, I thought the target data was the label just 0 or 1.
The model doesn't use that function to normalize target label.

The model.py is created based on a template, that's why there are many functions unnecessary for this model.

This normalize_target function is used only when the model loads target data from hard disk I/O. This ASVspoof model does NOT use it, and the loaded target is actually [] for this model. This is configured in

project-NN-Pytorch-scripts/project/03-asvspoof-mega/01_config.py

Line 61 in 10fe9fd

output_dirs = []

The target label (1/0) is inferred from the file names, and it doesn't go through the normalize_target function

project-NN-Pytorch-scripts/project/03-asvspoof-mega/lfcc-lcnn-attention-p2s/01/model.py

Line 392 in 10fe9fd

target = self._get_target(filenames)

Apologize for the missing documentation.

---

On 2nd question

"# case 2, loss is defined independent of pt_model" and "# case 1, pt_model.loss is available" in line #112 and line #120 of nn_manager.py file.

• case2: Loss() is defined independent from the model class, and it will receive output from model.forward() and target data from data I/O. It them computes the loss. This is usually the case for common models for regression tasks. Their model.forward() computes \hat_y, and Loss() computes L(\hat_y, y). This target y is loaded from data I/O and has to be normalized beforehand. The mean can be 0 and std can be 1, which is equivalent to NOT normalizing the target data.

• case1: loss() is defined as a method of the model class. This is convenient for other many models. This is inflexible for many advanced models where the loss() requires additional information from the model(). In this case, model.loss() is more flexible.

I like case2 because the Loss() can be separated from model(). This is clean for many other models (like this one https://github.com/nii-yamagishilab/project-NN-Pytorch-scripts/blob/master/project/01-nsf/hn-nsf/model.py)

For the ASVspoof model, I used case 2.

1. the loss is defined as Loss()
2. model.forward() return [a_softmax_act, target_vec, True] to nn_manager.py, and nn_manager gives it to the Loss () and calls Loss([a_softmax_act, target_vec, True], [])
3. Inside Loss()
   outputs[0] <-a_softmax_act,
   outputs[1] <- target_vec
   It then compute the loss by self.m_loss(outputs[0], outputs[1])

[TonyWangX]

03-asvspoof-mega was directly copied from the experiment sandbox on my workstation. It was gradually implemented from scratch. Thus, it is really in a mess.

If you have suggestions to make it easy to understand, your contribution is welcome!

[sn222333]

Hello,I met some problem.
Traceback (most recent call last):
File "/home/zwj/projectNNPytorchscriptsmaster/project/05-nn-vocoders/ilpcnet/main.py", line 210, in
main()
File "/home/zwj/projectNNPytorchscriptsmaster/project/05-nn-vocoders/ilpcnet/main.py", line 139, in main
nii_nn_wrapper.f_train_wrapper(args, model,
File "/home/zwj/projectNNPytorchscriptsmaster/project/05-nn-vocoders/ilpcnet/core_scripts/nn_manager/nn_manager.py", line 377, in f_train_wrapper
f_run_one_epoch(args, pt_model, loss_wrapper, device,
File "/home/zwj/projectNNPytorchscriptsmaster/project/05-nn-vocoders/ilpcnet/core_scripts/nn_manager/nn_manager.py", line 109, in f_run_one_epoch
data_gen = pt_model(data_in)
File "/root/anaconda3/envs/lpc/lib/python3.8/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, **kwargs)
TypeError: forward() missing 1 required positional argument: 'target'
Can you help me?Thank you.

[TonyWangX]

Hello @sn222333
Sorry for the trouble.

That error means that the model.forward(input, target) function in model.py only received input but not target.
The "target" contains features derived from waveform, e.g., lpc coef, gain, ...

What is the python command line when training the model?
The command line should include --model-forward-with-target.
This tells the script to feed "target" to model.forward().

In 05-nn-vocoders/ilpcnet/00_demo.sh, it is

python main.py --num-workers 5 --no-best 100 --epochs 400 --batch-size 256 \
          --lr 0.0001 --model-forward-with-target --temp-flag ${STAGEFLAG} \
          --not-save-each-epoch --trained-model trained_network_s1.pt\
          --cudnn-deterministic-toggle --cudnn-benchmark-toggle

[sn222333]

你好@sn222333 很抱歉给您带来麻烦。

该错误意味着**model.forward(input, target)**函数model.py只接收输入而不是目标。 “目标”包含源自波形的特征，例如 lpc coef、增益、...

训练模型时的python命令行是什么？ 命令行应包括**--model-forward-with-target**。 这告诉脚本将“目标”提供给 model.forward()。

在 05-nn-vocoders/ilpcnet/00_demo.sh 中，是

python main.py --num-workers 5 --no-best 100 --epochs 400 --batch-size 256 \
          --lr 0.0001 --model-forward-with-target --temp-flag ${STAGEFLAG} \
          --not-save-each-epoch --trained-model trained_network_s1.pt\
          --cudnn-deterministic-toggle --cudnn-benchmark-toggle

Thank you