when inference, how to set sigma value?

[hdmjdp]

when sigma=0.6

you can see in the high frenquency, has a line noise. how to remove it?

[syang1993]

I don't think this line is caused by sigma values. When squeezing data, I think it breaks the continuity of audio in time domain. Thus you will find such lines in frequency domain. For example, when using n_group=8 to train 16k audios, these lines will exist in 2k hz, 4k, 6k...
sample.zip

[dhgrs]

Please try sigma = 1 or train more iteration. It helped me.

[azraelkuan]

just, train more steps may make this noise disappear

150k sigma=0.6

208k sigma=0.6

[hdmjdp]

@syang1993 can your share your config json?

[hdmjdp]

@azraelkuan can you share your config json? your sigma=0.6 is during training?

[azraelkuan]

@hdmjdp keep the same with this repo except the wavenet filters, i change it to 128 for big batch size
0.6 is for inference

[syang1993]

@hdmjdp You can just use the default config.

@azraelkuan The lines also esixts in your 208K sample (2k ,4k, 6k).

[hdmjdp]

@syang1993 @azraelkuan
{
"train_config": {
"output_directory": "checkpoints",
"epochs": 100000,
"learning_rate": 1e-4,
"sigma": 1.0,
"iters_per_checkpoint": 2000,
"batch_size": 1,
"seed": 1234,
"checkpoint_path": ""
},
"data_config": {
"wav_dataroot":"/home/hdm/Documents/tts/data/wav/hb_cen_lily_sent-24K",
"mel_dataroot":"/home/hdm/Documents/wavenet/msc",
"segment_length": 18000,
"sampling_rate": 24000,
"filter_length": 2048,
"hop_length": 120,
"win_length": 2048,
"mel_fmin": 0.0,
"mel_fmax": 8000.0
},
"dist_config": {
"dist_backend": "nccl",
"dist_url": "tcp://localhost:54321"
},

"waveglow_config": {
    "n_mel_channels": 80,
    "n_flows": 12,
    "n_group": 8,
    "n_early_every": 4,
    "n_early_size": 2,
    "WN_config": {
        "n_layers": 8,
        "n_channels": 512,
        "kernel_size": 3
    }
}

}
this is my config json, can you give some adivice?

[syang1993]

@hdmjdp Hi, I tried the default config before with batch_size=4, it works well. So I think you don't need to change the config settings.

For noise line, you can try to set n_group=16, I believe it must exist in 1k, 2k, 3k... Increasing the sigma value during inference may alleviate this issue, but larger sigma may bring more noise into audios. The discontinuity caused by squeezing is the nature of this problem I think.

[hdmjdp]

@syang1993 thanks for replying. My dataset is 24k, and titanv can not use batchsize=4 when seqlen=18000. What is your gpu type?

[hdmjdp]

@syang1993 @azraelkuan Another question. Do you try train using fp16?

[hcwu1993]

@syang1993 hello! your generated sample is very good and it's a 16khz audio. in your config mel_fmax=8000? and i'm confused why not use the default mel_fmax = sample_rate/2(ljspeech ,22050). and is it important?

[zhangyizhong17]

@syang1993

[zhangyizhong17]

@syang1993 Hi, I want to ask something unrelated with waveglow, but also about the noise line.
I trained parallel wavenet using different datasets. One (Ljspeech 16k)got noise lines every 2k Hz, well the other (open sourced mandarin data from Databaker) every 800 Hz. They both share the same parameters. I always think it's the data that cause the noise line not squeezing data or something. (I m not sure) Have you ever tried parallel wavenet? Did you got the same issue? The noise line are really noisy.... want to figure out why.
thank you

[syang1993]

@hcwu1993 I used mel_fmax=8000 for 16Khz data. By the way, I don't think this param is so important.

[syang1993]

@zhangyizhong17 How do you sample in parallel wavenet? I tried parallel-wavenet with same database like waveglow, but I didn't find the noise line in the generated samples. I attached a sample from parallel-wavenet using predicted mels.
parallel.zip

So I don't think the noise line is caused by data itself.

[rafaelvalle]

@zhangyizhong17 can you provide a link to the parallel wavenet implementation that was used to generate the sample you shared?

[hdmjdp]

@syang1993 what do you use repo of parallel wavenet?

[syang1993]

@hdmjdp you mean the repo I used to train parallel wavenet or the mel-prediction method?

[zhangyizhong17]

@syang1993 sorry I couldn't find my sample any more. it was 2-3 month ago. my folder is like a mess.
Does your parallel wavenet model use MoL as loss function? I did it using gaussian mentioned in clarinet.

[zhangyizhong17]

@rafaelvalle
https://github.com/azraelkuan/parallel_wavenet_vocoder
my samples are generated based on azraelkuan's implementation.

[syang1993]

@zhangyizhong17 Yes, the sample I attached used MoL.

[hdmjdp]

@syang1993 Yes.

[rafaelvalle]

Closing due to inactivity.