Using convert_tacotron2_torch_to_tf.py returns an error

[KakoozaJerry]

I run this command to convert my tacotron2 model to a tf model
!python convert_tacotron2_torch_to_tf.py --torch_model_path /content/drive/MyDrive/Emergent/train/Day11_newData/coqui_tts-November-08-2021_09+51PM-33aa27e2/best_model.pth.tar --config_path /content/drive/MyDrive/Emergent/train/Day11_newData/coqui_tts-November-08-2021_09+51PM-33aa27e2/config.json --output_path /content/

Using this script https://github.com/coqui-ai/TTS/blob/33aa27e2d6a80d38a7ea8928ad14d032d2c53a40/TTS/bin/convert_tacotron2_torch_to_tf.py

The error I get is shown below

> Using model: tacotron2
(1, None, 80)
(1, None, 80)
[('embedding/embeddings:0', 'embedding.weight'),
 ('encoder/lstm/forward_lstm/lstm_cell_1/kernel:0',
  'encoder.lstm.weight_ih_l0'),
 ('encoder/lstm/forward_lstm/lstm_cell_1/recurrent_kernel:0',
  'encoder.lstm.weight_hh_l0'),
 ('encoder/lstm/backward_lstm/lstm_cell_2/kernel:0',
  'encoder.lstm.weight_ih_l0_reverse'),
 ('encoder/lstm/backward_lstm/lstm_cell_2/recurrent_kernel:0',
  'encoder.lstm.weight_hh_l0_reverse'),
 ('encoder/lstm/forward_lstm/lstm_cell_1/bias:0',
  ('encoder.lstm.bias_ih_l0', 'encoder.lstm.bias_hh_l0')),
 ('encoder/lstm/backward_lstm/lstm_cell_2/bias:0',
  ('encoder.lstm.bias_ih_l0_reverse', 'encoder.lstm.bias_hh_l0_reverse')),
 ('attention/v/kernel:0', 'decoder.attention.v.linear_layer.weight'),
 ('decoder/linear_projection/kernel:0',
  'decoder.linear_projection.linear_layer.weight'),
 ('decoder/stopnet/kernel:0', 'decoder.stopnet.1.linear_layer.weight'),
 ('encoder/convolutions_0/convolution1d/kernel:0',
  'encoder.convolutions.0.convolution1d.weight'),
 ('encoder/convolutions_0/convolution1d/bias:0',
  'encoder.convolutions.0.convolution1d.bias'),
 ('encoder/convolutions_0/batch_normalization/gamma:0',
  'encoder.convolutions.0.batch_normalization.weight'),
 ('encoder/convolutions_0/batch_normalization/beta:0',
  'encoder.convolutions.0.batch_normalization.bias'),
 ('encoder/convolutions_1/convolution1d/kernel:0',
  'encoder.convolutions.1.convolution1d.weight'),
 ('encoder/convolutions_1/convolution1d/bias:0',
  'encoder.convolutions.1.convolution1d.bias'),
 ('encoder/convolutions_1/batch_normalization/gamma:0',
  'encoder.convolutions.1.batch_normalization.weight'),
 ('encoder/convolutions_1/batch_normalization/beta:0',
  'encoder.convolutions.1.batch_normalization.bias'),
 ('encoder/convolutions_2/convolution1d/kernel:0',
  'encoder.convolutions.2.convolution1d.weight'),
 ('encoder/convolutions_2/convolution1d/bias:0',
  'encoder.convolutions.2.convolution1d.bias'),
 ('encoder/convolutions_2/batch_normalization/gamma:0',
  'encoder.convolutions.2.batch_normalization.weight'),
 ('encoder/convolutions_2/batch_normalization/beta:0',
  'encoder.convolutions.2.batch_normalization.bias'),
 ('encoder/convolutions_0/batch_normalization/moving_mean:0',
  'encoder.convolutions.0.batch_normalization.running_mean'),
 ('encoder/convolutions_0/batch_normalization/moving_variance:0',
  'encoder.convolutions.0.batch_normalization.running_var'),
 ('encoder/convolutions_1/batch_normalization/moving_mean:0',
  'encoder.convolutions.1.batch_normalization.running_mean'),
 ('encoder/convolutions_1/batch_normalization/moving_variance:0',
  'encoder.convolutions.1.batch_normalization.running_var'),
 ('encoder/convolutions_2/batch_normalization/moving_mean:0',
  'encoder.convolutions.2.batch_normalization.running_mean'),
 ('encoder/convolutions_2/batch_normalization/moving_variance:0',
  'encoder.convolutions.2.batch_normalization.running_var'),
 ('decoder/while/prenet/linear_layer_0/linear_layer/kernel:0',
  'decoder.prenet.linear_layers.0.linear_layer.weight'),
 ('decoder/while/prenet/linear_layer_1/linear_layer/kernel:0',
  'decoder.prenet.linear_layers.1.linear_layer.weight'),
 ('decoder/while/attention_rnn/kernel:0', 'decoder.attention_rnn.weight_ih'),
 ('decoder/while/attention_rnn/recurrent_kernel:0',
  'decoder.attention_rnn.weight_hh'),
 ('decoder/while/attention_rnn/bias:0', 'decoder.attention_rnn.bias_ih'),
 ('decoder/while/attention/query_layer/linear_layer/kernel:0',
  'decoder.attention.query_layer.linear_layer.weight'),
 ('decoder/attention/inputs_layer/linear_layer/kernel:0',
  'decoder.attention.inputs_layer.linear_layer.weight'),
 ('decoder/while/attention/v/linear_layer/kernel:0',
  'decoder.attention.v.linear_layer.weight'),
 ('decoder/while/attention/v/linear_layer/bias:0',
  'decoder.attention.v.linear_layer.bias'),
 ('decoder/while/attention/location_layer/location_conv1d/kernel:0',
  'decoder.attention.location_layer.location_conv1d.weight'),
 ('decoder/while/attention/location_layer/location_dense/kernel:0',
  'decoder.attention.location_layer.location_dense.linear_layer.weight'),
 ('decoder/while/decoder_rnn/kernel:0', 'decoder.decoder_rnn.weight_ih'),
 ('decoder/while/decoder_rnn/recurrent_kernel:0',
  'decoder.decoder_rnn.weight_hh'),
 ('decoder/while/decoder_rnn/bias:0', 'decoder.decoder_rnn.bias_ih'),
 ('decoder/while/linear_projection/linear_layer/kernel:0',
  'decoder.linear_projection.linear_layer.weight'),
 ('decoder/while/linear_projection/linear_layer/bias:0',
  'decoder.linear_projection.linear_layer.bias'),
 ('decoder/while/stopnet/linear_layer/kernel:0',
  'decoder.stopnet.1.linear_layer.weight'),
 ('decoder/while/stopnet/linear_layer/bias:0',
  'decoder.stopnet.1.linear_layer.bias'),
 ('postnet/convolutions_0/convolution1d/kernel:0',
  'postnet.convolutions.0.convolution1d.weight'),
 ('postnet/convolutions_0/convolution1d/bias:0',
  'postnet.convolutions.0.convolution1d.bias'),
 ('postnet/convolutions_0/batch_normalization/gamma:0',
  'postnet.convolutions.0.batch_normalization.weight'),
 ('postnet/convolutions_0/batch_normalization/beta:0',
  'postnet.convolutions.0.batch_normalization.bias'),
 ('postnet/convolutions_1/convolution1d/kernel:0',
  'postnet.convolutions.1.convolution1d.weight'),
 ('postnet/convolutions_1/convolution1d/bias:0',
  'postnet.convolutions.1.convolution1d.bias'),
 ('postnet/convolutions_1/batch_normalization/gamma:0',
  'postnet.convolutions.1.batch_normalization.weight'),
 ('postnet/convolutions_1/batch_normalization/beta:0',
  'postnet.convolutions.1.batch_normalization.bias'),
 ('postnet/convolutions_2/convolution1d/kernel:0',
  'postnet.convolutions.2.convolution1d.weight'),
 ('postnet/convolutions_2/convolution1d/bias:0',
  'postnet.convolutions.2.convolution1d.bias'),
 ('postnet/convolutions_2/batch_normalization/gamma:0',
  'postnet.convolutions.2.batch_normalization.weight'),
 ('postnet/convolutions_2/batch_normalization/beta:0',
  'postnet.convolutions.2.batch_normalization.bias'),
 ('postnet/convolutions_3/convolution1d/kernel:0',
  'postnet.convolutions.3.convolution1d.weight'),
 ('postnet/convolutions_3/convolution1d/bias:0',
  'postnet.convolutions.3.convolution1d.bias'),
 ('postnet/convolutions_3/batch_normalization/gamma:0',
  'postnet.convolutions.3.batch_normalization.weight'),
 ('postnet/convolutions_3/batch_normalization/beta:0',
  'postnet.convolutions.3.batch_normalization.bias'),
 ('postnet/convolutions_4/convolution1d/kernel:0',
  'postnet.convolutions.4.convolution1d.weight'),
 ('postnet/convolutions_4/convolution1d/bias:0',
  'postnet.convolutions.4.convolution1d.bias'),
 ('postnet/convolutions_4/batch_normalization/gamma:0',
  'postnet.convolutions.4.batch_normalization.weight'),
 ('postnet/convolutions_4/batch_normalization/beta:0',
  'postnet.convolutions.4.batch_normalization.bias'),
 ('postnet/convolutions_0/batch_normalization/moving_mean:0',
  'postnet.convolutions.0.batch_normalization.running_mean'),
 ('postnet/convolutions_0/batch_normalization/moving_variance:0',
  'postnet.convolutions.0.batch_normalization.running_var'),
 ('postnet/convolutions_1/batch_normalization/moving_mean:0',
  'postnet.convolutions.1.batch_normalization.running_mean'),
 ('postnet/convolutions_1/batch_normalization/moving_variance:0',
  'postnet.convolutions.1.batch_normalization.running_var'),
 ('postnet/convolutions_2/batch_normalization/moving_mean:0',
  'postnet.convolutions.2.batch_normalization.running_mean'),
 ('postnet/convolutions_2/batch_normalization/moving_variance:0',
  'postnet.convolutions.2.batch_normalization.running_var'),
 ('postnet/convolutions_3/batch_normalization/moving_mean:0',
  'postnet.convolutions.3.batch_normalization.running_mean'),
 ('postnet/convolutions_3/batch_normalization/moving_variance:0',
  'postnet.convolutions.3.batch_normalization.running_var'),
 ('postnet/convolutions_4/batch_normalization/moving_mean:0',
  'postnet.convolutions.4.batch_normalization.running_mean'),
 ('postnet/convolutions_4/batch_normalization/moving_variance:0',
  'postnet.convolutions.4.batch_normalization.running_var')]
['embedding.weight',
 'encoder.convolutions.0.batch_normalization.num_batches_tracked',
 'encoder.convolutions.1.batch_normalization.num_batches_tracked',
 'encoder.convolutions.2.batch_normalization.num_batches_tracked',
 'encoder.lstm.weight_ih_l0',
 'encoder.lstm.weight_hh_l0',
 'encoder.lstm.bias_ih_l0',
 'encoder.lstm.bias_hh_l0',
 'encoder.lstm.weight_ih_l0_reverse',
 'encoder.lstm.weight_hh_l0_reverse',
 'encoder.lstm.bias_ih_l0_reverse',
 'encoder.lstm.bias_hh_l0_reverse',
 'decoder.attention_rnn.bias_hh',
 'decoder.decoder_rnn.bias_hh',
 'postnet.convolutions.0.batch_normalization.num_batches_tracked',
 'postnet.convolutions.1.batch_normalization.num_batches_tracked',
 'postnet.convolutions.2.batch_normalization.num_batches_tracked',
 'postnet.convolutions.3.batch_normalization.num_batches_tracked',
 'postnet.convolutions.4.batch_normalization.num_batches_tracked',
 'coarse_decoder.prenet.linear_layers.0.linear_layer.weight',
 'coarse_decoder.prenet.linear_layers.1.linear_layer.weight',
 'coarse_decoder.attention_rnn.weight_ih',
 'coarse_decoder.attention_rnn.weight_hh',
 'coarse_decoder.attention_rnn.bias_ih',
 'coarse_decoder.attention_rnn.bias_hh',
 'coarse_decoder.attention.query_layer.linear_layer.weight',
 'coarse_decoder.attention.inputs_layer.linear_layer.weight',
 'coarse_decoder.attention.v.linear_layer.weight',
 'coarse_decoder.attention.v.linear_layer.bias',
 'coarse_decoder.attention.location_layer.location_conv1d.weight',
 'coarse_decoder.attention.location_layer.location_dense.linear_layer.weight',
 'coarse_decoder.decoder_rnn.weight_ih',
 'coarse_decoder.decoder_rnn.weight_hh',
 'coarse_decoder.decoder_rnn.bias_ih',
 'coarse_decoder.decoder_rnn.bias_hh',
 'coarse_decoder.linear_projection.linear_layer.weight',
 'coarse_decoder.linear_projection.linear_layer.bias',
 'coarse_decoder.stopnet.1.linear_layer.weight',
 'coarse_decoder.stopnet.1.linear_layer.bias']
 > Passing weights from Torch to TF ...
 | > embedding/embeddings:0 <-- embedding.weight
Traceback (most recent call last):
  File "convert_tacotron2_torch_to_tf.py", line 105, in <module>
    tf_vars = transfer_weights_torch_to_tf(tf_vars, dict(var_map), state_dict)
  File "/content/TTS/TTS/tts/tf/utils/convert_torch_to_tf_utils.py", line 79, in transfer_weights_torch_to_tf
    ), f" [!] weight shapes does not match: {tf_var.name} vs {torch_var_name} --> {tf_var.shape} vs {numpy_weight.shape}"
AssertionError:  [!] weight shapes does not match: embedding/embeddings:0 vs embedding.weight --> (66, 512) vs (67, 512)

[erogol]

you can only use this for the vanilla Tacotron2 model, not with DDC.

[KakoozaJerry]

Thanks @erogol , Is there any alternative for me to convert the Tacotron2 + DDC model ?

[erogol]

alas not yet