nsynth_generate fails due to wrong shape - "Encoding file should have 2 dims [time, channels], not 3"

[ahihi]

Ubuntu 18.04
Python 2.7.16
magenta-gpu 1.1.2
tensorflow-gpu 1.14.0

i ran 01_compute_input_embeddings.py, 02_compute_new_embeddings.py and 03_batch_embeddings.py successfully.

however the data now has a different shape than nsynth_generate seems to expect:

$ nsynth_generate --checkpoint_path=/home/sopi/open-nsynth-super/magenta/magenta/models/nsynth/wavenet-ckpt/model.ckpt-200000 --source_path=/home/sopi/open-nsynth-super/audio/workdir/embeddings_batched/batch0 --save_path=/home/sopi/open-nsynth-super/audio/workdir/audio_output/batch0 --batch_size=512 --alsologtostderr --gpu_number=0
WARNING: Logging before flag parsing goes to stderr.
W0701 15:06:16.675589 139689732515200 deprecation_wrapper.py:119] From /home/sopi/miniconda3/envs/open-nsynth-super2/lib/python2.7/site-packages/magenta/pipelines/statistics.py:132: The name tf.logging.info is deprecated. Please use tf.compat.v1.logging.info instead.

W0701 15:06:17.360516 139689732515200 deprecation_wrapper.py:119] From /home/sopi/miniconda3/envs/open-nsynth-super2/lib/python2.7/site-packages/magenta/music/note_sequence_io.py:60: The name tf.python_io.TFRecordWriter is deprecated. Please use tf.io.TFRecordWriter instead.

W0701 15:06:17.671566 139689732515200 lazy_loader.py:50]
The TensorFlow contrib module will not be included in TensorFlow 2.0.
For more information, please see:
  * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
  * https://github.com/tensorflow/addons
  * https://github.com/tensorflow/io (for I/O related ops)
If you depend on functionality not listed there, please file an issue.

W0701 15:06:17.672234 139689732515200 deprecation.py:323] From /home/sopi/miniconda3/envs/open-nsynth-super2/lib/python2.7/site-packages/tensorflow/python/util/deprecation.py:507: __init__ (from tensorflow.python.ops.init_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.initializers.variance_scaling instead with distribution=uniform to get equivalent behavior.
W0701 15:06:17.672956 139689732515200 deprecation_wrapper.py:119] From /home/sopi/miniconda3/envs/open-nsynth-super2/lib/python2.7/site-packages/magenta/models/nsynth/wavenet/nsynth_generate.py:110: The name tf.app.run is deprecated. Please use tf.compat.v1.app.run instead.

W0701 15:06:17.673314 139689732515200 deprecation_wrapper.py:119] From /home/sopi/miniconda3/envs/open-nsynth-super2/lib/python2.7/site-packages/magenta/models/nsynth/wavenet/nsynth_generate.py:50: The name tf.logging.set_verbosity is deprecated. Please use tf.compat.v1.logging.set_verbosity instead.

W0701 15:06:17.673423 139689732515200 deprecation_wrapper.py:119] From /home/sopi/miniconda3/envs/open-nsynth-super2/lib/python2.7/site-packages/magenta/models/nsynth/wavenet/nsynth_generate.py:53: The name tf.gfile.IsDirectory is deprecated. Please use tf.io.gfile.isdir instead.

W0701 15:06:17.673540 139689732515200 deprecation_wrapper.py:119] From /home/sopi/miniconda3/envs/open-nsynth-super2/lib/python2.7/site-packages/magenta/models/nsynth/wavenet/nsynth_generate.py:54: The name tf.gfile.ListDirectory is deprecated. Please use tf.io.gfile.listdir instead.

Traceback (most recent call last):
  File "/home/sopi/miniconda3/envs/open-nsynth-super2/bin/nsynth_generate", line 10, in <module>
    sys.exit(console_entry_point())
  File "/home/sopi/miniconda3/envs/open-nsynth-super2/lib/python2.7/site-packages/magenta/models/nsynth/wavenet/nsynth_generate.py", line 110, in console_entry_point
    tf.app.run(main)
  File "/home/sopi/miniconda3/envs/open-nsynth-super2/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 40, in run
    _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
  File "/home/sopi/miniconda3/envs/open-nsynth-super2/lib/python2.7/site-packages/absl/app.py", line 300, in run
    _run_main(main, args)
  File "/home/sopi/miniconda3/envs/open-nsynth-super2/lib/python2.7/site-packages/absl/app.py", line 251, in _run_main
    sys.exit(main(argv))
  File "/home/sopi/miniconda3/envs/open-nsynth-super2/lib/python2.7/site-packages/magenta/models/nsynth/wavenet/nsynth_generate.py", line 97, in main
    batch_files, sample_length=sample_length)
  File "/home/sopi/miniconda3/envs/open-nsynth-super2/lib/python2.7/site-packages/magenta/models/nsynth/wavenet/fastgen.py", line 159, in load_batch_encodings
    "[time, channels], not {}".format(data.ndim))
ValueError: Encoding file should have 2 dims [time, channels], not 3

checking one of the .npy files, i see it has shape (1, 125, 16), resulting from this reshape in 02_compute_new_embeddings.py:

#	reshape array
interpolated = np.reshape(interpolated, (1,) + interpolated.shape)

if i comment out this line, nsynth_generate starts to run. no idea whether this makes sense though… surely there is a reason for the reshape?

any advice would be appreciated.

[Deftaudio]

I have the same problem, have you found the solution to it?

[ahihi]

yes, just removing the reshape seems to be fine! i was able to generate audio after a few days and the results sound like i would expect.

[Deftaudio]

Thank you, that's what I'm doing now.

[where-is-brett]

I'm experiencing the same problem 3 years on. Glad to hear that removing the extra dimension solves the problem!