[SpecGAN] error generating moments

[niwtr]

Hi, there,
Tried to generate moments of SpecGAN using script:

python train_specgan.py moments --data_dir ../sc09/train/ ./train

Where '../sc09/train' is a folder containing the training split of SC09 dataset (all .WAV files). The script failed with error:

Traceback (most recent call last):
  File "train_specgan.py", line 756, in <module>
    moments(fps, args)
  File "train_specgan.py", line 594, in moments
    prefetch_gpu_num=args.data_prefetch_gpu_num)[0, :, 0, 0]
  File "/raid/home/ntr/wavegan/wavegan/loader.py", line 198, in decode_extract_and_batch
    return iterator.get_next()
  File "/raid/home/ntr/anaconda3/envs/py3tf1.2/lib/python3.6/site-packages/tensorflow/python/data/ops/iterator_ops.py", line 623, in get_next
    return self._next_internal()
  File "/raid/home/ntr/anaconda3/envs/py3tf1.2/lib/python3.6/site-packages/tensorflow/python/data/experimental/ops/prefetching_ops.py", line 259, in _next_internal
    output_types=self._flat_output_types)
  File "/raid/home/ntr/anaconda3/envs/py3tf1.2/lib/python3.6/site-packages/tensorflow/python/ops/gen_experimental_dataset_ops.py", line 460, in experimental_function_buffering_resource_get_next
    _six.raise_from(_core._status_to_exception(e.code, message), None)
  File "<string>", line 3, in raise_from
tensorflow.python.framework.errors_impl.OutOfRangeError: End of sequence
         [[{{node IteratorGetNext}} = IteratorGetNext[output_shapes=[[1,16384,1,1]], output_types=[DT_FLOAT]](IteratorFromStringHandleV2)]] [Op:ExperimentalFunctionBufferingResourceGetNext]

Is there anything I did wrong? Please help me!

Thanks first!

[chrisdonahue]

Can you print out len(fps) to see how large it is? I'm not sure but it seems like the script is not finding all of your files.

[hinash88]

I am having same issue, Printing the len(fps), it is 860 (My total training examples)

Traceback (most recent call last): File "train_specgan.py", line 760, in <module> moments(fps, args) File "train_specgan.py", line 614, in moments _X_lmags = np.concatenate(_X_lmags, axis=0) File "<__array_function__ internals>", line 6, in concatenate ValueError: need at least one array to concatenate

[sanchita-ghose]

@hinash88 have you fixed the error?

[alexanderbarnhill]

@hinash88 @sanchita-ghose

So I've been able to generate the moments, but the fact is this code is just not all that viable anymore unless you're running an older (1.x) version of TF. So here's what I did.

1. Run update script on train_specgan.py

tf_upgrade_v2 --infile train_specgan.py --outfile train_specgan_v2.py

2. This will produce some errors, as the contrib library no longer exists in tf. So for every instance of contrib in train_specgan.py you have to change it from: tf.contrib.signal to tf.signal

3. Then because of the way tensorflow works, nothing will actually be done until the session is allowed to run, and because tensorflow switched to eager execution at some point, this session to create the moments will actually never be run, and this is the cause of the need at least one array to concatenate error. So, the simplest fix is to add the following line at the top of the file:

tf.compat.v1.disable_eager_execution()

I hope this helps. I don't know a lot about tensorflow, as I try to stick to pytorch, so I'm not sure if the changes I've made are in line with the best practices, but it seems to be working for me. If you are stuck in a 1.x version then I can't help you, I don't think. Best of luck!

[antonjeran12]

@alexanderbarnhill ,

I tried what you have mentioned but I am still getting the error (need at least one array to concatenate).

I am using tensorflow gpu 1.14
Cuda compilation tools, release 10.0, V10.0.130
CUDA Version: 11.0