TypeError: __init__() got an unexpected keyword argument 'serialized_options'

[loretoparisi]

I finally was able to install all required packages, but after that:

Traceback (most recent call last):
  File "Predict.py", line 3, in <module>
    import Evaluate
  File "/Users/loretoparisi/Documents/Projects/AI/Wave-U-Net/Evaluate.py", line 2, in <module>
    import tensorflow as tf
  File "/Library/Python/2.7/site-packages/tensorflow/__init__.py", line 24, in <module>
    from tensorflow.python import pywrap_tensorflow  # pylint: disable=unused-import
  File "/Library/Python/2.7/site-packages/tensorflow/python/__init__.py", line 59, in <module>
    from tensorflow.core.framework.graph_pb2 import *
  File "/Library/Python/2.7/site-packages/tensorflow/core/framework/graph_pb2.py", line 15, in <module>
    from tensorflow.core.framework import node_def_pb2 as tensorflow_dot_core_dot_framework_dot_node__def__pb2
  File "/Library/Python/2.7/site-packages/tensorflow/core/framework/node_def_pb2.py", line 15, in <module>
    from tensorflow.core.framework import attr_value_pb2 as tensorflow_dot_core_dot_framework_dot_attr__value__pb2
  File "/Library/Python/2.7/site-packages/tensorflow/core/framework/attr_value_pb2.py", line 15, in <module>
    from tensorflow.core.framework import tensor_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__pb2
  File "/Library/Python/2.7/site-packages/tensorflow/core/framework/tensor_pb2.py", line 15, in <module>
    from tensorflow.core.framework import resource_handle_pb2 as tensorflow_dot_core_dot_framework_dot_resource__handle__pb2
  File "/Library/Python/2.7/site-packages/tensorflow/core/framework/resource_handle_pb2.py", line 22, in <module>
    serialized_pb=_b('\n/tensorflow/core/framework/resource_handle.proto\x12\ntensorflow\"r\n\x13ResourceHandleProto\x12\x0e\n\x06\x64\x65vice\x18\x01 \x01(\t\x12\x11\n\tcontainer\x18\x02 \x01(\t\x12\x0c\n\x04name\x18\x03 \x01(\t\x12\x11\n\thash_code\x18\x04 \x01(\x04\x12\x17\n\x0fmaybe_type_name\x18\x05 \x01(\tBn\n\x18org.tensorflow.frameworkB\x0eResourceHandleP\x01Z=github.com/tensorflow/tensorflow/tensorflow/go/core/framework\xf8\x01\x01\x62\x06proto3')
TypeError: __init__() got an unexpected keyword argument 'serialized_options'

I have linked the pre-trained nmodels in the checkpoints folder

(.env) ip-192-168-22-127:checkpoints loretoparisi$ tree -L 1
.
├── README.md
├── baseline_stereo -> /Users/loretoparisi/wavenet/models/baseline_stereo
└── full_multi_instrument -> /Users/loretoparisi/wavenet/models/full_multi_instrument

2 directories, 1 file

[loretoparisi]

[UPDATE]
This seems to be related to tensorflow/models#3995
Only a specific version of protobuf and tensorflow will work.

[f90]

So this issue did not resolve after switching out tensorflow-gpu with tensorflow, since you said in the other issue you do not have an NVIDIA GPU set up?

Which version of protobuf made it work in our case here, given that we use TF version 1.8? Then I can add that to the requirements.txt file for everyone, I suppose. I will be making some checks whether that interferes with the Linux setup though.
Thanks

[loretoparisi]

So see what happens:

(.env) ip-192-168-22-127:Wave-U-Net loretoparisi$ pip show protobuf
Name: protobuf
Version: 3.6.1
Summary: Protocol Buffers
Home-page: https://developers.google.com/protocol-buffers/
Author: None
Author-email: None
License: 3-Clause BSD License
Location: /Library/Python/2.7/site-packages
Requires: six, setuptools
Required-by: tensorflow, tensorboard
(.env) ip-192-168-22-127:Wave-U-Net loretoparisi$ pip show tensorflow
Name: tensorflow
Version: 1.12.0
Summary: TensorFlow is an open source machine learning framework for everyone.
Home-page: https://www.tensorflow.org/
Author: Google Inc.
Author-email: opensource@google.com
License: Apache 2.0
Location: /Library/Python/2.7/site-packages
Requires: enum34, keras-preprocessing, wheel, astor, backports.weakref, mock, tensorboard, termcolor, protobuf, gast, absl-py, grpcio, six, keras-applications, numpy
Required-by: 

This is with the requirements. It seems that pip has decided to go for Version: 1.12.0 even if the requirements spec is clearly 1.8.0 (!) - and I'm within virtualenv with the new requirements

(.env) ip-192-168-22-127:Wave-U-Net loretoparisi$ cat requirements.txt 
numpy==1.15.4
sacred==0.7.3
tensorflow==1.8.0
librosa==0.6.2
scikit-image==0.13.1
soundfile==0.10.2
scikits.audiolab==0.11.0
lxml==4.2.1
musdb==0.2.3
museval==0.2.0
google==2.0.1
protobuf==3.4.0

[f90]

OK that's super weird... Does the problem go away if you then uninstall that TF (including tensorboard) and then install TF 1.8 manually? Together with protobuf 3.4.0?

[loretoparisi]

Yes it is look here!

(.env) ip-192-168-22-127:Wave-U-Net loretoparisi$ pip install tensorflow==1.8.0
Collecting tensorflow==1.8.0
  Using cached https://files.pythonhosted.org/packages/9b/1e/d89f1369b5b8045e5aedf43718b45d2396d3c61e9cc56123c24b7758dd9f/tensorflow-1.8.0-cp27-cp27m-macosx_10_11_x86_64.whl
Requirement already satisfied: protobuf>=3.4.0 in /Library/Python/2.7/site-packages (from tensorflow==1.8.0) (3.6.1)
Requirement already satisfied: wheel in /Library/Python/2.7/site-packages (from tensorflow==1.8.0) (0.29.0)
Requirement already satisfied: enum34>=1.1.6 in /Library/Python/2.7/site-packages (from tensorflow==1.8.0) (1.1.6)
Collecting tensorboard<1.9.0,>=1.8.0 (from tensorflow==1.8.0)
  Using cached https://files.pythonhosted.org/packages/4d/1e/3bfb48ff165e331c0c5fdca5de79d497fa5d71f3cb2eee2733ff22e898df/tensorboard-1.8.0-py2-none-any.whl
Requirement already satisfied: astor>=0.6.0 in /Library/Python/2.7/site-packages (from tensorflow==1.8.0) (0.7.1)
Requirement already satisfied: backports.weakref>=1.0rc1 in /Library/Python/2.7/site-packages (from tensorflow==1.8.0) (1.0.post1)
Requirement already satisfied: mock>=2.0.0 in /Library/Python/2.7/site-packages (from tensorflow==1.8.0) (2.0.0)
Requirement already satisfied: termcolor>=1.1.0 in /Library/Python/2.7/site-packages (from tensorflow==1.8.0) (1.1.0)
Requirement already satisfied: gast>=0.2.0 in /Library/Python/2.7/site-packages (from tensorflow==1.8.0) (0.2.0)
Requirement already satisfied: absl-py>=0.1.6 in /Library/Python/2.7/site-packages (from tensorflow==1.8.0) (0.6.1)
Requirement already satisfied: grpcio>=1.8.6 in /Library/Python/2.7/site-packages (from tensorflow==1.8.0) (1.16.1)
Requirement already satisfied: six>=1.10.0 in /Library/Python/2.7/site-packages (from tensorflow==1.8.0) (1.11.0)
Requirement already satisfied: numpy>=1.13.3 in /Library/Python/2.7/site-packages (from tensorflow==1.8.0) (1.15.1)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf>=3.4.0->tensorflow==1.8.0) (40.6.2)
Requirement already satisfied: futures>=3.1.1; python_version < "3" in /Library/Python/2.7/site-packages (from tensorboard<1.9.0,>=1.8.0->tensorflow==1.8.0) (3.2.0)
Requirement already satisfied: bleach==1.5.0 in ./.env/lib/python2.7/site-packages (from tensorboard<1.9.0,>=1.8.0->tensorflow==1.8.0) (1.5.0)
Requirement already satisfied: html5lib==0.9999999 in ./.env/lib/python2.7/site-packages (from tensorboard<1.9.0,>=1.8.0->tensorflow==1.8.0) (0.9999999)
Requirement already satisfied: werkzeug>=0.11.10 in /Library/Python/2.7/site-packages (from tensorboard<1.9.0,>=1.8.0->tensorflow==1.8.0) (0.14.1)
Requirement already satisfied: markdown>=2.6.8 in /Library/Python/2.7/site-packages (from tensorboard<1.9.0,>=1.8.0->tensorflow==1.8.0) (3.0.1)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /Library/Python/2.7/site-packages (from mock>=2.0.0->tensorflow==1.8.0) (1.0.2)
Requirement already satisfied: pbr>=0.11 in /Library/Python/2.7/site-packages (from mock>=2.0.0->tensorflow==1.8.0) (4.2.0)
Installing collected packages: tensorboard, tensorflow
  Found existing installation: tensorboard 1.12.0
    Not uninstalling tensorboard at /Library/Python/2.7/site-packages, outside environment /Users/loretoparisi/Documents/Projects/AI/Wave-U-Net/.env
    Can't uninstall 'tensorboard'. No files were found to uninstall.
  Found existing installation: tensorflow 1.12.0
    Not uninstalling tensorflow at /Library/Python/2.7/site-packages, outside environment /Users/loretoparisi/Documents/Projects/AI/Wave-U-Net/.env
    Can't uninstall 'tensorflow'. No files were found to uninstall.
Successfully installed tensorboard-1.8.0 tensorflow-1.8.0
(.env) ip-192-168-22-127:Wave-U-Net loretoparisi$ pip show tensorflow
ùName: tensorflow
Version: 1.12.0
Summary: TensorFlow is an open source machine learning framework for everyone.
Home-page: https://www.tensorflow.org/
Author: Google Inc.
Author-email: opensource@google.com
License: Apache 2.0
Location: /Library/Python/2.7/site-packages
Requires: enum34, keras-preprocessing, wheel, astor, backports.weakref, mock, tensorboard, termcolor, protobuf, gast, absl-py, grpcio, six, keras-applications, numpy
Required-by:

It says Successfully installed tensorboard-1.8.0 tensorflow-1.8.0 but it shows me the 1.12.0 💣

[loretoparisi]

ok got it. Outside virtualenv there was a tensorflow 1.12.0 installed for Google' BERT argh!
Now I have it:

(.env) ip-192-168-22-127:Wave-U-Net loretoparisi$ pip show tensorflow
Name: tensorflow
Version: 1.8.0
Summary: TensorFlow helps the tensors flow
Home-page: https://www.tensorflow.org/
Author: Google Inc.
Author-email: opensource@google.com
License: Apache 2.0
Location: /Users/loretoparisi/Documents/Projects/AI/Wave-U-Net/.env/lib/python2.7/site-packages
Requires: protobuf, wheel, enum34, tensorboard, astor, backports.weakref, mock, termcolor, gast, absl-py, grpcio, six, numpy
Required-by: 

[f90]

Yes during installation in your log you can see that it wasn't able to deinstall TF 1.12
The big question is now, does it work finally using TF 1.8? And with which protobuf version?
Thanks for going through this haha

[loretoparisi]

It works now on macOS !!!

Training baseline model with difference output and input context (valid convolutions)
WARNING - Waveunet Prediction - No observers have been added to this run
INFO - Waveunet Prediction - Running command 'main'
INFO - Waveunet Prediction - Started
Producing source estimates for input mixture file audio_examples/The Mountaineering Club - Mallory/mix.mp3
Testing...
2018-11-15 21:09:29.334000: I tensorflow/core/platform/cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
Num of variables52
INFO:tensorflow:Restoring parameters from checkpoints/baseline_stereo/baseline_stereo-186093
INFO - tensorflow - Restoring parameters from checkpoints/baseline_stereo/baseline_stereo-186093
Pre-trained model restored for song prediction
/Library/Python/2.7/site-packages/scipy/signal/signaltools.py:2383: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use `arr[tuple(seq)]` instead of `arr[seq]`. In the future this will be interpreted as an array index, `arr[np.array(seq)]`, which will result either in an error or a different result.
  return y[keep]
INFO - Waveunet Prediction - Completed after 0:00:16

👍 💯 🥇

[f90]

One last question, did you have to install a specific protobuf version, or did it work with the one I am proposing for the requirements (3.4.0)?

[loretoparisi]

I can confirm that it works with this requirements

numpy==1.15.4
sacred==0.7.3
tensorflow==1.8.0
librosa==0.6.2
scikit-image==0.13.1
soundfile==0.10.2
scikits.audiolab==0.11.0
lxml==4.2.1
musdb==0.2.3
museval==0.2.0
google==2.0.1
protobuf==3.4.0

on macOS with CPU only.

[f90]

OK thanks, I will update the repository accordingly. Many thanks for helping me make this project better and more easy-to-use. Have fun with whatever you're applying it on!