Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

`python examples/keras/mnist-keras-v2.py` has wrong input shape #1194

Closed
see-- opened this issue May 17, 2019 · 2 comments

Comments

Projects
None yet
2 participants
@see--
Copy link
Contributor

commented May 17, 2019

It works if I use this line in keras contrib: https://github.com/tensorpack/tensorpack/blob/master/tensorpack/contrib/keras.py#L101

outputs = model.call(input_tensors[0])

I am not sure about the correct way to fix this. Maybe check if there is only one input_tensors?

If you're asking about an unexpected problem which you do not know the root cause,
use this template. PLEASE DO NOT DELETE THIS TEMPLATE, FILL IT:

If you already know the root cause to your problem,
feel free to delete everything in this template.

1. What you did:

(1) If you're using examples, what's the command you run:
python examples/keras/mnist-keras-v2.py
(2) If you're using examples, have you made any changes to the examples? Paste git status; git diff here:

(3) If not using examples, tell us what you did:

It's always better to copy-paste what you did than to describe them.

Please try to provide enough information to let other reproduce your issues.
Without reproducing the issue, we may not be able to investigate it.

2. What you observed:

(1) Include the ENTIRE logs here:

python3 examples/keras/mnist-keras-v2.py 
/usr/local/lib/python3.5/dist-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters
[0517 10:28:01 @logger.py:90] Argv: examples/keras/mnist-keras-v2.py
[0517 10:28:01 @fs.py:101] WRN Env var $TENSORPACK_DATASET not set, using /home/steffen/tensorpack_data for datasets.
[0517 10:28:02 @gpu.py:43] WRN Found nvidia-smi. But TensorFlow was not built with CUDA support and could not use GPUs!
[0517 10:28:02 @input_source.py:223] Setting up the queue 'QueueInput/input_queue' for CPU prefetching ...
WARNING:tensorflow:From /usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
[0517 10:28:02 @trainers.py:47] Building graph for a single training tower ...
WARNING:tensorflow:From /usr/local/lib/python3.5/dist-packages/tensorflow/python/keras/layers/core.py:143: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.
Instructions for updating:
Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.
WARNING:tensorflow:From /usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
[0517 10:28:02 @model_utils.py:67] List of Trainable Variables: 
name               shape             #elements
-----------------  --------------  -----------
conv2d/kernel:0    [3, 3, 1, 32]           288
conv2d/bias:0      [32]                     32
conv2d_1/kernel:0  [3, 3, 32, 32]         9216
conv2d_1/bias:0    [32]                     32
conv2d_2/kernel:0  [3, 3, 32, 32]         9216
conv2d_2/bias:0    [32]                     32
conv2d_3/kernel:0  [3, 3, 32, 32]         9216
conv2d_3/bias:0    [32]                     32
dense/kernel:0     [1568, 512]          802816
dense/bias:0       [512]                   512
dense_1/kernel:0   [512, 10]              5120
dense_1/bias:0     [10]                     10
Number of trainable variables: 12
Number of parameters (elements): 836522
Storage space needed for all trainable variables: 3.19MB
[0517 10:28:02 @base.py:209] Setup callbacks graph ...
[0517 10:28:02 @keras.py:131] Using Keras learning phase dropout/keras_learning_phase:0 in the graph!
[0517 10:28:02 @inference_runner.py:149] [InferenceRunner] Building tower 'InferenceTower' on device /gpu:0 ...
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 1659, in _create_c_op
    c_op = c_api.TF_FinishOperation(op_desc)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Shape must be rank 4 but is rank 5 for 'InferenceTower/conv2d/Conv2D' (op: 'Conv2D') with input shapes: [1,?,28,28,1], [3,3,1,32].

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "examples/keras/mnist-keras-v2.py", line 72, in <module>
    ModelSaver()
  File "/home/steffen/tensorpack/tensorpack/contrib/keras.py", line 299, in fit
    self.trainer.train_with_defaults(callbacks=callbacks, **kwargs)
  File "/home/steffen/tensorpack/tensorpack/train/base.py", line 344, in train_with_defaults
    steps_per_epoch, starting_epoch, max_epoch)
  File "/home/steffen/tensorpack/tensorpack/train/base.py", line 314, in train
    self.setup_callbacks(callbacks, monitors)
  File "/home/steffen/tensorpack/tensorpack/utils/argtools.py", line 176, in wrapper
    return func(*args, **kwargs)
  File "/home/steffen/tensorpack/tensorpack/train/base.py", line 211, in setup_callbacks
    self._callbacks.setup_graph(weakref.proxy(self))
  File "/home/steffen/tensorpack/tensorpack/callbacks/base.py", line 59, in setup_graph
    self._setup_graph()
  File "/home/steffen/tensorpack/tensorpack/callbacks/group.py", line 70, in _setup_graph
    cb.setup_graph(self.trainer)
  File "/home/steffen/tensorpack/tensorpack/callbacks/base.py", line 59, in setup_graph
    self._setup_graph()
  File "/home/steffen/tensorpack/tensorpack/callbacks/inference_runner.py", line 153, in _setup_graph
    self._tower_func(*self._input_source.get_input_tensors())
  File "/home/steffen/tensorpack/tensorpack/tfutils/tower.py", line 290, in __call__
    output = self._tower_fn(*args)
  File "/home/steffen/tensorpack/tensorpack/contrib/keras.py", line 172, in get_cost
    outputs = model_caller(*input_tensors)
  File "/home/steffen/tensorpack/tensorpack/contrib/keras.py", line 101, in __call__
    outputs = model.call(input_tensors)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/keras/engine/sequential.py", line 230, in call
    return super(Sequential, self).call(inputs, training=training, mask=mask)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/keras/engine/network.py", line 815, in call
    mask=masks)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/keras/engine/network.py", line 1002, in _run_internal_graph
    output_tensors = layer.call(computed_tensor, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/keras/layers/convolutional.py", line 194, in call
    outputs = self._convolution_op(inputs, self.kernel)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/nn_ops.py", line 966, in __call__
    return self.conv_op(inp, filter)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/nn_ops.py", line 591, in __call__
    return self.call(inp, filter)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/nn_ops.py", line 208, in __call__
    name=self.name)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/gen_nn_ops.py", line 1026, in conv2d
    data_format=data_format, dilations=dilations, name=name)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/op_def_library.py", line 788, in _apply_op_helper
    op_def=op_def)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 3300, in create_op
    op_def=op_def)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 1823, in __init__
    control_input_ops)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 1662, in _create_c_op
    raise ValueError(str(e))
ValueError: Shape must be rank 4 but is rank 5 for 'InferenceTower/conv2d/Conv2D' (op: 'Conv2D') with input shapes: [1,?,28,28,1], [3,3,1,32].

It's always better to copy-paste what you observed instead of describing them.

It's always better to paste as much as possible, although sometimes a partial log is OK.

Tensorpack typically saves stdout to its training log.
If stderr is relevant, you can run a command with CMD 2>&1 | tee logs.txt
to save both stdout and stderr to one file.

(2) Other observations, if any:
For example, CPU/GPU utilization, output images, tensorboard curves, if relevant to your issue.

3. What you expected, if not obvious.

If you expect higher speed, please read
http://tensorpack.readthedocs.io/tutorial/performance-tuning.html
before posting.

If you expect certain accuracy, only in one of the two conditions can we help with it:
(1) You're unable to reproduce the accuracy documented in tensorpack examples.
(2) It appears to be a tensorpack bug.

Otherwise, how to train a model to certain accuracy is a machine learning question.
We do not answer machine learning questions and it is your responsibility to
figure out how to make your models more accurate.

4. Your environment:

  • Paste the output of this command: python3 -c 'import tensorpack.tfutils as u; print(u.collect_env_info())'
    If this command failed, tell us your version of Python/TF/tensorpack.
  • You can install Tensorpack master by pip install -U git+https://github.com/ppwwyyxx/tensorpack.git
    and see if your issue is already solved.
  • If you're not using tensorpack under a normal command line shell (e.g.,
    using an IDE or jupyter notebook), please retry under a normal command line shell.
  • Include relevant hardware information, e.g. number of GPUs used for training, amount of RAM.
-------------------  -----------------------------------------------------------
sys.platform         linux
Python               3.5.2 (default, Nov 12 2018, 13:43:14) [GCC 5.4.0 20160609]
Tensorpack           v0.9.4-62-g1e9342a-dirty
Numpy                1.16.1
TensorFlow           1.13.1/b'v1.13.1-0-g6612da8951'
TF Compiler Version  4.8.5
TF CUDA support      False
TF MKL support       False
Free RAM             4.15/7.71 GB
CPU Count            4
cv2                  3.4.3
msgpack              0.5.6
python-prctl         False
-------------------  -----------------------------------------------------------

You may often want to provide extra information related to your issue, but
at the minimum please try to provide the above information accurately to save effort in the investigation.

@ppwwyyxx

This comment has been minimized.

Copy link
Collaborator

commented May 17, 2019

Thanks for the report! The correct fix is outputs = model.call(*input_tensors). I'll push it soon.

@see--

This comment has been minimized.

Copy link
Contributor Author

commented May 17, 2019

Thanks! :)

@ppwwyyxx ppwwyyxx closed this in 0831fe9 May 17, 2019

@ppwwyyxx ppwwyyxx added the examples label May 22, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.