Example in Keras guide does not work when training with tf.data datasets #22207
Description
Please go to Stack Overflow for help and support:
https://stackoverflow.com/questions/tagged/tensorflow
If you open a GitHub issue, here is our policy:
- It must be a bug, a feature request, or a significant problem with documentation (for small docs fixes please send a PR instead).
- The form below must be filled out.
- It shouldn't be a TensorBoard issue. Those go here.
Here's why we have that policy: TensorFlow developers respond to issues. We want to focus on work that benefits the whole community, e.g., fixing bugs and adding features. Support only helps individuals. GitHub also notifies thousands of people when issues are filed. We want them to see you communicating an interesting problem, rather than being redirected to Stack Overflow.
System information
- Have I written custom code (as opposed to using a stock example script provided in TensorFlow):no
- OS Platform and Distribution (e.g., Linux Ubuntu 16.04):Windows 10
- Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:na
- TensorFlow installed from (source or binary):binary
- TensorFlow version (use command below):1.11.0-dev20180907
- Python version:v3.6.3:2c5fed8
- Bazel version (if compiling from source):na
- GCC/Compiler version (if compiling from source):na
- CUDA/cuDNN version:na
- GPU model and memory:na
- Exact command to reproduce:na
You can collect some of this information using our environment capture script:
https://github.com/tensorflow/tensorflow/tree/master/tools/tf_env_collect.sh
You can obtain the TensorFlow version with
python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)"
Describe the problem
Describe the problem clearly here. Be sure to convey here why it's a bug in TensorFlow or a feature request.
When going through the tutorial in https://www.tensorflow.org/guide/keras, training with numpy arrays works, but training with tf.data datasets only works if you first perform training with numpy arrays. I expected that you could train with tf.data datasets without first going through numpy arrays. This issue was first identified in #20827 but it was closed as solved at the moment, for some nightly build over tensorflow v1.9.
Source code / logs
Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached. Try to provide a reproducible test case that is the bare minimum necessary to generate the problem.
The code below works if one uncomments the line #model.fit(data, labels, epochs=10, batch_size=32). If it runs as it is, it will give an error, showed below.
Code:
import tensorflow as tf
from tensorflow import keras
model = keras.Sequential()
# Adds a densely-connected layer with 64 units to the model:
model.add(keras.layers.Dense(64, activation='relu'))
# Add another:
model.add(keras.layers.Dense(64, activation='relu'))
# Add a softmax layer with 10 output units:
model.add(keras.layers.Dense(10, activation='softmax'))
model.compile(optimizer=tf.train.AdamOptimizer(0.001),
loss='categorical_crossentropy',
metrics=['accuracy'])
# Configure a model for categorical classification.
model.compile(optimizer=tf.train.RMSPropOptimizer(0.01),
loss=keras.losses.categorical_crossentropy,
metrics=[keras.metrics.categorical_accuracy])
import numpy as np
data = np.random.random((1000, 32))
labels = np.random.random((1000, 10))
#model.fit(data, labels, epochs=10, batch_size=32)
# Instantiates a toy dataset instance:
dataset = tf.data.Dataset.from_tensor_slices((data, labels))
dataset = dataset.batch(32)
dataset = dataset.repeat()
# Don't forget to specify `steps_per_epoch` when calling `fit` on a dataset.
model.fit(dataset, epochs=10, steps_per_epoch=30)
Error:
Traceback (most recent call last):
File "...\tensorflow\python\framework\op_def_library.py", line 510, in _apply_op_helper
preferred_dtype=default_dtype)
File "...\tensorflow\python\framework\ops.py", line 1145, in internal_convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
File "...\tensorflow\python\ops\variables.py", line 799, in _TensorConversionFunction
"of type '%s'" % (dtype.name, v.dtype.name))
ValueError: Incompatible type conversion requested to type 'float64' for variable of type 'float32'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "...\temp.py", line 34, in <module>
model.fit(dataset, epochs=10, steps_per_epoch=30)
File "...\tensorflow\python\keras\engine\training.py", line 1440, in fit
validation_split=validation_split)
File "...\tensorflow\python\keras\engine\training.py", line 944, in _standardize_user_data
class_weight, batch_size)
File "...\tensorflow\python\keras\engine\training.py", line 984, in _standardize_weights
self._set_inputs(x)
File "...\tensorflow\python\training\checkpointable\base.py", line 426, in _method_wrapper
method(self, *args, **kwargs)
File "...\tensorflow\python\keras\engine\training.py", line 1198, in _set_inputs
self._symbolic_set_inputs(inputs, training=training)
File "...\tensorflow\python\training\checkpointable\base.py", line 426, in _method_wrapper
method(self, *args, **kwargs)
File "...\tensorflow\python\keras\engine\training.py", line 1282, in _symbolic_set_inputs
outputs = self.call(dummy_input_values, training=training)
File "...\tensorflow\python\keras\engine\sequential.py", line 232, in call
inputs, training=training, mask=mask)
File "...\tensorflow\python\keras\engine\sequential.py", line 250, in _call_and_compute_mask
x = layer.call(x, **kwargs)
File "...\tensorflow\python\keras\layers\core.py", line 947, in call
outputs = gen_math_ops.mat_mul(inputs, self.kernel)
File "...\tensorflow\python\ops\gen_math_ops.py", line 4856, in mat_mul
name=name)
File "...\tensorflow\python\framework\op_def_library.py", line 546, in _apply_op_helper
inferred_from[input_arg.type_attr]))
TypeError: Input 'b' of 'MatMul' Op has type float32 that does not match type float64 of argument 'a'.
Process finished with exit code 1