Training YOLO object detection with PlaidML?

[borderlineinteractive]

Hi, I was wondering if training of YOLO is possible with PlaidML. So far, I could not find a YOLO implementation in pure Keras 2.0.8. I tried keras-yolo2 from this source: (https://github.com/experiencor/keras-yolo2), but I get the following error when I try to start the training using Tiny Yolo:

TypeError: Failed to convert object of type <class 'plaidml.tile.Value'> to Tensor. Contents: lambda_1_target Placeholder FLOAT32(<tile.Value SymbolicDim UINT64()>, <tile.Value SymbolicDim UINT64()>, <tile.Value SymbolicDim UINT64()>, <tile.Value SymbolicDim UINT64()>, <tile.Value SymbolicDim UINT64()>). Consider casting elements to a supported type

I guess that this is due to tensorflow specific code and this warning:

UserWarning: output_shape argument not specified for layer lambda_1 and cannot be automatically inferred with the Theano backend. Defaulting to output shape [(None, 13, 13, 5, 6), (None, 1, 1, 1, 10, 4)] (same as input shape). If the expected output shape is different, specify it via the output_shape argument.

Did anybody have success with implementing YOLO on PlaidML or does anybody have a recommendation how to fix this?

Here is the full output from training using the keras-yolo2 code:

(plaidml) iMac:keras-yolo2 username$ python train.py -c config.json
/Users/username/plaidml/lib/python2.7/site-packages/h5py/init.py:36: FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type.
from ._conv import register_converters as _register_converters
('Seen labels:\t', {'raccoon': 217})
('Given labels:\t', [u'raccoon'])
('Overlap labels:\t', set(['raccoon']))
INFO:plaidml:Opening device "amd_radeon_pro_580_compute_engine.0"
(13, 13)
/Users/username/plaidml/lib/python2.7/site-packages/keras/layers/core.py:629: UserWarning: output_shape argument not specified for layer lambda_1 and cannot be automatically inferred with the Theano backend. Defaulting to output shape [(None, 13, 13, 5, 6), (None, 1, 1, 1, 10, 4)] (same as input shape). If the expected output shape is different, specify it via the output_shape argument.
.format(self.name, input_shape))

---

Layer (type) Output Shape Param # Connected to

input_1 (InputLayer) (None, 416, 416, 3) 0

---

model_1 (Model) multiple 15739760 input_1[0][0]

---

DetectionLayer (Conv2D) (None, 13, 13, 30) 30750 model_1[1][0]

---

reshape_1 (Reshape) (None, 13, 13, 5, 6) 0 DetectionLayer[0][0]

---

input_2 (InputLayer) (None, 1, 1, 1, 10, 4 0

---

lambda_1 (Lambda) [(None, 13, 13, 5, 6) 0 reshape_1[0][0]
input_2[0][0]

Total params: 15,770,510
Trainable params: 15,764,398
Non-trainable params: 6,112

---

Traceback (most recent call last):
File "train.py", line 105, in
main(args)
File "train.py", line 101, in main
debug = config['train']['debug'])
File "/Users/username/plaidml/keras-yolo2/frontend.py", line 301, in train
self.model.compile(loss=self.custom_loss, optimizer=optimizer)
File "/Users/username/plaidml/lib/python2.7/site-packages/keras/engine/training.py", line 850, in compile
sample_weight, mask)
File "/Users/username/plaidml/lib/python2.7/site-packages/keras/engine/training.py", line 450, in weighted
score_array = fn(y_true, y_pred)
File "/Users/username/plaidml/keras-yolo2/frontend.py", line 87, in custom_loss
mask_shape = tf.shape(y_true)[:4]
File "/Users/username/plaidml/lib/python2.7/site-packages/tensorflow/python/ops/array_ops.py", line 285, in shape
return shape_internal(input, name, optimize=True, out_type=out_type)
File "/Users/username/plaidml/lib/python2.7/site-packages/tensorflow/python/ops/array_ops.py", line 309, in shape_internal
input_tensor = ops.convert_to_tensor(input)
File "/Users/username/plaidml/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1014, in convert_to_tensor
as_ref=False)
File "/Users/username/plaidml/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1104, in internal_convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
File "/Users/username/plaidml/lib/python2.7/site-packages/tensorflow/python/framework/constant_op.py", line 235, in _constant_tensor_conversion_function
return constant(v, dtype=dtype, name=name)
File "/Users/username/plaidml/lib/python2.7/site-packages/tensorflow/python/framework/constant_op.py", line 214, in constant
value, dtype=dtype, shape=shape, verify_shape=verify_shape))
File "/Users/username/plaidml/lib/python2.7/site-packages/tensorflow/python/framework/tensor_util.py", line 521, in make_tensor_proto
"supported type." % (type(values), values))
TypeError: Failed to convert object of type <class 'plaidml.tile.Value'> to Tensor. Contents: lambda_1_target Placeholder FLOAT32(<tile.Value SymbolicDim UINT64()>, <tile.Value SymbolicDim UINT64()>, <tile.Value SymbolicDim UINT64()>, <tile.Value SymbolicDim UINT64()>, <tile.Value SymbolicDim UINT64()>). Consider casting elements to a supported type.

Thanks a lot!

[kpot]

It seems to me that this implementation strictly depends on TensorFlow being used. So much so that I wouldn't even call it a Keras implementation. For example, here I see direct calls to TF all over the place. Such code will not work with any other Keras backend, including PlaidML.

[tzerrell]

Yes, as @kpot suggested running this requires either a TF backend or a rewrite of this code to not involve direct TF calls.

[HenslerSoftware]

Has anybody managed to get this to work with PlaidML or is there another repo where we can train a object detection model using PlaidML?

[hemangjoshi37a]

same issue here.
Please any developer help.