Plaidml backend slower than TF backend

[ptuchster]

I'd tried both plaidml and tf as backend for keras on my mbp with radeon pro 560x, but I do not get performance from using plaidml.

Here is example of my simple model:

os.environ["KERAS_BACKEND"] = "plaidml.keras.backend"
import keras

model = keras.Sequential()
model.add(keras.layers.LSTM(100, input_shape=(N, N_F), return_sequences=True))
model.add(keras.layers.LSTM(100, dropout=0.3))
model.add(keras.layers.Dense(train_y.shape[1], activation='sigmoid'))

model.compile(loss=keras.losses.mean_absolute_error, optimizer=keras.optimizers.Adam(), metrics=[mean_abs_sec_diff])

When I start training I see gpu access string: INFO:plaidml:Opening device "metal_amd_radeon_pro_560x.0"

I have over 1m training data, and also tried different batch size (even 1, what extremely slow), and usually plaidml training 10-15% slower than cpu tensorflow.
It feels like plaidml doesn't reallly use my gpu.

plaidbench 0.5.0
plaidml 0.6.4
plaidml-keras 0.6.4
with python 3

Regards.

[denise-k]

Hi @ptuchster - thanks for reporting this.

You may want to take a look at #171 which discusses slowness in LSTM/embedding layers due to inefficient implementations of scatter/gather.

To summarize: we're aware of the slowness in this case and our team has scoped out fixing this; however, there's a nontrivial amount of engineering effort under the hood that's required to fix scatter/gather. Additionally, we are replacing our current way of representing operations in PlaidML (Tile) with a new representation (Stripe) in the near future, and it makes more sense to implement these changes in Stripe rather than Tile. Improving scatter/gather performance is one of our top priorities for Stripe, and we plan to have these changes available soon.

Thanks for using PlaidML!

[iperov]

but actually plaidml really slower than tf in most cases.
plaidml faster than tf only on small models.

[ptuchster]

but actually plaidml really slower than tf in most cases.
plaidml faster than tf only on small models.

I don't really think that 3 layer model really "big". Actually, yesterday I worked around small model without any memory layers, so simple dense sequence, and it's really don't work. All way why I really wanted to test out plaidml is compatibility with amd gpu. So no luck after all. Cpu flavor TF really faster for me now.

[scode-cmd]

PlaidML is also slower on my Macbook with Vega 20 GPU than TensorFlow CPU. My model is a CNN with 8 million parameters (no RNN layers). I can see that my GPU usage is 100% when training. It's disappointing that PlaidML has 0 performance gains.