Method to check for GPU op support

[Remper]

So, given the basic word2vec example being bound to CPU (#514), we can see that tf.nn.embedding_lookup doesn't work on GPU. Therefore, ops that use embedding_lookup internally doesn't support GPU either (for example, nce_loss).
Can we have explicit info in the documentation on which operations are currently GPU-capable and which are not?
For example, are tf.gather or LogUniformCandidateSampler GPU-capable?

[Remper]

@vrv is there a way to check CPU/GPU capabilities of an operation from python? Something like print out a list of available backends for each operation?
Because I could update the documentation myself, but I was wondering if there is a more efficient way than just digging sources one operation at the time.

[yaroslavvb]

There doesn't seem to be a more efficient way. Kernel registrations are
done through C++ registry mechanism, so some non-trivial piping would need
to be done to expose it in Python.

[yaroslavvb]

BTW, when tracking capabilities through code, one thing to keep in mind is
that there are logical devices and "physical" devices that don't
necessarily coincide. For instance, the following
from core/kernels/cwise_op_add.cc shows that "Add" for int32 is registered
for GPU logical device, but the actual implementation will run on CPU.
(This was done to avoid crossing logical device boundary for efficiency
reasons)

REGISTER_KERNEL_BUILDER(Name("Add")
.Device(DEVICE_GPU)
.HostMemory("x")
.HostMemory("y")
.HostMemory("z")
.TypeConstraint("T"),
BinaryOp<CPUDevice, functor::add>);

[girving]

The fact that word2vec is bound to CPU is historical: tf.gather works fine on GPU now. I believe LogUniformCandidateSampler is still CPU-only, though.

Unfortunately making the docs not depend on kernel registrations helps significantly, and we'd lose those advantages if we presented this information in the docs. However, a runtime feature for checking which ops have which kernels would be great, so I'll mark this as contributions welcome.

[ibab]

A tool that could produce an HTML table with an overview of

• which ops exist
• which dtypes are supported
• which backends are supported

would be pretty amazing when starting out with tensorflow, and would also be useful for making sure that things are defined in a consistent way.
The table could then also be included in the documentation.
(Or should this rather be a feature of TensorBoard?)
This could be built on top of the runtime checking that's the topic of this issue.