[pytorch-dml] Torch Transfomer

[gmlwns2000]

I tried to run my model in DML and got an issue with executing TransformerEncoder layer.
I followed this tutorial
And it created the following issue.

RuntimeError: Could not run 'aten::bmm' with arguments from the 'UNKNOWN_TENSOR_TYPE_ID' backend. This could be because the operator doesn't exist for this backend, or was omitted during the selective/custom build process (if using custom build). If you are a Facebook employee using PyTorch on mobile, please visit https://fburl.com/ptmfixes for possible resolutions. 'aten::bmm' is only available for these backends: [CPU, SparseCPU, BackendSelect, Named, AutogradOther, AutogradCPU, AutogradCUDA, AutogradXLA, AutogradNestedTensor, UNKNOWN_TENSOR_TYPE_ID, AutogradPrivateUse1, AutogradPrivateUse2, AutogradPrivateUse3, Tracer, Autocast, Batched, VmapMode].

CPU: registered at D:\a\_work\1\s\build\aten\src\ATen\RegisterCPU.cpp:5926 [kernel]
SparseCPU: registered at D:\a\_work\1\s\build\aten\src\ATen\RegisterSparseCPU.cpp:558 [kernel]
BackendSelect: fallthrough registered at D:\a\_work\1\s\aten\src\ATen\core\BackendSelectFallbackKernel.cpp:3 [backend fallback]
Named: fallthrough registered at D:\a\_work\1\s\aten\src\ATen\core\NamedRegistrations.cpp:11 [kernel]
AutogradOther: registered at D:\a\_work\1\s\torch\csrc\autograd\generated\VariableType_1.cpp:9665 [autograd kernel]
AutogradCPU: registered at D:\a\_work\1\s\torch\csrc\autograd\generated\VariableType_1.cpp:9665 [autograd kernel]
AutogradCUDA: registered at D:\a\_work\1\s\torch\csrc\autograd\generated\VariableType_1.cpp:9665 [autograd kernel]
AutogradXLA: registered at D:\a\_work\1\s\torch\csrc\autograd\generated\VariableType_1.cpp:9665 [autograd kernel]
AutogradNestedTensor: registered at D:\a\_work\1\s\torch\csrc\autograd\generated\VariableType_1.cpp:9665 [autograd kernel]
UNKNOWN_TENSOR_TYPE_ID: registered at D:\a\_work\1\s\torch\csrc\autograd\generated\VariableType_1.cpp:9665 [autograd kernel]
AutogradPrivateUse1: registered at D:\a\_work\1\s\torch\csrc\autograd\generated\VariableType_1.cpp:9665 [autograd kernel]
AutogradPrivateUse2: registered at D:\a\_work\1\s\torch\csrc\autograd\generated\VariableType_1.cpp:9665 [autograd kernel]
AutogradPrivateUse3: registered at D:\a\_work\1\s\torch\csrc\autograd\generated\VariableType_1.cpp:9665 [autograd kernel]
Tracer: registered at D:\a\_work\1\s\torch\csrc\autograd\generated\TraceType_1.cpp:11324 [kernel]
Autocast: registered at D:\a\_work\1\s\aten\src\ATen\autocast_mode.cpp:254 [kernel]
Batched: registered at D:\a\_work\1\s\aten\src\ATen\BatchingRegistrations.cpp:1020 [kernel]
VmapMode: fallthrough registered at D:\a\_work\1\s\aten\src\ATen\VmapModeRegistrations.cpp:33 [backend fallback]

[foemre]

Are you receiving this error at a line similar towith torch.cuda.amp.autocast()? I'm receiving a similar error too.

[foemre]

Ok it seems that aten::bmm function is not implemented. In another issue #176 it is mentioned that some functions are not implemented yet.

[Adele101]

Thanks for your submission! We are actively adding new operator support to PyTorch-DML, so thank you for your patience and being one of our early users. Please visit the PyTorch-DirectML Operator Roadmap for guidance on current and future support.