Tests fail with float16 on OpenCL while they should work.

[ghisvail]

Same context as in #491, but with float16 compute support. Running the tests on pocl produces the following errors:

======================================================================
ERROR: pygpu.tests.test_elemwise.test_elemwise_f16(<built-in function neg>, 'float16')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/usr/lib/python2.7/dist-packages/pygpu/tests/support.py", line 44, in f
    func(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/pygpu/tests/test_elemwise.py", line 46, in elemwise1_ops_array
    out_g = op(g)
  File "/usr/lib/python2.7/dist-packages/pygpu/_array.py", line 184, in __neg__
    return elemwise1(self, '-')
  File "/usr/lib/python2.7/dist-packages/pygpu/elemwise.py", line 35, in elemwise1
    k = GpuElemwise(a.context, oper, args, convert_f16=convert_f16)
  File "pygpu/_elemwise.pyx", line 143, in pygpu._elemwise.GpuElemwise.__cinit__ (pygpu/_elemwise.c:3140)
GpuArrayException: Could not initialize C GpuElemwise instance

======================================================================
ERROR: pygpu.tests.test_elemwise.test_elemwise_f16(<built-in function add>, 'float16', 'float16', (50,))
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/usr/lib/python2.7/dist-packages/pygpu/tests/support.py", line 44, in f
    func(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/pygpu/tests/test_elemwise.py", line 126, in elemwise2_ops_array
    out_g = op(ag, bg)
  File "/usr/lib/python2.7/dist-packages/pygpu/_array.py", line 24, in __add__
    return elemwise2(self, '+', other, self, broadcast=True)
  File "/usr/lib/python2.7/dist-packages/pygpu/elemwise.py", line 76, in elemwise2
    k = GpuElemwise(ary.context, oper, args, convert_f16=convert_f16)
  File "pygpu/_elemwise.pyx", line 143, in pygpu._elemwise.GpuElemwise.__cinit__ (pygpu/_elemwise.c:3140)
GpuArrayException: Could not initialize C GpuElemwise instance

======================================================================
ERROR: pygpu.tests.test_elemwise.test_elemwise_f16(<built-in function iadd>, 'float16', 'float16', (50,))
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/usr/lib/python2.7/dist-packages/pygpu/tests/support.py", line 44, in f
    func(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/pygpu/tests/test_elemwise.py", line 150, in ielemwise2_ops_array
    out_g = op(ag, bg)
  File "/usr/lib/python2.7/dist-packages/pygpu/_array.py", line 30, in __iadd__
    return ielemwise2(self, '+', other, broadcast=True)
  File "/usr/lib/python2.7/dist-packages/pygpu/elemwise.py", line 94, in ielemwise2
    k = GpuElemwise(a.context, oper, args, convert_f16=convert_f16)
  File "pygpu/_elemwise.pyx", line 143, in pygpu._elemwise.GpuElemwise.__cinit__ (pygpu/_elemwise.c:3140)
GpuArrayException: Could not initialize C GpuElemwise instance

Again, perhaps pytest skipif could help here.

[ghisvail]

Logs of the Debian CI are available here

[ghisvail]

Is there a way to detect whether the current device is float16 or CUDA capable within pygpu?

[nouiz]

We only support well one type of float16 called float16 storage. That mean we store on the GPU RAM on float16, but the computation is still done in float32. So all CUDA capabable GPU support it. float16 computation aren't well supported except via the cudnn library wrapper in Theano. NVIDIA don't recommand to use it. It is hard to use and they come up with mixed precission in Volta that will be easier to use as it loose less precission.

…

On Thu, Aug 17, 2017 at 1:24 PM Ghislain Antony Vaillant < ***@***.***> wrote: Is there a way to detect whether the current device is float16 or CUDA capable within pygpu? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#492 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AALC-70YPAphzr6xPC2zaFug9-Yj2mjrks5sZHc7gaJpZM4OlCvz> .

[ghisvail]

So all CUDA capabable GPU support it.

Ok, so that factors to detecting whether the current device is CUDA capable. The reason I am asking is because, assuming such function to detect CUDA capability exists in pygpu, then it could be used together with unittest.skipif for skipping the failing tests on the Debian CI (which uses pocl).

[nouiz]

I don't know if float16 storage is supported in OpenCL or not. I'll let @abergeron answer that. But sadly we don't have time for better OpenCL support. Contribuation are always welcome.

…

On Thu, Aug 17, 2017 at 2:06 PM Ghislain Antony Vaillant < ***@***.***> wrote: So all CUDA capabable GPU support it. Ok, so that factors to detecting whether the current device is CUDA capable. The reason I am asking is because, assuming such function to detect CUDA capability exists in pygpu, then it could be used together with unittest.skipif for skipping the failing tests on the Debian CI (which uses pocl). — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#492 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AALC-97tQpQNsjAMtcxy_MfqAVBxbHnjks5sZIEvgaJpZM4OlCvz> .

[ghisvail]

I don't know if float16 storage is supported in OpenCL or not.

Hmmm, that's not what I am asking. Is there a way to call some function in pygpu which returns whether the device is CUDA capable? Forget about OpenCL.

[abergeron]

As far as I know float16 storage is part of the core requirements for OpenCL since 1.0.

The problem you are getting here is not because the device doesn't support float16, but most probably because there was a compilation error in the kernel, which is a problem that we have to fix.

You can try a Debug build to get more information out of this error (Including a source dump and the compiler error log).