Double precision issue using Python binding and GPU with double and single CUDA threads

[dschmitz89]

Hello,

we are using gpufit for a fitting problem where double and single precision make a difference. While at single precision we see a very good agreement with our CPU Python based results, changing to double precision yields wrong results. We have not found any bug in our code so far, so we are thinking it might be an issue of the mix of hardware and software

We are using a Pascal GPU (https://devblogs.nvidia.com/inside-pascal/) which has FP32 and FP64 CUDA threads. Since Gpufit handles the usage of blocks and threads, does it also automatically check the precision of the utilized CUDA thread? Running double precision calculations on a single precision thread might cause the errors we observed.

For the Matlab files an option for double precisiion support was added in the last update. For the python binding, I am not sure if it is correctly supported. Could you comment on this?

[jkfindeisen]

I'm not sure what you mean by option for double precision support in the Matlab files? I thought that one needs to switch between double and single arithmetic before building the CUDA-C sources.

Are the wrong results with double precision completely off or just slightly wrong?

The best is always a minimal example showing the issue. For example you could publicly fork Gpufit and add a model, then add a Matlab/Python example script with some minimal data sufficient to show the problem. Then everyone could test it on the available hardware.

[adrianjp88]

It's not necessary to specify, in the code, which cores (FP32/FP64) are used for the computation.
See CUDA FAQ:

Q: How do I get double precision floating point to work in my kernel?
You need to add the switch "-arch sm_13" (or a higher compute capability) to your nvcc command line...

The number 13 stands for the compute capability version. Since Gpufit supports compute capabilities 2.0 and higher, the kernels of Gpufit support double precision computation by default.

The Matlab, Python and Java bindings currently don't support double precision data.

[dschmitz89]

Thanks for your quick replies. We will check our code once more.
Our input data is single precision, we just need the optimization stopping criteria in double precision indeed.
If necessary, we will upload a MWE.

[superchromix]

I should add that, at this time, double precision is not officially supported in the Gpufit library. Although the framework exists in the C code, none of the bindings will work with double precision.

The option to compile Gpufit with double precision math was added for debugging purposes, and the code has not been fully tested in this mode. On many GPUs the code will run drastically slower with double precision floating point.

Feel free to tinker with the bindings to get double precision working for your application - this should not be too difficult. Let us know how it goes.

[jkfindeisen]

The state of double precision support hasn't changed since then. I added a few sentences in the documentation about it. If there is demand, an enhancement request could be opened up. The originally reported errors could not be reproduced because insufficient information was given to reproduce them. If however only the stopping criteria must be given in double precision then maybe a suitable rescaling of the estimator might instead solve the problem.