Compatible with JuliaGPU

[bdas123]

Hello!

Is Nonconvex.jl compatible with JuliaGPU to speed up the optimization process?

[mohamed82008]

Most optimisation algorithms do not use the GPU. This is not a limitation of Nonconvex but a limitation of the algorithm packages. If you know a package that implements an algorithm taking advantage of the GPU, please open an issue and I can try to wrap it.

[mohamed82008]

Even if the algorithm doesn't support GPUs, the more common use of GPUs in optimization is in the evaluation of the objective or constraint functions and their gradients. This can be written using GPU operations. Since we use Zygote by default, functions using the GPU arrays should differentiate just fine with autodiff in some cases. In other cases, you may need to define your own gradient function and pass it to Nonconvex. Again, nothing is stopping you from writing your own functions and gradients which use the JuliaGPU ecosystem and pass the functions over to Nonconvex for optimization. In this case, the optimization algorithm will be running on the CPU trying to figure out the next point x to evaluate the objective and constraInts at, but the objective and constraint evaluation will be taking advantage of the GPU. Depending on your problem, the GPU can be utilised in different ways.

[mohamed82008]

I am closing this issue since it is not really actionable. If you have a specific case that you want to use the GPU for, I will be happy to help with that. In that case, please open another issue.