Possibility of future migration to Vulkan API

[0xE1]

While CUDA is king in field of GPU Computations,
their licensing is very strict and expensive when used on non-workstation environment.

Any insight on possibility of migrating to Vulkan API for hardware agnosticism?

AMD Radeon have gained a lot of ground lately (and they have favorable licensing), and Intel is going to join the Dedicated GPU field in near future.

[XJDHDR]

Vulkan is not a compute API so DAIN can't use it at all. OpenCL is the alternative to CUDA. As for why DAIN's devs went for CUDA instead of OpenCL, maybe one of them will elaborate.

Maybe AMD has gained a lot of ground lately but Nvidia still has a large marketshare lead over them in gaming PCs at least: https://store.steampowered.com/hwsurvey
Intel is welcome to join the dedicated GPU field but whether they will successfully do so is another question entirely. This isn't the first time they tried to build dedicated GPUs.

[0xE1]

Vulkan is advertised as both Graphic and Compute API,
while OpenCL is an alternative to CUDA, Vulkan while not a direct alternative - is supported by AMD, NVidia and Intel, NVidia even have better support for it than for OpenCL.
There are handful of articles on using it.

[Sam2much96]

Can we port it to vulkan ourselves? I really need this software for automated animation

[XJDHDR]

@Inhumanity96
DAIN is licensed under the MIT so you're welcome to try. That depends entirely on whether Vulkan has the features that DAIN needs though.

@0xE1
Despite being marketed as a compute API, Vulkan has, from what I've read, a very limited subset of one aimed at rendering work, which I highly doubt is what DAIN is doing with CUDA. If this limited API doesn't do what DAIN needs, it doesn't matter how well supported Vulkan is.

Indeed, there is no reason for the Khronos group to release OpenCL 3 if Vulkan is an equivalent compute API and already available. In fact, that begs the question of why go for CUDA in the first place is OpenCL is already available and does what DAIN needs.

[paulovilli]

Vulkan is/will be the most ubiquitous graphics API (save perharps yet for OpenGL) and is available by default on AMD, NVidia and Intel graphics drivers on Windows and Linux (plus possible over Apple's Metal via MoltenVK, available on newer versions of Android and even other systems), so from on-user-machine gaming perspective, porting this to Vulkan would make the most sense for single-source best availability.

OpenCL on the other hand is not as easily available and is going through some rough patches. It can't be reasonably assumed as available for all end-users like Vulkan and it might be hard to ensure all needed features are always available for a broad target audience (more so now that OpenCL 3 specification was released reducing core functions).

That being said, OpenCL over Vulkan exists so porting to OpenCL and shipping with the necessary compatibility layer might be enough.

Also to consider, there are 3rd-party efforts to provide CUDA over Vulkan compatibility layers (eg: VUDA) and Nvidia is gradually making it more open, so likely soon there might be more development in that direction, thus possibly eliminating the need to actually port this (unless performance or licensing issues require it).

references:
https://www.anandtech.com/show/15746/opencl-30-announced-hitting-reset-on-compute-frameworks
https://www.phoronix.com/scan.php?page=news_item&px=VUDA-Vulkan-CUDA
https://www.anandtech.com/show/5238/nvidia-releases-cuda-41-cuda-goes-llvm-and-open-source-kind-of

[Johnreidsilver]

Starting from CUDA the easiest path would be to convert to HIP that can export back to native CUDA and to OpenCL/AMD(ROCm) and perhaps that should eventually work in all Vulkan systems?

https://rocmdocs.amd.com/en/latest/Programming_Guides/HIP-porting-guide.html#hip-porting-guide

[nihui]

The in-development DAIN vulkan implementation

https://github.com/nihui/dain-ncnn-vulkan

[paulovilli]

this is an excelent development, @nihui! thanks for sharing it here! Em ter, 25 de ago de 2020 12:50, nihui <notifications@github.com> escreveu:

…

The in-development DAIN vulkan implementation https://github.com/nihui/dain-ncnn-vulkan — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#77 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGS6JGDB4LCLWJ65MH252GTSCPMTRANCNFSM4NAV4RVA> .

[XJDHDR]

Vulkan is/will be the most ubiquitous graphics API

As I've repeatedly said, DAIN doesn't need a graphics API. It needs a compute API. If Vulkan doesn't provide the necessary compute functionality, it doesn't matter how popular it is or will be. And I highly doubt that it does. It makes no sense for the Khronos group to maintain two competing compute APIs so the most sensible conclusion that they are maintaining both OpenCL and Vulkan because each doesn't have the other's functionality.

available on newer versions of Android and even other systems), so from on-user-machine gaming perspective, porting this to Vulkan would make the most sense for single-source best availability.

Do you truly believe that it makes sense to run DAIN on Android phones and tablets, with significantly less power than PCs and servers?

OpenCL on the other hand is not as easily available and is going through some rough patches. It can't be reasonably assumed as available for all end-users like Vulkan and it might be hard to ensure all needed features are always available for a broad target audience (more so now that OpenCL 3 specification was released reducing core functions).

I recommend re-reading Anandtech's writeup. OpenCL 3 does not have reduced core functionality. All of the functionality of previous versions are present. The only change in this regard is that the 2.x functions are now optional, for good reason as no one wanted to implement them when compulsory. Making them compulsory for 3 would have only repeated the problem.

That being said, OpenCL over Vulkan exists

OpenCL does not exist over Vulkan but alongside it.

Also to consider, there are 3rd-party efforts to provide CUDA over Vulkan compatibility layers (eg: VUDA)

Please re-read Phoronix's article. VUDA is not a compatibility layer. It is a library that allows devs that are writing Vulkan programs to write them in CUDA syntax. You still need to target the Vulkan API to make it work in the first place.

The in-development DAIN vulkan implementation

Good luck with your project but is using ncnn really a good idea? It appears to be a library whose target audience is mainly neural network apps on smartphones and tablets. The supported platform matrix section indicates that it's mainly designed to run on ARM CPUs whereas x86 CPUs and all GPUs are not optimised. Since Vulkan also only appears to be tacked on to accelerate select functions with most of the work done by the CPU, to say that it will make DAIN use Vulkan is stretching things.

[nihui]

Beta prebuild binary is available via https://github.com/nihui/dain-ncnn-vulkan/actions ci artifact
It should work on any graphics card supporting vulkan api on windows/linux/macos
Input two adjacent video frame images, and generate an interpolated frame at any time between.

If you have any questions, you can send an issue on my project 😃

[lextra2]

Beta prebuild binary is available via https://github.com/nihui/dain-ncnn-vulkan/actions ci artifact
It should work on any graphics card supporting vulkan api on windows/linux/macos
Input two adjacent video frame images, and generate an interpolated frame at any time between.

If you have any questions, you can send an issue on my project 😃

@nihui
Thank you for your amazing work, as always. I had hoped for you to pick this up. I've been using your waifu2x version for well over a year now (yours is by far the best available version of that tool, and it works on any platform!!). I also started to use the 2 other, newer upscaling tools of yours.