Neural Network Usage

As of now, this installer pulls 2 major super resolution backends:

VSGAN

The VSGAN VapourSynth plugin feeds VapourSynth clips into trained PyTorch GAN models, like ESRGAN. Models will need to be manually trained or downloaded. In the VapourSynth script itself, the "path" argument of the VSGAN function should point towards the directory of the ,pth file. For detailed usage, see:

• https://github.com/imPRAGMA/VSGAN/wiki
• https://forum.doom9.org/showthread.php?t=176961

The GameUpscale Wiki hosts quite a few specific and general purpose models, and info on training models yourself:

• https://upscale.wiki/wiki/Model_Database

You can take a peek at models others are working on in the GameUpscale Discord:

• https://discord.gg/xS39yR8

Note that VSGAN will run on the CPU, but it's very slow. The CPU only version will be installed with AMD and Intel graphics cards. Also note that, as of the time of this writing, all input will be converted to 8 bits before processing.

MXNet

WolframRhodium's Super-Resolution-Zoo repo has a huge assortment of pre-trained MXNet models, ranging from relatively fast ones like IDN, to high quality ones like ESRGAN and ARAN, to models focused on specific artifacts or use cases. There's also a function in the muvsfunc script designed to work with these models. Instead of downloading the massive superres repo and copying arguments into your VapourSynth scripts by hand, there's a script to automate the process:

• Double click _select_neural_networks.bat
• A table with the available neural network categories will appear. Pick one.
• SVN will selectively pull that whole category, if necessary, and you'll be prompted to pick a model.
• All .vpy scripts in the "VapourSynthScripts" folder with a name ending in "Auto" will have any line starting with "sr_args=" line changed the appropriate arguments for that model.
• Custom arguments (like block size, number of GPUs, etc) can be overwritten like so: manual_sr_args = sr_args manual_sr_args['block_w']=64
• All those arguments can then be fed into the super_resolution function: muf.super_resolution(clip, **manual_sr_args)

For detailed usage, examples, and support, see:

• https://github.com/WolframRhodium/muvsfunc/blob/master7/muvsfunc.py#L4329
• https://github.com/WolframRhodium/muvsfunc/blob/master/Collections/examples/super_resolution_mxnet.vpy
• https://forum.doom9.org/showthread.php?t=175721

Note that muf.super_resolution converts everything to FP32. You can change it back by running something like this after the filter: 'clip = mvs.Depth(clip, depth=16)'

Other Methods

WolframRhodium also has an OpenCV (and possibly AMD compatible?) example that can be modified to work with other frameworks: "

• https://github.com/WolframRhodium/muvsfunc/blob/master/Collections/examples/super_resolution_opencv.vpy

There's also a numpy variant (that's already imported with this repo):

• https://github.com/WolframRhodium/muvsfunc/blob/master/Collections/muvsfunc_numpy.py

Currently, OpenCV's loader doesn't seem to support PyTorch models, but I've successfully run Caffe models on AMD GPUs.

The VapourSynth Fatpack comes with an OpenCL based, Waifu2X scaling filter:

• https://github.com/HomeOfVapourSynthEvolution/VapourSynth-Waifu2x-w2xc

However, quality and speed are not very good. There's also a faster Caffe version, but either should only be used as a last resort.