As bloc97 described in his pseudo-preprint, the Anime4K algorithm is actually quite simple.
I, however, will only give a brief overview of the algorithm. If you want to read more, I highly suggest bloc97's preprint.
The Algorithm itself can be summed up in five mayor steps, one of which beign the initial upscaling of the image.
This Step is all about scaling the image up, like you normally would.
Let's assume we start with the following image:
We just scale the Image up using a arbitrary Image interpolation algorithm (bloc97 suggests bicubic interpolation, which is also what I went with)
This is our Image after scaling it up using bicubic interpolation:
In this Step we create "Maps" from the input Image. These Maps contain information about the Image's Structure and are used in the following steps.
The Original (GLSL) implementation uses Texture buffers to hold the Data, my implementation just uses a secondary data image instead.
So, let's start with the first step:
Calculating the Luminance of the input image into a channel of our data image (I'll call this channel "Luminance" from now on).
The Luminance is just a grayscale version of the image and looks like this:
We now use the Luminance we just calculated to create a gradient map of the image. The gradient map is stored into another channel of our data image (I'll call this one "Gradient").
In the Original GLSL code, this step comes last. However, I figured I could put it earlyer so I can use my data image for temprary values. Also, this step only uses the Luminance anyway...
Now, let's see what our Gradient map looks like:
Cool, so what now? Well, we take our Luminance and run a Gauss Filter over it.
The result of this filter is then saved into another channel of our data Image. This one I'll call "Gaussed Luminance".
The Gaussed Luminance looks something like this, tho it's not really interesting since it's just a blurred version of our Luminance:
So far, so good. Now, Let's use the Luminance we calculated and our Gaussed Luminance to find where we have lines in our Image.
And let's save that information into another channel of our data image, and call it "Line Map".
This Line Map just contains, well, the lines in the image. Which we need since we want to enhance them... Quite straight- forward, right?
Let's just quickly take a look at the Line map (Left is original, Right is inverted):
Now, before we get to the spicy stuff, we just quickly have to run another Gauss Filter over the Line Map.
The result of this step just overrides our previous Line Map. I'll give it a unique name anyway, just for reference. I think "Gaussed Line Map" fits quite well.
This is how our Gaussed Line Map looks like (Again, Left is original, Right is inverted):
The next two steps may seem familiar if you already know how Anime4K 0.9 works - that is because they work almost the same.
In this step all we do is push colors in the image toward the edges. We use the the Luminance and Line Map we calculated previously for this step.
After doing this, we get a Image that looks nearly the same. But looking at the edges of lines and different colors reveals that they are lighter than in the original (this is especially noticeable at lines)
The following Image highlights the differences between the upscaled version of the image and the image after the color push step.
As you can see, the changes are mostly around the edges between colors and lines.
In this step, we refine the edges and lines in the Image. To do this, we use the Luminance, Gradient Map and Line Map we calculated previously.
This also gets rid of some noise that is caused by the interpolation.
If we were working with Anime4K 0.9, we'd be done now.
However, Anime4K 1.0+ introduced an additionaly FXAA step to help make the image look smoother (and better).
Sadly, I couldn't figure out how the FXAA actually works, so I cannot show you a Image of this step.
If you want to read more about the FXAA step, here is an explanation how it works (and how you can implement it).
Hold on - Before we save the Image, we have to dump the alpha channel of the image.
Otherwise, we'd get a output image that still contains the line map in its alpha channel, resulting in a image with transparent lines (which looks really weird).
The output of this step is our final output image (assuming we only do one pass):
To better show the differences, heres a zoomed- in comparison between bicubic interpolation and the final Anime4K image:
The Difference between the versions is how much they process the Image, resulting in RC2_Fast beign, well, faster than RC2.
Heres a Table that shows the Key Differences:
| RC2 | RC2_Fast | RC2_UltraFast | |
|---|---|---|---|
| Line Detect | 7 Px Gaussian | 5 Px Gaussian | No Gaussing |
| Compute Gradient | Uses Line Map | Uses Line Map | No Line Map |
| Thin Lines | Yes | No | No |
| FXAA | Yes | No | No |
All art assets used are for demonstration and educational purposes. All rights are reserved to their original owners. If you (as a person or a company) own the art and do not wish it to be associated with this project, please contact me (eg by opening a Github issue) and I will gladly take it down.
Images used for Demonstration are from Cells At Work.
- This Document is based on Anime4K 1.0 RC2 (FULL). Other (newer) Versions of Anime4K may work somewhat differently
- I wrote this Document almost two months after actively working on the Project. It may not be completely correct.
- I do, by no means claim to be an expert when it comes to Anime4K. (Tho I think this explanation should capture the overall principle quite well)
- Credits for Anime4K go to bloc97