Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better computed alpha #1942

Merged

Conversation

RunDevelopment
Copy link
Member

This PR makes a minor improvement to convenient_upscale (and therefor all upscaling nodes).

When computing alpha using the black and white image upscales, we get 3 channels that all represent the computed alpha. We previously simply took the mean of those 3 and called it a day, but we can do better. By using the mean to interpolate between the min and max of those 3 channels, we get a lot better results near 0 and 1.

Why? If we model AI upscaling as simply adding noise to the true high-res alpha, then it makes sense that the mean of 3 noisy versions of the same image will never give us true 0 or 1. The mean will always be slightly above 0 and slightly below 1. By using the mean to interpolate between the min and max, we can get much closer to true 0 and 1.
However, the tradeoff is that we also get more noise. Since fluctuations in min and max will now affect the output a lot, we will get more noise in the final alpha. This noise is of course bounded by the amount of noise the upscaler produces, so it's not too bad in practice.

Examples

Here are some non-cherry-picked examples. The images were DDS compressed and contained either binary alpha or full alpha with clearly defined edges. They were upscaled 4x using either UltraSharp or AnimeSharp. The output alpha has been multiplied by 16x to make the noise more visible. I also zoomed in 2x.

(Open the images in a new tab and switch between them to see the differences.)

Old (np.mean) New
image image
image image
image image

As we can see, the new method produces generally sharper results with a lot less haloing around edges.

Copy link
Member

@joeyballentine joeyballentine left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like an improvement to me. Good idea

@joeyballentine joeyballentine merged commit f6316f6 into chaiNNer-org:main Jul 16, 2023
10 checks passed
@RunDevelopment RunDevelopment deleted the better-cumputed-alpha branch July 16, 2023 17:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants