Tinting towards blue/purple

[karrirasinmaki]

Hey!

First of all, awesome work with the library.

I've been colorizing some old movie clips from the golden swing era and there's one issue commonly seen between clips. The model seems to tint colors towards blue/purple, especially in the case of soldier uniforms. (Seems to be the same issue as reported in #38.)

Attached there are two images of the same frame, one with video model another with artistic model.

Original and with the artistic model.

With the video model.

Ps. Another commonly found glitch is the floor tends to have green tinting in it.

[karrirasinmaki]

I wasn't sure if this was the right place to bring up these issues, but I thought might be useful.

[jantic]

@karrirasinmaki Oh yeah, this set of color issues is the bane of my existence. It's slowly getting better with the newer models, but it's a difficult issue. The issue with blue/purple bias really boils down to this: Things like clothing and cars can be pretty much any color, so the model takes a bet on blue/purple because on average they're not bad bets. It's basically the model cheating the exam.

I'll keep this open just to let people know it's known, but I can tell you this much: It'll probably take quite a long time to completely get rid of this problem.

[karrirasinmaki]

@jantic Thank you for the response! The AI is clearly getting clever, learning how to cheat the test.

I wish I'd have the knowledge and skills to help. Best of luck!

[ysonggit]

Indeed, I find this phenomena as well. A typical example is Audrey Hepburn's cream color dress in Roman Holiday:

Original B&W	Colorized

The ground truth is:

Just bring it up for further optimization.

I can not compliment your work any more, this is a sensational application that will reprint the historical footages of the world!

[linuxthor]

I have had this issue too and understand why it happens but in this instance I would suggest two things that (in my opinion) can slightly improve the result:

1. Adjust the levels in the source image first
2. Experiment with a range of different 'render factor' settings

By adjusting the levels first and picking a lower 'render factor' the result is still objectively coloured 'wrong' but I feel like it moves a bit more in the right direction and has a slightly more lifelike feel.

[5000Stadia]

With my limited understanding of what's going on besides the explanations of cheating the test I have an assumption and possible suggestion to possibly have some hands on to direct the cheating.

Assumption: It seems like purple/blue is a frequent guess to cheat the test, but there seems to be multiple guess hues going on, one example is the green floor. I assume theres a level of certainty going on here, grass and trees are often green, good guess, high certainty. Color of dress? Who knows.

Suggestion: is there a way similar to the render factor that we can tweak the hue on the low certainly color choices? For example if in seeing the playback it looks like purple guesses should be more blue, you could dial the blue in the low certainty purple choices.

I know easier said then done but thought I'd share the idea. It moves farther away from the ai model but at some point, unless there's color sources of near same images arn't we going to need some user input?

[jantic]

@theman8631 I definitely agree with the general idea that the way forward for much better results is to take the strengths of fully automated colorization and marry that with human creativity and intelligence. I think there's a bit of a diminishing returns on fully automated at this point as to how far you can go with that due to the nature of the problem and the limits of current tech.

That being said I only differ on what I think would be the best way to do this- I think just having the users directly suggest the colors with a dot or scribble on the object would be more intuitive and productive. This is something I plan on exploring pretty soon, but it won't be going into this current project. Rather it'll be in the commercial work for my startup DeOldify.

[5000Stadia]

Well sign me up for whatever that software ends up becoming!

…

On Thu, Mar 11, 2021 at 2:18 PM Jason Antic ***@***.***> wrote: @theman8631 <https://github.com/theman8631> I definitely agree with the general idea that the way forward for much better results is to take the strengths of fully automated colorization and marry that with human creativity and intelligence. I think there's a bit of a diminishing returns on fully automated at this point as to how far you can go with that due to the nature of the problem and the limits of current tech. That being said I only differ on what I think would be the best way to do this- I think just having the users directly suggest the colors with a dot or scribble on the object would be more intuitive and productive. This is something I plan on exploring pretty soon, but it won't be going into this current project. Rather it'll be in the commercial work for my startup DeOldify. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#239 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFQO4FGQ73EBH5QOXK5KBRLTDEXSJANCNFSM4NGUAEDQ> .

[jantic]

Going to close this. The problem is acknowledged but there's no plans to improve upon this in this repo.