Color artefacts

[rolkar]

In this DPReview post some artefacts are evident.

http://www.dpreview.com/forums/post/56218184

[erikrk]

Looks like it could be ringing caused by bicubic interpolation. Maybe we
could do bilinear instead, or would that compromise resolution too much?
Remember that we are only dealing with chominance resolution. Luminance
resolution is not affected by this interpolation. Any better suggestions
when it comes to interpolation?

Do we have X3F for this or some other Quattro X3F with serious fringing
issues?

On vie, 2015-07-31 at 04:23 -0700, Roland Karlsson wrote:

In this DPReview post some artefacts are evident.

http://www.dpreview.com/forums/post/56218184

—
Reply to this email directly or view it on GitHub.

[erikrk]

Another thought, maybe resampling should not be done at the YUV level.
Maybe it should be converted back before resampling and then converted
to YUV again? Would that matter when it comes to colored ringing?

On vie, 2015-07-31 at 17:58 +0200, Erik Karlsson wrote:

Looks like it could be ringing caused by bicubic interpolation. Maybe we
could do bilinear instead, or would that compromise resolution too much?
Remember that we are only dealing with chominance resolution. Luminance
resolution is not affected by this interpolation. Any better suggestions
when it comes to interpolation?

Do we have X3F for this or some other Quattro X3F with serious fringing
issues?

On vie, 2015-07-31 at 04:23 -0700, Roland Karlsson wrote:

In this DPReview post some artefacts are evident.

http://www.dpreview.com/forums/post/56218184

—
Reply to this email directly or view it on GitHub.

[rolkar]

How about a less cosher upscaling?

My idea is to do all computations at 5 Mpixel and then
upscale by copying every pixel to 2x2 values and then
use the topmost layer to modify the four pixels in all
three layers.

/Roland

[erikrk]

You mean nearest neighbor?

On vie, 2015-07-31 at 09:55 -0700, Roland Karlsson wrote:

How about a less cosher upscaling?

My idea is to do all computations at 5 Mpixel and then
upscale by copying every pixel to 2x2 values and then
use the topmost layer to modify the four pixels in all
three layers.

/Roland

—
Reply to this email directly or view it on GitHub.

[erikrk]

I am doing some experimentation with various interpolation methods and dp2q_SDIM0003.X3F, but the differences are not very obvious. I would need an X3F file with more obvious fringing issues.

[rolkar]

Yes ... the first 5 to 20 MP upscaling is nearest neighbor.
But, then you use the topmost layer to modify the result.
The assumption being that all three layers have the same
micro contrast.

/Roland

On 2015-07-31 18:58, Erik Karlsson wrote:

You mean nearest neighbor?

On vie, 2015-07-31 at 09:55 -0700, Roland Karlsson wrote:

How about a less cosher upscaling?

My idea is to do all computations at 5 Mpixel and then
upscale by copying every pixel to 2x2 values and then
use the topmost layer to modify the four pixels in all
three layers.

/Roland

—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub
#67 (comment).

[erikrk]

Our Quattro expansion has another edge-related problem that is apparent in dp2q_SDIM0003.X3F. At the edge between the light blue sky and the dark gray building, there is a dark blue line. This is obviously a combination of the blue color of the sky and the darkness of the building. Changing the interpolation to bilinear (maybe necessary to get rid of fringing) unsurprisingly worsens this problem. Nearest neighbor makes it even worse.

SPP does not seem to have this problem at all. Neither does it have the fringing problem. Some trickery going on? Edge detection maybe?

[erikrk]

I have tested converting back from YUV to BMT before resampling too. There is no visible difference.

[erikrk]

As I mentioned, nearest neighbor does not seem to work very well at all.
The problem is that if the luminance gradient is steeper than the
chrominance gradient you will get color bleeding at edges. Interpolation
algorithms that generate steep gradients unfortunately tend to overshoot
and and ring though. You essentially have to choose between bleeding and
fringing. SPP does neither. How is that possible?

On vie, 2015-07-31 at 10:44 -0700, Roland Karlsson wrote:

Yes ... the first 5 to 20 MP upscaling is nearest neighbor.
But, then you use the topmost layer to modify the result.
The assumption being that all three layers have the same
micro contrast.

/Roland

On 2015-07-31 18:58, Erik Karlsson wrote:

You mean nearest neighbor?

On vie, 2015-07-31 at 09:55 -0700, Roland Karlsson wrote:

How about a less cosher upscaling?

My idea is to do all computations at 5 Mpixel and then
upscale by copying every pixel to 2x2 values and then
use the topmost layer to modify the four pixels in all
three layers.

/Roland

—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub
#67 (comment).

—
Reply to this email directly or view it on GitHub.

[rolkar]

I assume they have a pattern based algorithm.
It is all about finding patterns for 4 pixels and their
surrounding. It is possible to tabulate all alternatives
or to find some simple rules.

In our case. It is not likely that a dark gray area has a one
pixel dark blue border if the next pixel is a part of a light
blue area. It is possible, but not likely. And - if that dark
blue border disappears is much more agreeable than the
upscaling inventing one.

Hmmmm ... this might explain why Quattro seems so sharp.
All edges are generated by using some nice patterns.

/Roland

On 2015-07-31 20:11, Erik Karlsson wrote:

As I mentioned, nearest neighbor does not seem to work very well at all.
The problem is that if the luminance gradient is steeper than the
chrominance gradient you will get color bleeding at edges. Interpolation
algorithms that generate steep gradients unfortunately tend to overshoot
and and ring though. You essentially have to choose between bleeding and
fringing. SPP does neither. How is that possible?

On vie, 2015-07-31 at 10:44 -0700, Roland Karlsson wrote:

Yes ... the first 5 to 20 MP upscaling is nearest neighbor.
But, then you use the topmost layer to modify the result.
The assumption being that all three layers have the same
micro contrast.

/Roland

On 2015-07-31 18:58, Erik Karlsson wrote:

You mean nearest neighbor?

On vie, 2015-07-31 at 09:55 -0700, Roland Karlsson wrote:

How about a less cosher upscaling?

My idea is to do all computations at 5 Mpixel and then
upscale by copying every pixel to 2x2 values and then
use the topmost layer to modify the four pixels in all
three layers.

/Roland

—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub
#67 (comment).

—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub
#67 (comment).

[erikrk]

In any case, our current approach of simply upsampling the chrominance
and combining it with the luminance does not seem to perform very well
at all at edges. I don't think this can be remedied just by
interpolating better. The luminance most probably has to be taken into
account when the chrominance is upsampled. If there is a sharp edge in
the luminance, the edge in the chrominance should be in the same place
and just as sharp.

On vie, 2015-07-31 at 11:48 -0700, Roland Karlsson wrote:

I assume they have a pattern based algorithm.
It is all about finding patterns for 4 pixels and their
surrounding. It is possible to tabulate all alternatives
or to find some simple rules.

In our case. It is not likely that a dark gray area has a one
pixel dark blue border if the next pixel is a part of a light
blue area. It is possible, but not likely. And - if that dark
blue border disappears is much more agreeable than the
upscaling inventing one.

Hmmmm ... this might explain why Quattro seems so sharp.
All edges are generated by using some nice patterns.

/Roland

On 2015-07-31 20:11, Erik Karlsson wrote:

As I mentioned, nearest neighbor does not seem to work very well at
all.
The problem is that if the luminance gradient is steeper than the
chrominance gradient you will get color bleeding at edges.
Interpolation
algorithms that generate steep gradients unfortunately tend to
overshoot
and and ring though. You essentially have to choose between bleeding
and
fringing. SPP does neither. How is that possible?

On vie, 2015-07-31 at 10:44 -0700, Roland Karlsson wrote:

Yes ... the first 5 to 20 MP upscaling is nearest neighbor.
But, then you use the topmost layer to modify the result.
The assumption being that all three layers have the same
micro contrast.

/Roland

On 2015-07-31 18:58, Erik Karlsson wrote:

You mean nearest neighbor?

On vie, 2015-07-31 at 09:55 -0700, Roland Karlsson wrote:

How about a less cosher upscaling?

My idea is to do all computations at 5 Mpixel and then
upscale by copying every pixel to 2x2 values and then
use the topmost layer to modify the four pixels in all
three layers.

/Roland

—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub

#67 (comment).

—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub
#67 (comment).

—
Reply to this email directly or view it on GitHub.

[mmroden]

What about a variant of nearest neighbor, where the pixels just use the
same information as it would if the pixels were all the same size? That
is, for a given pixel, use the exact value of the layers at each point, no
interpolation.

On Friday, July 31, 2015, Erik Karlsson notifications@github.com wrote:

In any case, our current approach of simply upsampling the chrominance
and combining it with the luminance does not seem to perform very well
at all at edges. I don't think this can be remedied just by
interpolating better. The luminance most probably has to be taken into
account when the chrominance is upsampled. If there is a sharp edge in
the luminance, the edge in the chrominance should be in the same place
and just as sharp.

On vie, 2015-07-31 at 11:48 -0700, Roland Karlsson wrote:

I assume they have a pattern based algorithm.
It is all about finding patterns for 4 pixels and their
surrounding. It is possible to tabulate all alternatives
or to find some simple rules.

In our case. It is not likely that a dark gray area has a one
pixel dark blue border if the next pixel is a part of a light
blue area. It is possible, but not likely. And - if that dark
blue border disappears is much more agreeable than the
upscaling inventing one.

Hmmmm ... this might explain why Quattro seems so sharp.
All edges are generated by using some nice patterns.

/Roland

On 2015-07-31 20:11, Erik Karlsson wrote:

As I mentioned, nearest neighbor does not seem to work very well at
all.
The problem is that if the luminance gradient is steeper than the
chrominance gradient you will get color bleeding at edges.
Interpolation
algorithms that generate steep gradients unfortunately tend to
overshoot
and and ring though. You essentially have to choose between bleeding
and
fringing. SPP does neither. How is that possible?

On vie, 2015-07-31 at 10:44 -0700, Roland Karlsson wrote:

Yes ... the first 5 to 20 MP upscaling is nearest neighbor.
But, then you use the topmost layer to modify the result.
The assumption being that all three layers have the same
micro contrast.

/Roland

On 2015-07-31 18:58, Erik Karlsson wrote:

You mean nearest neighbor?

On vie, 2015-07-31 at 09:55 -0700, Roland Karlsson wrote:

How about a less cosher upscaling?

My idea is to do all computations at 5 Mpixel and then
upscale by copying every pixel to 2x2 values and then
use the topmost layer to modify the four pixels in all
three layers.

/Roland

—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub

#67 (comment).

—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub
#67 (comment).

—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub
#67 (comment).

[erikrk]

I suppose you mean replicating each chroma pixel as a 2x2 square and
then just adding the top layer as luminance. The problem with this is
that all 4 pixels will have the same color but they may have very
different luminances. This is exactly what causes the mentioned problem
where a dark blue line appears between a dark gray area and a light blue
area. This looks pretty ugly and it is not present in SPP output. Using
bicubic interpolation reduces this problem, but it's still there to some
extent, and you also get ringing.

On vie, 2015-07-31 at 12:28 -0700, mmroden wrote:

What about a variant of nearest neighbor, where the pixels just use
the
same information as it would if the pixels were all the same size?
That
is, for a given pixel, use the exact value of the layers at each
point, no
interpolation.

On Friday, July 31, 2015, Erik Karlsson notifications@github.com
wrote:

In any case, our current approach of simply upsampling the
chrominance
and combining it with the luminance does not seem to perform very
well
at all at edges. I don't think this can be remedied just by
interpolating better. The luminance most probably has to be taken
into
account when the chrominance is upsampled. If there is a sharp edge
in
the luminance, the edge in the chrominance should be in the same
place
and just as sharp.

On vie, 2015-07-31 at 11:48 -0700, Roland Karlsson wrote:

I assume they have a pattern based algorithm.
It is all about finding patterns for 4 pixels and their
surrounding. It is possible to tabulate all alternatives
or to find some simple rules.

In our case. It is not likely that a dark gray area has a one
pixel dark blue border if the next pixel is a part of a light
blue area. It is possible, but not likely. And - if that dark
blue border disappears is much more agreeable than the
upscaling inventing one.

Hmmmm ... this might explain why Quattro seems so sharp.
All edges are generated by using some nice patterns.

/Roland

On 2015-07-31 20:11, Erik Karlsson wrote:

As I mentioned, nearest neighbor does not seem to work very well
at
all.
The problem is that if the luminance gradient is steeper than
the
chrominance gradient you will get color bleeding at edges.
Interpolation
algorithms that generate steep gradients unfortunately tend to
overshoot
and and ring though. You essentially have to choose between
bleeding
and
fringing. SPP does neither. How is that possible?

On vie, 2015-07-31 at 10:44 -0700, Roland Karlsson wrote:

Yes ... the first 5 to 20 MP upscaling is nearest neighbor.
But, then you use the topmost layer to modify the result.
The assumption being that all three layers have the same
micro contrast.

/Roland

On 2015-07-31 18:58, Erik Karlsson wrote:

You mean nearest neighbor?

On vie, 2015-07-31 at 09:55 -0700, Roland Karlsson wrote:

How about a less cosher upscaling?

My idea is to do all computations at 5 Mpixel and then
upscale by copying every pixel to 2x2 values and then
use the topmost layer to modify the four pixels in all
three layers.

/Roland

—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub

#67 (comment).

—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub

#67 (comment).

—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub
#67 (comment).

—
Reply to this email directly or view it on GitHub.

[mmroden]

We could experiment with wackiness-- a few different methods, then choose
the median value. Or we could do an edge detection algorithm, and then a
weighted average of one method vs another near strong luminance edges
(something like (edge)(cubic interp) + (1-edge)(nearest neighbor)) to try
to control ringing near edges, assuming ringing looks different in the
different methods.

On Friday, July 31, 2015, Erik Karlsson notifications@github.com wrote:

I suppose you mean replicating each chroma pixel as a 2x2 square and
then just adding the top layer as luminance. The problem with this is
that all 4 pixels will have the same color but they may have very
different luminances. This is exactly what causes the mentioned problem
where a dark blue line appears between a dark gray area and a light blue
area. This looks pretty ugly and it is not present in SPP output. Using
bicubic interpolation reduces this problem, but it's still there to some
extent, and you also get ringing.

On vie, 2015-07-31 at 12:28 -0700, mmroden wrote:

What about a variant of nearest neighbor, where the pixels just use
the
same information as it would if the pixels were all the same size?
That
is, for a given pixel, use the exact value of the layers at each
point, no
interpolation.

On Friday, July 31, 2015, Erik Karlsson <notifications@github.com
javascript:_e(%7B%7D,'cvml','notifications@github.com');>
wrote:

In any case, our current approach of simply upsampling the
chrominance
and combining it with the luminance does not seem to perform very
well
at all at edges. I don't think this can be remedied just by
interpolating better. The luminance most probably has to be taken
into
account when the chrominance is upsampled. If there is a sharp edge
in
the luminance, the edge in the chrominance should be in the same
place
and just as sharp.

On vie, 2015-07-31 at 11:48 -0700, Roland Karlsson wrote:

I assume they have a pattern based algorithm.
It is all about finding patterns for 4 pixels and their
surrounding. It is possible to tabulate all alternatives
or to find some simple rules.

In our case. It is not likely that a dark gray area has a one
pixel dark blue border if the next pixel is a part of a light
blue area. It is possible, but not likely. And - if that dark
blue border disappears is much more agreeable than the
upscaling inventing one.

Hmmmm ... this might explain why Quattro seems so sharp.
All edges are generated by using some nice patterns.

/Roland

On 2015-07-31 20:11, Erik Karlsson wrote:

As I mentioned, nearest neighbor does not seem to work very well
at
all.
The problem is that if the luminance gradient is steeper than
the
chrominance gradient you will get color bleeding at edges.
Interpolation
algorithms that generate steep gradients unfortunately tend to
overshoot
and and ring though. You essentially have to choose between
bleeding
and
fringing. SPP does neither. How is that possible?

On vie, 2015-07-31 at 10:44 -0700, Roland Karlsson wrote:

Yes ... the first 5 to 20 MP upscaling is nearest neighbor.
But, then you use the topmost layer to modify the result.
The assumption being that all three layers have the same
micro contrast.

/Roland

On 2015-07-31 18:58, Erik Karlsson wrote:

You mean nearest neighbor?

On vie, 2015-07-31 at 09:55 -0700, Roland Karlsson wrote:

How about a less cosher upscaling?

My idea is to do all computations at 5 Mpixel and then
upscale by copying every pixel to 2x2 values and then
use the topmost layer to modify the four pixels in all
three layers.

/Roland

—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub

#67 (comment).

—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub

#67 (comment).

—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub
#67 (comment).

—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub
#67 (comment).

[rolkar]

Yes, we can do some edge detection research :)
Or, we can do the pattern based upscaling I was talking about.
In any case, we cannot just use some simple non adaptive upscaling.
This is actually kind of fun :)

[erikrk]

I flag this as important since it seems to be a general Quattro issue.