New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

copy() and blend() methods produce colors aren't exactly 255 or 0 #258

Closed
processing-bugs opened this Issue Feb 10, 2013 · 5 comments

Comments

Projects
None yet
5 participants
@processing-bugs

processing-bugs commented Feb 10, 2013

Original author: b...@processing.org (June 07, 2010 01:18:22)

This bug automatically added from:
http://dev.processing.org/bugs/show_bug.cgi?id=1420

Comment from hungerburg, 2009-12-27 15:27

My sketch requests images, on loading filters them THRESHOLD and then
copies the new image over an old one with img.copy(...)

The sketch later reads pixels from the image. Unlike the starting image,
that comes from the data directory and is a b/w png, alpha in newly fetched
images becomes FE and black becomes in hex() not FF000000 but FE000000. The
OPAQUE filter does not help.

Original issue: http://code.google.com/p/processing/issues/detail?id=219

@processing-bugs

This comment has been minimized.

Show comment
Hide comment
@processing-bugs

processing-bugs Feb 10, 2013

From b...@processing.org on June 07, 2010 01:18:23
Comment from fry, 2010-02-17 20:02

can you post a very short example that shows the problem?

processing-bugs commented Feb 10, 2013

From b...@processing.org on June 07, 2010 01:18:23
Comment from fry, 2010-02-17 20:02

can you post a very short example that shows the problem?

@processing-bugs

This comment has been minimized.

Show comment
Hide comment
@processing-bugs

processing-bugs Feb 10, 2013

From b...@processing.org on June 07, 2010 01:18:23
Comment from hungerburg, 2010-02-18 03:54

Created an attachment (id=401)
img.copy() changes alpha from 255 to 254

Hello fry!

My report was a little confusing. I created a minimal sketch, its clearer now.
There is just one image that is copied over itself. Look at the console output,
press shift for a second, after the image is swapped, alpha goes from 255 to
254. filtering does not help. loadPixels neither.

PImage img; // image we want to track
PImage nimg; // next image
color c; // under mouse

void setup()
{
frameRate(1);
size (200, 200);
img = loadImage("some.jpg");
nimg = loadImage("some.jpg");
//nimg.filter(OPAQUE);
}

void draw()
{
image(img, 0, 0, width, height);
if(keyPressed && keyCode == SHIFT) {
img.copy(nimg, 0, 0, nimg.width, nimg.height, 0, 0, width, height);
//img.loadPixels();
println("image swapped.");
}
c = img.pixels[mouseY*img.width+mouseX];
println(mouseX+", "+mouseY+", "+alpha(c));
}

processing-bugs commented Feb 10, 2013

From b...@processing.org on June 07, 2010 01:18:23
Comment from hungerburg, 2010-02-18 03:54

Created an attachment (id=401)
img.copy() changes alpha from 255 to 254

Hello fry!

My report was a little confusing. I created a minimal sketch, its clearer now.
There is just one image that is copied over itself. Look at the console output,
press shift for a second, after the image is swapped, alpha goes from 255 to
254. filtering does not help. loadPixels neither.

PImage img; // image we want to track
PImage nimg; // next image
color c; // under mouse

void setup()
{
frameRate(1);
size (200, 200);
img = loadImage("some.jpg");
nimg = loadImage("some.jpg");
//nimg.filter(OPAQUE);
}

void draw()
{
image(img, 0, 0, width, height);
if(keyPressed && keyCode == SHIFT) {
img.copy(nimg, 0, 0, nimg.width, nimg.height, 0, 0, width, height);
//img.loadPixels();
println("image swapped.");
}
c = img.pixels[mouseY*img.width+mouseX];
println(mouseX+", "+mouseY+", "+alpha(c));
}

@processing-bugs

This comment has been minimized.

Show comment
Hide comment
@processing-bugs

processing-bugs Feb 10, 2013

From b...@processing.org on June 07, 2010 01:18:23
Comment from fry, 2010-02-18 06:00

k, the problem is just that the image functions (like copy) aren't super
accurate. they sacrifice accuracy for speed, so there's some rounding error
as things are transferred. unfortunately that's what you're seeing here.
there's not much we can do about it short of making the image copy() method
far slower, so i think we're mostly stuck with it.

thanks for the report, i'll keep this around in case we can move to a
better solution in the future.

processing-bugs commented Feb 10, 2013

From b...@processing.org on June 07, 2010 01:18:23
Comment from fry, 2010-02-18 06:00

k, the problem is just that the image functions (like copy) aren't super
accurate. they sacrifice accuracy for speed, so there's some rounding error
as things are transferred. unfortunately that's what you're seeing here.
there's not much we can do about it short of making the image copy() method
far slower, so i think we're mostly stuck with it.

thanks for the report, i'll keep this around in case we can move to a
better solution in the future.

@hungerburg

This comment has been minimized.

Show comment
Hide comment
@hungerburg

hungerburg Feb 25, 2013

Hello fry, I did submit the report back then. Feel free to close, your answer satisfies the condition, I'd say.

Kind regards

Peter

hungerburg commented Feb 25, 2013

Hello fry, I did submit the report back then. Feel free to close, your answer satisfies the condition, I'd say.

Kind regards

Peter

@mcslee

This comment has been minimized.

Show comment
Hide comment
@mcslee

mcslee Aug 4, 2014

I believe this issue could be improved at a marginal performance hit, without introducing floating point operations. The PImage functions seem to make common use of an optimization that uses integer-multiplication-with-bitshift to simulate floating point multiplication on a byte i.e. v2 = (v1 * f) >> 8;

The factor used is typically an alpha mask byte, ranging from 0-255 (0x00 - 0xff).

If (alpha + 1) were used instead (i.e. v2 = (v1 * (f+1)) >> 8), making the multiplication domain 1-256, the results would be more accurate, and this should not cause any overflow problems since multiplying by 256 is the equivalent of shifting by one byte (<< 8).

This would only introduce one extra integer addition to these operation. Seems worth it, IMO.

Currently, PImage.blend(#000000, #ffffff, BLEND) does not return #ffffff, nor does PImage.blend(#ffffff, #000000, BLEND) return #000000. That's a very unexpected behavior.

The ideal function would use floating point arithmetic to compute perfect blending, i.e. v2 = (v1 * f / 255.f). I wrote a sketch to compare the errors introduces by integer optimizations for the optimization as-is, vs. if it used f+1.
screen shot 2014-08-03 at 10 58 17 pm

The brightness of the colors are the intensity of error, measured as absolute value difference between the optimized function and the ideal floating point computation. It clearly shows that the error patterns are very similar, but that the (f+1) version introduces less error and gets all the edge cases correct.

I did the operation in the red, green, and blue byte just to prove that it doesn't matter which byte this integer-arithmetic operation is done in.

The source code for this demonstration sketch is pasted in full below:

float[][] error0 = new float[256][256];
float[][] error1 = new float[256][256];

float totalError0 = 0;
float totalError1 = 0;

float maxError0 = 0;
float maxError1 = 0;

int RED_SHIFT = 16;
int GREEN_SHIFT = 8;
int BLUE_SHIFT = 0;

void setup() {

size(50 + 768, 610);
background(#ffffff);
smooth();

noStroke();
fill(#000000);
text("Approximating v' = v * (alpha / 255.f) over domain v, alpha => [0, 255]", 10, 14);

doErrors(RED_SHIFT);
doErrors(GREEN_SHIFT);
doErrors(BLUE_SHIFT);
}

void doErrors(int shift) {

totalError0 = totalError1 = maxError0 = maxError1 = 0;

for (int value = 0; value <= 0xff; ++value) {
for (int alpha = 0x00; alpha <= 0xff; ++alpha) {
int vfast0 = ((value << shift) * alpha) >>> (8+shift);
int vfast1 = ((value << shift) * (alpha+1)) >>> (8+shift);
float vslow = value * (alpha/255.f);
float e0 = abs(vfast0 - vslow);
float e1 = abs(vfast1 - vslow);
error0[value][alpha] = e0;
error1[value][alpha] = e1;
totalError0 += e0;
totalError1 += e1;
maxError0 = max(maxError0, e0);
maxError1 = max(maxError1, e1);

}

}

String v = (shift > 0) ? ("(v << " + shift + ")") : "v";

drawErrors(shift, error0, 10 + (16-shift)_270/8, 20,
"v' = (" + v + " * alpha) >>> " + (8+shift) + "\n" +
"total error: " + totalError0 + " max: " + maxError0
);
drawErrors(shift, error1, 10 + (16-shift)_270/8, 316,
"v' = (" + v + " * (alpha+1)) >>> " + (8+shift) + "\n" +
"total error: " + totalError1 + " max: " + maxError1);

}

void drawErrors(int shift, float[][] errors, int x, int y, String formula) {
stroke(#000000);
noFill();
rect(x, y, 257, 257);
noStroke();
for (int i = 0; i < 256; ++i) {
for (int j = 0; j < 256; ++j) {
fill(min(0xff, (int) (255.f * errors[i][j])) << 24 | (0xff << shift));
rect(x + 1 + i, y + 1 + j, 1, 1);
}
}
fill(#000000);
text(formula, x, y + 272);
}

mcslee commented Aug 4, 2014

I believe this issue could be improved at a marginal performance hit, without introducing floating point operations. The PImage functions seem to make common use of an optimization that uses integer-multiplication-with-bitshift to simulate floating point multiplication on a byte i.e. v2 = (v1 * f) >> 8;

The factor used is typically an alpha mask byte, ranging from 0-255 (0x00 - 0xff).

If (alpha + 1) were used instead (i.e. v2 = (v1 * (f+1)) >> 8), making the multiplication domain 1-256, the results would be more accurate, and this should not cause any overflow problems since multiplying by 256 is the equivalent of shifting by one byte (<< 8).

This would only introduce one extra integer addition to these operation. Seems worth it, IMO.

Currently, PImage.blend(#000000, #ffffff, BLEND) does not return #ffffff, nor does PImage.blend(#ffffff, #000000, BLEND) return #000000. That's a very unexpected behavior.

The ideal function would use floating point arithmetic to compute perfect blending, i.e. v2 = (v1 * f / 255.f). I wrote a sketch to compare the errors introduces by integer optimizations for the optimization as-is, vs. if it used f+1.
screen shot 2014-08-03 at 10 58 17 pm

The brightness of the colors are the intensity of error, measured as absolute value difference between the optimized function and the ideal floating point computation. It clearly shows that the error patterns are very similar, but that the (f+1) version introduces less error and gets all the edge cases correct.

I did the operation in the red, green, and blue byte just to prove that it doesn't matter which byte this integer-arithmetic operation is done in.

The source code for this demonstration sketch is pasted in full below:

float[][] error0 = new float[256][256];
float[][] error1 = new float[256][256];

float totalError0 = 0;
float totalError1 = 0;

float maxError0 = 0;
float maxError1 = 0;

int RED_SHIFT = 16;
int GREEN_SHIFT = 8;
int BLUE_SHIFT = 0;

void setup() {

size(50 + 768, 610);
background(#ffffff);
smooth();

noStroke();
fill(#000000);
text("Approximating v' = v * (alpha / 255.f) over domain v, alpha => [0, 255]", 10, 14);

doErrors(RED_SHIFT);
doErrors(GREEN_SHIFT);
doErrors(BLUE_SHIFT);
}

void doErrors(int shift) {

totalError0 = totalError1 = maxError0 = maxError1 = 0;

for (int value = 0; value <= 0xff; ++value) {
for (int alpha = 0x00; alpha <= 0xff; ++alpha) {
int vfast0 = ((value << shift) * alpha) >>> (8+shift);
int vfast1 = ((value << shift) * (alpha+1)) >>> (8+shift);
float vslow = value * (alpha/255.f);
float e0 = abs(vfast0 - vslow);
float e1 = abs(vfast1 - vslow);
error0[value][alpha] = e0;
error1[value][alpha] = e1;
totalError0 += e0;
totalError1 += e1;
maxError0 = max(maxError0, e0);
maxError1 = max(maxError1, e1);

}

}

String v = (shift > 0) ? ("(v << " + shift + ")") : "v";

drawErrors(shift, error0, 10 + (16-shift)_270/8, 20,
"v' = (" + v + " * alpha) >>> " + (8+shift) + "\n" +
"total error: " + totalError0 + " max: " + maxError0
);
drawErrors(shift, error1, 10 + (16-shift)_270/8, 316,
"v' = (" + v + " * (alpha+1)) >>> " + (8+shift) + "\n" +
"total error: " + totalError1 + " max: " + maxError1);

}

void drawErrors(int shift, float[][] errors, int x, int y, String formula) {
stroke(#000000);
noFill();
rect(x, y, 257, 257);
noStroke();
for (int i = 0; i < 256; ++i) {
for (int j = 0; j < 256; ++j) {
fill(min(0xff, (int) (255.f * errors[i][j])) << 24 | (0xff << shift));
rect(x + 1 + i, y + 1 + j, 1, 1);
}
}
fill(#000000);
text(formula, x, y + 272);
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment