Skip to content
Permalink
Browse files Browse the repository at this point in the history
Handle pixel formats with odd shift values
Our fast paths assume that each channel fits in to a separate byte.
That means the shift needs to be a multiple of 8. Start actually
checking this so that a client cannot trip us up and possibly cause
incorrect code exection.

Issue found by Pavel Cheremushkin from Kaspersky Lab.
  • Loading branch information
CendioOssman committed Nov 15, 2019
1 parent 91bdaa6 commit 05e2849
Show file tree
Hide file tree
Showing 2 changed files with 12 additions and 0 deletions.
6 changes: 6 additions & 0 deletions common/rfb/PixelFormat.cxx
Expand Up @@ -206,6 +206,12 @@ bool PixelFormat::is888(void) const
return false;
if (blueMax != 255)
return false;
if ((redShift & 0x7) != 0)
return false;
if ((greenShift & 0x7) != 0)
return false;
if ((blueShift & 0x7) != 0)
return false;

return true;
}
Expand Down
6 changes: 6 additions & 0 deletions tests/unit/pixelformat.cxx
Expand Up @@ -170,6 +170,12 @@ void is888Tests()

do888Test(false, 8, 8, false, false, 0, 0, 0, 0, 0, 0);

/* Odd shifts */

do888Test(false, 32, 24, false, true, 255, 255, 255, 0, 8, 18);
do888Test(false, 32, 24, false, true, 255, 255, 255, 0, 11, 24);
do888Test(false, 32, 24, false, true, 255, 255, 255, 4, 16, 24);

printf("\n");
}

Expand Down

0 comments on commit 05e2849

Please sign in to comment.