yuv_table overflow on RGBU

[GoogleCodeExporter]

yuv to rgb table uses 2.6 fixed point times 8.0 = 10.6 signed values.

But 2.018 produces a value out of range:
#define RGBU(i) { \
  static_cast<int16>(2.018 * 64 * (i - 128) + 0.5), \
  static_cast<int16>(-0.391 * 64 * (i - 128) + 0.5), \
  0, \
  0 \
}

For RGBU(0xFF)
Since it only affects 2 values, the best solution is to clamp the table entries.

Original issue reported on code.google.com by fbarch...@google.com on 8 Nov 2011 at 7:00

[GoogleCodeExporter]

The table is fixed.
But in next CL, I'll remove the table method completely.
The SSSE3 version uses multiplies, and the C version mimics the SSSE3 for bit 
exactness

Original comment by fbarch...@chromium.org on 18 Nov 2011 at 9:43

• Changed state: Fixed