How to convert the signed 16 bit output into a disparity value?

[mcelhennyi]

I have gotten this library to work with my point cloud conversion code when using the 8-bit output. I would like to get the 16-bit signed output to work for better precision, but cant understand the scale value.

I have noticed other implementations using unsigned 16 bit outputs with a scale value division on each short of the disparity image to get the disparity value in float form. What is the correct conversion for this library, I cannot seem to find this in other issues, docs or code.

Thanks!

[mcelhennyi]

Self closing, for some reason I thought the output called for a 16bit signed. Just realized its Unsigned.

[mcelhennyi]

Reopening to focus on the scale value.

[mcelhennyi]

This is what I landed on through experimentation, is this correct?

bool subpixel = true;
.....
float tmpDisp = (((unsigned short*)row_ptr)[x]) & 0x0fff;
unsigned short fractionalShort = ((unsigned short*)row_ptr)[x] >> 12;
float fractional = 0.0f;
if(fractionalShort != 0)
{
    fractional = 1.0f / (float)fractionalShort;
}
disparityValue = (float)tmpDisp / computeConstraints.scaleFactor;

// Add on the fractional portion
disparityValue += fractional;

tmpDisp is my libSGM output disparity value for the given x,y pixel and disparityValue would be the 0-128(max) value.

[atakagi-fixstars]

Hi, @mcelhennyi

In subpixel mode, the output disparity value is scaled by sgm::StereoSGM::SUBPIXEL_SCALE (its actual value is 16) .
You can get the floating point value by dividing the output disparity value by SUBPIXEL_SCALE(or multiplying by 1/SUBPIXEL_SCALE).

With OpenCV, you can convert all pixels by convertTo function.
https://github.com/fixstars/libSGM/blob/master/sample/reprojection/stereosgm_reprojection.cpp#L250

Related issue: #21

Regards,

[mcelhennyi]

Hi, @mcelhennyi

In subpixel mode, the output disparity value is scaled by sgm::StereoSGM::SUBPIXEL_SCALE (its actual value is 16) .
You can get the floating point value by dividing the output disparity value by SUBPIXEL_SCALE(or multiplying by 1/SUBPIXEL_SCALE).

With OpenCV, you can convert all pixels by convertTo function.
https://github.com/fixstars/libSGM/blob/master/sample/reprojection/stereosgm_reprojection.cpp#L250

Related issue: #21

Regards,

Thanks for the reply,

And just for confirmation, the fractional part is stored in the SUBPIXEL_SHIFT MSBs?

[atakagi-fixstars]

Hi, @mcelhennyi

And just for confirmation, the fractional part is stored in the SUBPIXEL_SHIFT MSBs?

The fractional part is stored in the SUBPIXEL_SHIFT LSBs, not MSBs.

int mask = (1 << SUBPIXEL_SHIFT) - 1; // 0x0f

unsigned short output_value = ...;
int fractional_part = output_value & mask;
int integer_part = output_value >> SUBPIXEL_SHIFT;

Regards,

[mcelhennyi]

@atakagi-fixstars When I use the LSB version like you mentioned, I get very large (>maxDisparity) disparity values after I do the shifting. However when I use the MSB like I mentioned above, I get accurate disparity values.

Here is my LSB code and MSB code:

Here my ((unsigned short*)row_ptr)[x] is the output value pixel of the disparity calculation. computeConstraints.fractionalBits is SUBPIXEL_SHIFT. computeConstraints.scaleFactor is the SUBPIXEL_SCALE

unsigned short fractionalShort;
float tmpDisp;
if(computeConstraints.lsbFractionalBits)
{
    // LSBs hold the fractional bits
    // Calculate the bits used for integer vs fractional
    unsigned int fractionalFilter = (1 << computeConstraints.fractionalBits) - 1; // Creates an all 1's filter for "fractionalBits" bits
    fractionalShort = ((unsigned short*)row_ptr)[x] & fractionalFilter;
    tmpDisp = (((unsigned short*)row_ptr)[x]) >> computeConstraints.fractionalBits;
}
else
{
    // MSBs hold the fractional bits
    // Calculate the bits used for integer vs fractional
    unsigned int integerFilter = 0xffff /* Max 16 bit value */ >> (computeConstraints.fractionalBits);
    fractionalShort = ((unsigned short*)row_ptr)[x] >> ((sizeof(uint16_t) * 8) - computeConstraints.fractionalBits);
    tmpDisp = (((unsigned short*)row_ptr)[x]) & integerFilter;
}

// Convert fractional (int) to decimal (float)
float fractional = 0.0f;
if(fractionalShort != 0)
{
    fractional = 1.0f / (float)fractionalShort;
}

// Scale the value
disparityValue = (float)tmpDisp / computeConstraints.scaleFactor;

// Add on the fractional portion
disparityValue += fractional;

The output disparityValue is much larger than the maxDisparity set in the libSGM (256)

Here is the fractional from the MSB version:

And here is the fractional from the LSB version:

They appear to have a similar shaped output but the scale of the LSB variant is way off. In this case, the center point of the image has no fractional part (0) and an 11 integer output value (prior to scaling)

This is what I get when I remove the integer part being divided by the scale:

Meaning this line:

// Scale the value
disparityValue = (float)tmpDisp / computeConstraints.scaleFactor;

changes to this:

// Scale the value
disparityValue = (float)tmpDisp;

So the question is:
Do the fractional parts live in the MSBs, OR, is the scale to not be used in subpixel mode? ....or am I totally off in some other way?

[atakagi-fixstars]

When I use the LSB version like you mentioned, I get very large (>maxDisparity) disparity values after I do the shifting

Maybe it is invalid disparity value(-1), which is represented by ((unsigned short)-16)=65520 in subpixel mode,
You need to check if the disparity value is invalid before reconstruction, and do not reconstruct from invalid disparity.

This is sample code to get the floating point disparity value and check if it is invalid.

int SUBPIXEL_SCALE = 16;
unsigned short output_disparity_u16 = ...; // maybe contains invalid value 65520
short output_disparity_s16 = (short)output_disparity_u16 ; // invalid value changes to -16
float output_disparity_32f = (float)output_disparity_s16 / SUBPIXEL_SCALE; // invalid value changes to -1

bool is_invalid = output_disparity_32f < 0.f;

So the question is:
Do the fractional parts live in the MSBs, OR, is the scale to not be used in subpixel mode? ....or am I totally off in some other way?

The libSGM implements subpixel computation with LSB fractional part.
Here is the part in which subpixel disparity is computed.
https://github.com/fixstars/libSGM/blob/master/src/winner_takes_all.cu#L56

At this line the integer part is shifted up, and at this line the fractional part is added.

Maybe something is wrong with your operations.

Regards,

[mcelhennyi]

Oh I see now, I was shifting AND scaling. But now I realize dividing by 16 is the same as shifting by 4 bits and break the parts down manually.

Thanks for the help.