Intensity ratio algorithm

[joanlyq]

Hi team, thx for the quick reply last time. I also wanna quickly check with you if the intensity ratio algorithm is unavailable at the moment. I checked both GUI and python library, the RGBIntensityRatioMasker is not working. I tried to dig into the published article a bit but can't seem to find the difference of the threshold and intensity ration masker.

GlintMaskGenerator/glint_mask_generator/glint_algorithms.py

Lines 36 to 46 in 46e8a1f

	"""Create and return a glint mask for RGB imagery.
	Parameters
	----------
	percent_diffuse
	An estimate of the percentage of pixels in an image that show pure diffuse reflectance, and thus no specular
	reflectance (glint).
	threshold
	The threshold on the specular reflectance estimate image to convert into a mask.
	e.g. if more than 50% specular reflectance is unacceptable, use 0.5.
	"""

just by reading this couple of lines, it seems that the intensity ratio is also using the threshold with an additional parameter with diffuse reflectance?

[tayden]

Hi @joanlyq,

The intensity ratio algorithm is not really supported in the released version. It was something we played around with early on, but we found that simple thresholding was effective enough.

The ThresholdAlgorithm is pretty simple, and just classifies a pixel in the image as "glint" if any of the band pixel values are greater than the threshold passed in. e.g. if using the thresholds (0.9, 0.8, 0.7), you'll get a mask back that specifies glint for any pixel where the Red band value is greater than 0.9, or the Green band is greater than 0.8, or the Blue band is greater than 0.7. (Note: as a preprocessing step, the pixel values are normalized to fall between 0 and 1).

The IntensityRatioAlgorithm on the other hand is more complex. For each pixel it tries to estimate how much of the pixel is composed of specular and diffuse reflectance. It does this by calculating something the authors call "intensity ratio" of each pixel (intensity ratio = max(R,G,B)/(max(R,G,B)-min(R,G,B)). Then, using a user estimation of the proportion of the image that can confidently be said to have no glint, it finds a intensity ratio value for a pixel that represents purely diffuse reflection. Knowing this intensity ratio value for a purely diffuse pixel, it estimates the component of specular reflection in all of the remaining pixels. Finally, a threshold on the amount of specular reflection that is acceptable for a single pixel is used to mask out the pixels that are too-glinty.

The issue, with the intensity ratio method is that the idea depends on the assumption that the image is of mostly uniform colour, except for areas of glint. Failing that, the idea that the calculated intensity ratio corresponds to the proportion of specular and diffuse reflection in each pixel doesn't really hold up. To overcome that limitation, the authors suggest segmenting different areas of the image first by coloured regions and running the algorithm multiple times for each region.

In the end, using the intensity ratio algorithm adds quite a bit of processing overhead to do it properly and it was ultimately abandoned in favour of the naive-yet-effective approach of the ThresholdAlgorithm.

[joanlyq]

Hey Taylor, thx for the detailed explanation. It makes sense. I've also tried some other DL glint detector and they all have their own trouble. I totally agree, anything that is effective is good, no need to be complicated. Cheers!