Skip to content
This repository has been archived by the owner on Nov 16, 2023. It is now read-only.

Thermal imagery normalization #4

Closed
pbaer opened this issue Jul 18, 2018 · 8 comments
Closed

Thermal imagery normalization #4

pbaer opened this issue Jul 18, 2018 · 8 comments

Comments

@pbaer
Copy link
Member

pbaer commented Jul 18, 2018

Our thermal data is in a raw 16-bit PNG format. We need to both a) figure out exactly what the raw values correspond to so that we can normalize the data (i.e. 0 degrees Celsius should be the same 16-bit pixel value in any image) for training and b) convert it an 8-bit representation for human visual inspection.
@Marcel-Simon
Copy link
Contributor

Marcel-Simon commented Jul 18, 2018 •
edited

It seems the reason why the 16 bit images are all gray is because the values are all pretty high, while still having very small variance.

In: np.unique(img, return_counts=True)
Out: (array([51241, 51361, 51381, 51401, 51421, 51441, 51461, 51481, 51501,
        51521, 51541, 51561, 51581, 51601, 51621, 51641, 51661, 51681,
        51701, 51721, 51741, 51761, 51781, 51801, 51821, 51841, 51861,
        51881, 51901, 51921, 51941, 51961, 51981, 52001, 52021, 52041,
        52061, 52101, 52181]),
 array([    1,     2,    13,     6,    31,    50,   204,   313,   880,
         1086,  3079,  6116, 11211, 12168, 27619, 25012, 45961, 33892,
        50997, 30580, 34756, 14925, 13214,  5228,  4758,  1819,  1709,
          762,   615,   323,   163,    80,    40,    35,    10,     8,
           10,     2,     2], dtype=int64))

However, the image reading itself seems fine. We just need proper normalization. For example, try this:

img = np.array(PIL.Image.open(image_path)).astype(np.float64)
mi = np.percentile(img, 1)
ma = np.percentile(img, 99)
plt.imshow((img - mi)/(ma-mi), vmin=0, vmax=1)

It is also important to set vmin=0, vmax=1 when using pyplot, as it will mess up scaling because there are negative and values >1 in the array after normalization. Also note that int is automatically converted to float during the devision as mi and ma are floats.

Also be careful when converting to 8 bit, because you will get unhandled overflows:

In: np.array([255,256,257]).astype(np.uint8)
Out: array([255,   0,   1], dtype=uint8)

@jomalsan
Copy link
Contributor

The normalization that you have above will work for viewing the images, however I would not train on data that has been normalized in this way. Because the normalization takes the entire image into account, the portion of the image that contains a seal will be different numerically depending on if the image is primarily ice (low value) or liquid (high value)

@Marcel-Simon
Copy link
Contributor

It's true that the distribution might be very different locally. It would be helpful to get some statistics as this issue certainly influences the architecture of recognition models. Let's make a possible action item out of this:

  • Collect statistics of the IR image values across different images and for local regions containing seals. This could be as simple as an histogram, but we also need a way to compare the statistics of different images and somehow measure how different they are from each other. It would be also good to know, if the temperature value of the seals is consistent or not.

@jomalsan
Copy link
Contributor

I am working on putting together a Jupyter notebook that explains what we currently know and then building a python script for other people to include

@jomalsan
Copy link
Contributor

Thanks to Eric for his help on this project. The code is checked into "src/ir-normalization" and the normalized images have been added as a tar file to blob storage as "ArcticSealsTrain1807221152_N.tar". For the new normalized 8-bit thermal images, we have changed "16BIT" to "8BIT_N" in the file names to keep it straight.

@pbaer
Copy link
Member Author

pbaer commented Jul 25, 2018 via email

@Marcel-Simon
Copy link
Contributor

Marcel-Simon commented Jul 25, 2018 via email

@pbaer
Copy link
Member Author

pbaer commented Jul 25, 2018 via email

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@pbaer @jomalsan @Marcel-Simon