Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Difference in Infragram & ImageSequencer ndvi-red results #34

Closed
ccpandhare opened this issue Jul 1, 2017 · 34 comments

Comments

Projects
None yet
3 participants
@ccpandhare
Copy link
Collaborator

commented Jul 1, 2017

Original

original

Image Sequencer Output

Full Image

Image Sequencer

Infragram Output

Full Image

Infragram

Is the algorithm we are using for ndvi-red correct?

    function changePixel(r, g, b, a) {
      var ndvi = 255 * (b - r) / (1.00 * b + r);
      return [ndvi, ndvi, ndvi, a];
    }

@ccpandhare ccpandhare added the bug label Jul 1, 2017

@jywarren

This comment has been minimized.

Copy link

commented Jul 1, 2017

@ccpandhare

This comment has been minimized.

Copy link
Collaborator Author

commented Jul 1, 2017

Changed the code to this:

    function changePixel(r, g, b, a) {
      var ndvi = 255 * (b - r) / (1.00 * b + r);
      ndvi = (ndvi>0)?ndvi:0;
      return [ndvi, ndvi, ndvi, a];
    }

Output:
Output

I will check the infragram-js repository for details.

@ccpandhare ccpandhare moved this from Bugs to Work On to In Progress in Image Sequencer GSoC Project Jul 1, 2017

@ccpandhare

This comment has been minimized.

Copy link
Collaborator Author

commented Jul 1, 2017

I think Infragram uses the same algorithm.

ndvi = (nir, vis) ->
        n = nir.width * nir.height;
        d = new Float64Array(n);
        for i in [0...n]
                d[i] = (nir.data[i] - vis.data[i]) / (nir.data[i] + vis.data[i]);
return new JsImage(d, nir.width, nir.height, 1);
if mode == "ndvi"
       [r,g,b] = get_channels(img)
       ndvi_img = ndvi(r,b)
       # this isn't correct for NDVI; we want values from -1 to 1:
       # [[min],[max]] = ndvi_img.extrema()
       min = -1
       max = 1
       normalize = (x) -> (x - min) / (max - min)
       result = jsColorify(ndvi_img, (x) -> colormap(normalize(x)))
       update_colorbar(min, max)

Isn't it?

@jywarren

This comment has been minimized.

Copy link

commented Jul 6, 2017

Ah, i wonder -- do you need to be doing:

var ndvi = (255 / 2) * (1 + b - r) / (1.00 * b + r);

So that values from -1 to 1 are mapped from 0-255?

@ccpandhare

This comment has been minimized.

Copy link
Collaborator Author

commented Jul 6, 2017

I think NDVI should just be

var ndvi = (b - r) / (b + r)

Wikipedia

Update : Should It be the modulus of this?

@jywarren

This comment has been minimized.

Copy link

commented Jul 6, 2017

The issue is that there's a difference between what the NDVI value /is/ and what we can display in an image. Infragram.org decides that since there are negative values in NDVI, we should map actual NDVI values of -1 to 1, which are:

var ndvi = (b - r) / (b + r)

to the full range we can show in an image, which is 0 to 255, so, broken into two steps:

var ndvi = (b - r) / (b + r)
var pixel_value = 255 * (ndvi + 1) / 2;
@ccpandhare

This comment has been minimized.

Copy link
Collaborator Author

commented Jul 7, 2017

I am afraid that doesn't do the task either... Will look into this.
This is the output when I used the above code:
Image

@ccpandhare

This comment has been minimized.

Copy link
Collaborator Author

commented Jul 7, 2017

Somehow, We aren't reaching the extremes. I mean, The parts which are darker in the Infragram version are lighter here and the lighter ones are darker here.

@jywarren

This comment has been minimized.

Copy link

commented Jul 7, 2017

That's odd -- I ran this a few times manually with NDVI outputs:

ndvi = 1
> 1
255 * (ndvi + 1) / 2
> 255
ndvi = -1
> -1
255 * (ndvi + 1) / 2
> 0
ndvi = 0.33
> 0.33
255 * (ndvi + 1) / 2
> 169.57500000000002
ndvi = -0.33
> -0.33
255 * (ndvi + 1) / 2
> 85.425

This seems about right, no? Can you try outputting some of the pixel color values to be sure the NDVI is falling between -1 and 1? I don't really see how it could do otherwise.

@ccpandhare

This comment has been minimized.

Copy link
Collaborator Author

commented Jul 7, 2017

Yes, the ndvi value is between -1 and 1.
Here's a PasteBin Link to the ndvi of each of the pixels of this image, for reference.

But interestingly, most ndvi values are close to zero.

@jywarren

This comment has been minimized.

Copy link

commented Jul 7, 2017

Can you output the R and B values? I wonder if they're not quite right... wrong scale or something? Maybe they're 0-100 for some reason?

@ccpandhare

This comment has been minimized.

Copy link
Collaborator Author

commented Jul 7, 2017

Sure,
This PasteBin Link has r, b and ndvi values.

@jywarren

This comment has been minimized.

Copy link

commented Jul 7, 2017

Hmm, now that I think about it, isn't the last image you posted correct? That is, for areas of 0 NDVI, it should appear 50% grey. Infragram is rendering those black, which would be -1.

Do you want to post a question to the site on the topic and ask Chris Fastie and Ned Horning to confirm this?

https://publiclab.org/tag/ndvi has a lot of good info on this, and maybe test images others have used.

You could also ask a question there!

@ccpandhare

This comment has been minimized.

Copy link
Collaborator Author

commented Jul 7, 2017

Sure will do that!
Thanks for the link!

@ccpandhare

This comment has been minimized.

Copy link
Collaborator Author

commented Jul 8, 2017

This is the link to the question I have asked on the website.

@ccpandhare ccpandhare moved this from Things to Work On to In Progress in Image Sequencer GSoC Project Jul 8, 2017

@ccpandhare

This comment has been minimized.

Copy link
Collaborator Author

commented Jul 8, 2017

I have some good news on this front, finally!

A Relief, Finally!

Chris Fastie suggested I try an image he gave. I think I got the right output from ImageSeqeucer.
It even matched that of Infragram!
But I do have Some interesting news here.

Original

Original

Infragram Output

Infragram

Image Sequencer Old Algorithm Output

This is the output of the algorithm you wrote earlier:

      var ndvi = 255 * (b - r) / (1.00 * b + r);
      return [ndvi, ndvi, ndvi, a];

Old ImageSequencer

Image Sequencer New Algorithm Output

This is the output using the newer algorithm:

      var ndvi = (b - r) / (b + r);
      var x = 255 * (ndvi+1) / 2;
      return [x, x, x, a];

New ImageSequencer

What is surprising is that the older algorithm performed better. Whereas we had established that the newer algorithm maps all values. Moreover, Chris said that the new algorithm is correct, too!

So why didn't the older algorithm work for the previous image, is also another question.

@jywarren

This comment has been minimized.

Copy link

commented Jul 8, 2017

I think the older algorithm is flooring anything below zero! But you should post these results to Chris and ask him too:

https://publiclab.org/questions/ccpandhare/07-08-2017/how-to-verify-if-my-programmatically-generated-ndvi-version-of-an-image-is-correct

@ccpandhare

This comment has been minimized.

Copy link
Collaborator Author

commented Jul 8, 2017

Okay, Done that!

@ccpandhare

This comment has been minimized.

Copy link
Collaborator Author

commented Jul 13, 2017

Chris suggested another image. I tried that out. The same issue was there:

  • Old algorithm : broken at negative values
  • New Algorithm : faint.

So I created another algorithm "Improved Algorithm":

var ndvi = 255 * (b - r) / (b + r);
ndvi = (ndvi > 0) ? ndvi : 0;
return [ndvi,ndvi,ndvi,a];

This is basically the old algorithm, with the difference that I map negative values of ndvi to 0.
This algorithm produced the exact same result as Infragram! I am not sure if this algorithm is correct!

Original Image

Original

Infragram Output

Infragram

Improved Algorithm

Improved Algorithm

Old Algorithm

Old Algorithm

New Algorithm

New Algorithm

@jywarren

This comment has been minimized.

Copy link

commented Jul 13, 2017

@ccpandhare

This comment has been minimized.

Copy link
Collaborator Author

commented Jul 13, 2017

This is definitely not the right method.
It worked for this:
Original

But not for this:
Original

Interestingly, mapping all negatives to 0 and all positives to 255 worked for the second one.

So are we missing something in the algorithm?

Maybe our mapping isn't correct.
Maybe we shouldn't have a linear map at all.
Maybe we should have a linear map for a certain range of ndvi values and them map values exceeding them to 0 or 255. Something like:

-1 to -x : 0
-x to 0 : linear function
0 to x : linear function
x to 1 : 255

What do you think about this? This will solve all test cases. But I am afraid this isn't right.

On the other hand: I totally agree with this:

I think this may show how infragram.org is destroying some information when in non colorized mode.

This is certainly the correct explanation.

@ccpandhare

This comment has been minimized.

Copy link
Collaborator Author

commented Jul 13, 2017

I think there is data rounding off in Infragram as you pointed out.

@jywarren

This comment has been minimized.

Copy link

commented Jul 13, 2017

@Fastie

This comment has been minimized.

Copy link

commented Jul 13, 2017

Jeff might be right that the lower contrast version ("new algorithm") is correct. The NDVI values for that photo are mostly between -0.2 and +0.3, so there should be very little black or white in the grayscale image, assuming that -1 is black and +1 is white. Here is the histogram of NDVI values computed by Fiji:
w25floathist

However, Ned's Photo Monitoring plugin produces an image with high contrast just like the "Infragram" or "Improved Algorithm" versions above. So Infragram and the Photo Monitoring plugin might be computing NDVI correctly but mapping the grayscales strangely. Ned could help with the mapping done by the plugin.

Below is the histogram of NDVI values for Clayton's Chicka Boom row crop photo with all NDVI values > 0.

claytonndvihist

@jywarren

This comment has been minimized.

Copy link

commented Jul 13, 2017

@ccpandhare

This comment has been minimized.

Copy link
Collaborator Author

commented Jul 22, 2017

I will open up an issue for the histogram module.
Also, What colormap should we use then? Or should we use one at all?

@jywarren

This comment has been minimized.

Copy link

commented Jul 26, 2017

I'm not sure -- isn't a basic histogram just a graph, not colorized? Maybe stick with the simplest version for now?

@ccpandhare

This comment has been minimized.

Copy link
Collaborator Author

commented Jul 26, 2017

Yes, true.
I was actually referring to the colormap we were going to apply to ndvi-red in order to get better contrast for the viewers...

@jywarren

This comment has been minimized.

Copy link

commented Jul 26, 2017

@ccpandhare

This comment has been minimized.

Copy link
Collaborator Author

commented Jul 26, 2017

No no, I totally understand that you have a lot to look after!

Actually we were getting different outputs for ndvi-red than what Infragram gives.
After a lot of tests, we concluded that Image Sequencer gave the right results, and that Infragram uses some "colormap" to enhance the image for better viewing.

The histogram thing was completely different, Sorry. That must have confused you - I shouldn't have said everything in one comment.

@jywarren

This comment has been minimized.

Copy link

commented Jul 26, 2017

@ccpandhare

This comment has been minimized.

Copy link
Collaborator Author

commented Jul 26, 2017

Yes there is a separate colormap module. Yes, this does solve my question. I was under the impression that we should have a colormap too.
Okay then, I think this can be closed now and ndvi-red is good to go!

@jywarren

This comment has been minimized.

Copy link

commented Jul 26, 2017

@ccpandhare

This comment has been minimized.

Copy link
Collaborator Author

commented Jul 26, 2017

It could be renamed to ndvi and default to R and B channels.
As of now it doesn't accept any such parameters.

@ccpandhare ccpandhare removed this from In Progress in Image Sequencer GSoC Project Aug 9, 2017

@ccpandhare ccpandhare closed this Aug 20, 2017

@ccpandhare ccpandhare self-assigned this Aug 22, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.