# Difference in Infragram & ImageSequencer ndvi-red results #34

Closed
opened this issue Jul 1, 2017 · 34 comments

Projects
None yet
3 participants
Collaborator

## Infragram Output

#### Full Image

Is the algorithm we are using for `ndvi-red` correct?

``````    function changePixel(r, g, b, a) {
var ndvi = 255 * (b - r) / (1.00 * b + r);
return [ndvi, ndvi, ndvi, a];
}
``````

### jywarren commented Jul 1, 2017

 Looks like negative values are getting output as positive in ImageSequencer, perhaps? See the black tree trunks? 🌲 Those are zero or negative values. Maybe negative values aren't getting zeroed out? … On Jul 1, 2017 12:26 PM, "Chinmay Pandhare" ***@***.***> wrote: Original Thumbnail↓ Full Image Here [image: Original] Image Sequencer Output Full Image [image: Image Sequencer] Infragram Output Full Image [image: Infragram] Is the algorithm we are using for ndvi-red correct? function changePixel(r, g, b, a) { var ndvi = 255 * (b - r) / (1.00 * b + r); return [ndvi, ndvi, ndvi, a]; } — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#34>, or mute the thread .
Collaborator Author

### ccpandhare commented Jul 1, 2017

 Changed the code to this: `````` function changePixel(r, g, b, a) { var ndvi = 255 * (b - r) / (1.00 * b + r); ndvi = (ndvi>0)?ndvi:0; return [ndvi, ndvi, ndvi, a]; } `````` Output: I will check the `infragram-js` repository for details.

### ccpandhare moved this from Bugs to Work On to In Progress in Image Sequencer GSoC ProjectJul 1, 2017

Collaborator Author

### ccpandhare commented Jul 1, 2017

 I think Infragram uses the same algorithm. ``````ndvi = (nir, vis) -> n = nir.width * nir.height; d = new Float64Array(n); for i in [0...n] d[i] = (nir.data[i] - vis.data[i]) / (nir.data[i] + vis.data[i]); return new JsImage(d, nir.width, nir.height, 1); `````` ``````if mode == "ndvi" [r,g,b] = get_channels(img) ndvi_img = ndvi(r,b) # this isn't correct for NDVI; we want values from -1 to 1: # [[min],[max]] = ndvi_img.extrema() min = -1 max = 1 normalize = (x) -> (x - min) / (max - min) result = jsColorify(ndvi_img, (x) -> colormap(normalize(x))) update_colorbar(min, max) `````` Isn't it?

### jywarren commented Jul 6, 2017

 Ah, i wonder -- do you need to be doing: `var ndvi = (255 / 2) * (1 + b - r) / (1.00 * b + r);` So that values from -1 to 1 are mapped from 0-255?
Collaborator Author

### ccpandhare commented Jul 6, 2017 • edited

 I think NDVI should just be `var ndvi = (b - r) / (b + r)` Wikipedia Update : Should It be the modulus of this?

### jywarren commented Jul 6, 2017

 The issue is that there's a difference between what the NDVI value /is/ and what we can display in an image. Infragram.org decides that since there are negative values in NDVI, we should map actual NDVI values of -1 to 1, which are: `var ndvi = (b - r) / (b + r)` to the full range we can show in an image, which is 0 to 255, so, broken into two steps: ```var ndvi = (b - r) / (b + r) var pixel_value = 255 * (ndvi + 1) / 2;```
Collaborator Author

### ccpandhare commented Jul 7, 2017

 I am afraid that doesn't do the task either... Will look into this. This is the output when I used the above code:
Collaborator Author

### ccpandhare commented Jul 7, 2017

 Somehow, We aren't reaching the extremes. I mean, The parts which are darker in the Infragram version are lighter here and the lighter ones are darker here.

### jywarren commented Jul 7, 2017 • edited

 That's odd -- I ran this a few times manually with NDVI outputs: ```ndvi = 1 > 1 255 * (ndvi + 1) / 2 > 255 ndvi = -1 > -1 255 * (ndvi + 1) / 2 > 0 ndvi = 0.33 > 0.33 255 * (ndvi + 1) / 2 > 169.57500000000002 ndvi = -0.33 > -0.33 255 * (ndvi + 1) / 2 > 85.425``` This seems about right, no? Can you try outputting some of the pixel color values to be sure the NDVI is falling between -1 and 1? I don't really see how it could do otherwise.
Collaborator Author

### ccpandhare commented Jul 7, 2017

 Yes, the `ndvi` value is between -1 and 1. Here's a PasteBin Link to the ndvi of each of the pixels of this image, for reference. But interestingly, most ndvi values are close to zero.

### jywarren commented Jul 7, 2017

 Can you output the R and B values? I wonder if they're not quite right... wrong scale or something? Maybe they're 0-100 for some reason?
Collaborator Author

### ccpandhare commented Jul 7, 2017

 Sure, This PasteBin Link has r, b and ndvi values.

### jywarren commented Jul 7, 2017

 Hmm, now that I think about it, isn't the last image you posted correct? That is, for areas of 0 NDVI, it should appear 50% grey. Infragram is rendering those black, which would be -1. Do you want to post a question to the site on the topic and ask Chris Fastie and Ned Horning to confirm this? https://publiclab.org/tag/ndvi has a lot of good info on this, and maybe test images others have used. You could also ask a question there!
Collaborator Author

### ccpandhare commented Jul 7, 2017

 Sure will do that! Thanks for the link!
Collaborator Author

### ccpandhare commented Jul 8, 2017

 This is the link to the question I have asked on the website.

### ccpandhare moved this from Things to Work On to In Progress in Image Sequencer GSoC ProjectJul 8, 2017

Collaborator Author

### ccpandhare commented Jul 8, 2017

I have some good news on this front, finally!

### A Relief, Finally!

Chris Fastie suggested I try an image he gave. I think I got the right output from ImageSeqeucer.
It even matched that of Infragram!
But I do have Some interesting news here.

### Image Sequencer Old Algorithm Output

This is the output of the algorithm you wrote earlier:

```      var ndvi = 255 * (b - r) / (1.00 * b + r);
return [ndvi, ndvi, ndvi, a];```

### Image Sequencer New Algorithm Output

This is the output using the newer algorithm:

```      var ndvi = (b - r) / (b + r);
var x = 255 * (ndvi+1) / 2;
return [x, x, x, a];```

What is surprising is that the older algorithm performed better. Whereas we had established that the newer algorithm maps all values. Moreover, Chris said that the new algorithm is correct, too!

So why didn't the older algorithm work for the previous image, is also another question.

### jywarren commented Jul 8, 2017

 I think the older algorithm is flooring anything below zero! But you should post these results to Chris and ask him too: https://publiclab.org/questions/ccpandhare/07-08-2017/how-to-verify-if-my-programmatically-generated-ndvi-version-of-an-image-is-correct
Collaborator Author

### ccpandhare commented Jul 8, 2017

 Okay, Done that!
Collaborator Author

### ccpandhare commented Jul 13, 2017

Chris suggested another image. I tried that out. The same issue was there:

• Old algorithm : broken at negative values
• New Algorithm : faint.

So I created another algorithm "Improved Algorithm":

``````var ndvi = 255 * (b - r) / (b + r);
ndvi = (ndvi > 0) ? ndvi : 0;
return [ndvi,ndvi,ndvi,a];
``````

This is basically the old algorithm, with the difference that I map negative values of ndvi to 0.
This algorithm produced the exact same result as Infragram! I am not sure if this algorithm is correct!

### jywarren commented Jul 13, 2017

 Ok, this is good to know. I think this may show how infragram.org is destroying some information when in non colorized mode. To do better we should preserve sub zero values and pass the whole thing to the colorized for easier viewing. I think we're good here then! … On Jul 13, 2017 9:44 AM, "Chinmay Pandhare" ***@***.***> wrote: Chris suggested another image. I tried that out. The same issue was there: - Old algorithm : broken at negative values - New Algorithm : faint. So I created another algorithm "Improved Algorithm": var ndvi = 255 * (b - r) / (b + r); ndvi = (ndvi > 0) ? ndvi : 0; return [ndvi,ndvi,ndvi,a]; This is basically the old algorithm, with the difference that I map negative values of ndvi to 0. This algorithm produced the exact same result as Infragram! I am not sure if this algorithm is correct! Original Image [image: Original] Infragram Output [image: Infragram] Improved Algorithm [image: Improved Algorithm] Old Algorithm [image: Old Algorithm] New Algorithm [image: New Algorithm] — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#34 (comment)>, or mute the thread .
Collaborator Author

### ccpandhare commented Jul 13, 2017

 This is definitely not the right method. It worked for this: But not for this: Interestingly, mapping all negatives to 0 and all positives to 255 worked for the second one. So are we missing something in the algorithm? Maybe our mapping isn't correct. Maybe we shouldn't have a linear map at all. Maybe we should have a linear map for a certain range of ndvi values and them map values exceeding them to 0 or 255. Something like: ``````-1 to -x : 0 -x to 0 : linear function 0 to x : linear function x to 1 : 255 `````` What do you think about this? This will solve all test cases. But I am afraid this isn't right. On the other hand: I totally agree with this: I think this may show how infragram.org is destroying some information when in non colorized mode. This is certainly the correct explanation.
Collaborator Author

### ccpandhare commented Jul 13, 2017

 I think there is data rounding off in Infragram as you pointed out.

### jywarren commented Jul 13, 2017

 Yes. I think the lower contrast image is actually the correct one. To make it more readable we should colormap it. … On Jul 13, 2017 10:04 AM, "Chinmay Pandhare" ***@***.***> wrote: I think there is data rounding off in Infragram as you pointed out. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#34 (comment)>, or mute the thread .

### Fastie commented Jul 13, 2017

 Jeff might be right that the lower contrast version ("new algorithm") is correct. The NDVI values for that photo are mostly between -0.2 and +0.3, so there should be very little black or white in the grayscale image, assuming that -1 is black and +1 is white. Here is the histogram of NDVI values computed by Fiji: However, Ned's Photo Monitoring plugin produces an image with high contrast just like the "Infragram" or "Improved Algorithm" versions above. So Infragram and the Photo Monitoring plugin might be computing NDVI correctly but mapping the grayscales strangely. Ned could help with the mapping done by the plugin. Below is the histogram of NDVI values for Clayton's Chicka Boom row crop photo with all NDVI values > 0.

### jywarren commented Jul 13, 2017

 very helpful. i think we're on the right track then, and part of the NDVI module may need to be a config option to set the output range (defaulting to -1=>1 mapping to 0-255). Also, i think we should make a histogram module! Kinda weird, but it could output a histogram graph given an image input! but let's think about that later (it could use plotly, though inefficiently, and would probably break some module rules) … On Thu, Jul 13, 2017 at 10:44 AM, Fastie ***@***.***> wrote: Jeff might be right that the lower contrast version ("new algorithm") is correct. The NDVI values for that photo are mostly between -0.2 and +0.3, so there should be very little black or white in the grayscale image, assuming that -1 is black and +1 is white. Here is the histogram of NDVI values computed by Fiji: [image: w25floathist] However, Ned's Photo Monitoring plugin produces an image with high contrast just like the "Infragram" or "Improved Algorithm" versions above. So Infragram and the Photo Monitoring plugin might be computing NDVI correctly but mapping the grayscales strangely. Ned could help with the mapping done by the plugin. Below is the histogram of NDVI values for Clayton's Chicka Boom row crop photo with all NDVI values > 0. [image: claytonndvihist] — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#34 (comment)>, or mute the thread .
Collaborator Author

### ccpandhare commented Jul 22, 2017

 I will open up an issue for the histogram module. Also, What colormap should we use then? Or should we use one at all?

### jywarren commented Jul 26, 2017

 I'm not sure -- isn't a basic histogram just a graph, not colorized? Maybe stick with the simplest version for now?
Collaborator Author

### ccpandhare commented Jul 26, 2017

 Yes, true. I was actually referring to the colormap we were going to apply to ndvi-red in order to get better contrast for the viewers...

### jywarren commented Jul 26, 2017

 Oh haha ok yeah. So last I remembered we broke colormap into its own module. Are we talking about what the default is or what we should show in the example demonstrating ndvi? Again, apologies if I'm slow at the moment. … On Jul 26, 2017 7:00 AM, "Chinmay Pandhare" ***@***.***> wrote: Yes, true. I was actually referring to the colormap we were going to apply to ndvi-red in order to get better contrast for the viewers... — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#34 (comment)>, or mute the thread .
Collaborator Author

### ccpandhare commented Jul 26, 2017

 No no, I totally understand that you have a lot to look after! Actually we were getting different outputs for `ndvi-red` than what Infragram gives. After a lot of tests, we concluded that Image Sequencer gave the right results, and that Infragram uses some "colormap" to enhance the image for better viewing. The histogram thing was completely different, Sorry. That must have confused you - I shouldn't have said everything in one comment.

### jywarren commented Jul 26, 2017

 But didn't I see a separate colormap module you wrote? I think we can just consider ndvi separately from color mapping; after all, Infragram does them as two separate steps. Does that answer your question? … On Jul 26, 2017 9:19 AM, "Chinmay Pandhare" ***@***.***> wrote: No no, I totally understand that you have a lot to look after! Actually we were getting different outputs for ndvi-red than what Infragram gives. After a lot of tests, we concluded that Image Sequencer gave the right results, and that Infragram uses some "colormap" to enhance the image for better viewing. The histogram thing was completely different, Sorry. That must have confused you - I shouldn't have said everything in one comment. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#34 (comment)>, or mute the thread .
Collaborator Author

### ccpandhare commented Jul 26, 2017

 Yes there is a separate colormap module. Yes, this does solve my question. I was under the impression that we should have a colormap too. Okay then, I think this can be closed now and `ndvi-red` is good to go!

### jywarren commented Jul 26, 2017

 Wait, do we need an NDVI module which accepts an option for which channels to use, default to red/blue? Rather than this? Or maybe ndvi-red is just a wrapper if ndvi which defaults to red? Or am I overthinking this. … On Jul 26, 2017 10:22 AM, "Chinmay Pandhare" ***@***.***> wrote: Yes there is a separate colormap module. Yes, this does solve my question. I was under the impression that we should have a colormap too. Okay then, I think this can be closed now and ndvi-red is good to go! — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#34 (comment)>, or mute the thread .
Collaborator Author

### ccpandhare commented Jul 26, 2017

 It could be renamed to `ndvi` and default to R and B channels. As of now it doesn't accept any such parameters.