[question] Single band tiff with colortable

[moovida]

Hello, I would like to visualize single band tiff (an example would be a digital elevation model map that contains floating point values of the elevation in the pixel positions). The "image" doesn't have an internal colortable, but would come with a sidecar file describing it.

I was wondering if it is possible to visualize such a file and if yes, what would be the best way to plug into the image production to apply the colortable and transform from physical value to color pixel.

Thanks for any information you might throw in.

[brendan-duncan]

If I were to guess, I would say you would read the tiff as an HDR image
HdrImage floatImage = TiffDecoder().decodeHdrImage(imageData);
Which I would expect should handle the single channel float tiff, though it's been a long time since I was in that code and don't remember what state it's in.

Then you'd want to do the sidecar colortable processing yourself, probably storing the results in a regular Image for display.

There's nothing built-in that wold do all that together, and it's pretty specific thing to be added. It will also be pretty slow, depending on the size of the image. Dart isn't the best language for doing this sort of CPU heavy work. But it should be doable without too much effort (unless my floating-point TIFF decoder is buggy).

[brendan-duncan]

A quick test shows I never finished writing the float tiff decoder. I can try and get float support added, but I can't promise it'll be soon (work and life make working on this stuff hard, as much as I enjoy it). EXR should support float textures, if that's an option.

[brendan-duncan]

I was able to add 32-bit and 16-bit float format to the TIFF decoder without too much fuss. I'll hold off on pushing a new version until I can test it a bit more, it was done in a bit of a rush. If this is something that would work for your project, you can always temporarily point your package dependency to the git version.

[moovida]

Woot @brendan-duncan , thanks a lot for your reply. I will take some time to test this with a couple of datasets this week. I will keep you posted on the results if that is ok.
Thanks for your work!

[brendan-duncan]

I went ahead and pushed the new version with the float tiff to pub. I can always push a new version if I need to fix anything, and this way I won't forget to do it later :-).

[moovida]

Hi @brendan-duncan , finally been able to take some time to test this.

The result looks very promising. But I have a few questions. Feel free to redirect me to the documentation. I looked but couldn't find the answers.

• what is the right way to access the values of the image?

I tested the following two ways, that seem to be equivalent:

      for (var row = 0; row < image.height; row++) {
        for (var col = 0; col < image.width; col++) {
          var redValue = image.red.getFloat(col, row);
          stdout.write("$redValue  ");
        }
        stdout.write("\n");
      }
      stdout.write("==========\n");
      var float32list = image.toFloatRgba();
      int index = 0;
      for (var row = 0; row < image.height; row++) {
        for (var col = 0; col < image.width; col++) {
          var value = float32list[index];
          index += 4;
          stdout.write("$value  ");
        }
        stdout.write("\n");
      }

Is one of the above better?

• it seems that the values are kept inside the bands as floating point and duplicated in order to fill all the bands with the same values. This is a triplication of the memory footprint. Is that required by the library design?
• is there a way to tell seamlessly if an image has a single band and might contain physical data?
• if the data were 64bit numbers, would the library figure it out properly?

Sorry for the many questions.

---

There is one issue with novalues, i.e. values that are marked inside the tiff as non existing value placeholders.
In the following I refer to the attached mini test tiff digital elevation map. The zip contains the same version in tiff and asc, the second being an ascii representation of the tiff's values.

If I print out the values, the result is:

800.0  900.0  1000.0  1000.0  1200.0  1250.0  1300.0  1350.0  1450.0  1500.0  
600.0  -10000.0  750.0  850.0  860.0  900.0  1000.0  1200.0  1250.0  1500.0  
500.0  550.0  700.0  750.0  800.0  850.0  900.0  1000.0  1100.0  1500.0  
400.0  410.0  650.0  700.0  750.0  800.0  850.0  490.0  450.0  1500.0  
450.0  550.0  430.0  500.0  600.0  700.0  800.0  500.0  450.0  1500.0  
500.0  600.0  700.0  750.0  760.0  770.0  850.0  1000.0  1150.0  1500.0  
600.0  700.0  750.0  800.0  780.0  790.0  1000.0  1100.0  1250.0  1500.0  
800.0  910.0  980.0  1001.0  1150.0  1200.0  1250.0  1300.0  1450.0  1500.0  

while the original is:

800.0 900.0 1000.0 1000.0 1200.0 1250.0 1300.0 1350.0 1450.0 1500.0 
600.0 -9999.0.0 750.0 850.0 860.0 900.0 1000.0 1200.0 1250.0 1500.0
500.0 550.0 700.0 750.0 800.0 850.0 900.0 1000.0 1100.0 1500.0 
400.0 410.0 650.0 700.0 750.0 800.0 850.0 490.0 450.0 1500.0 
450.0 550.0 430.0 500.0 600.0 700.0 800.0 500.0 450.0 1500.0
500.0 600.0 700.0 750.0 760.0 770.0 850.0 1000.0 1150.0 1500.0 
600.0 700.0 750.0 800.0 780.0 790.0 1000.0 1100.0 1250.0 1500.0 
800.0 910.0 980.0 1001.0 1150.0 1200.0 1250.0 1300.0 1450.0 1500.0

The only wrong value is the -9999.0 that instead is read as -10000.0. I checked with a hex editor and the tiff really contains -9999.0, so I am not sure why it is "rounded" to -10000.0

Anyways, this is a great start. We work on an open source library for hydro/geomorphological risk computation and this is the first time I think that mobile devices are starting to be ready to do this. :-D

Final note: as you guessed, plugging in a coloring of the map might lead to quite bad performance. I wonder if there is a way to do this upon reading. Would it be possible to place the phisical value in the depth channel and use rbg then for the colorvalues converted upon reading?

Sorry for the long read.

singleband.zip

[brendan-duncan]

toFloatArray() is significantly slower than getFloat, since it allocates a Float32List and does a for-loop over all the pixels of the image to fill it. For 32-bit float images, image.red.data will be a Float32List with the pixel values so you can access them directly as an array without the extra pixel-to-index code in getFloat.

The other channels being filled for a single channel hdr image was done out of laziness and not for any other reason. I should clean that up, it would save a lot of wasted memory.

There's nothing about an image that would indicate "physical" values. If I fix up the "don't waste memory by filling the other channels" issue, then at least you'd be able to tell how many channels the image has.

I didn't add 64-bit float to HdrImage, so if that's something you'll need, I'd need to do that. I don't think it would be too much work.

I'll take a look at the "rounding" issue. Definitely strange.

Remapping the image and using the color table on load is quite a specific need for a generic library, it's a quick way to end up with a lot of spaghetti code. But maybe there's something I could add, like a custom callback for setting the pixel on load or something. I'm not entirely sure, but I'll think about it. Performance is defiantly a concern for Dart code, it's not the fastest language for CPU heavy tasks. It's justifiably the most common complaint for this library. A lot of people will write native libraries to do CPU heavy functions.

One thing to note is when I wrote this library years ago, it was for a personal project...I hadn't intended on writing a general purpose image processing library, and there are many design decisions in here I would have done very differently if it were done again as a proper image library. I also have the excuse of writing it before Dart 1.0 even came out, and the language has changes considerably since it was first written. I just had fun and got a bit carried away figuring out image formats. Oh well, c'est la vie, maybe one day myself or someone else will clean it up. In the mean time, at least I can help fix things up as I can.

[brendan-duncan]

The rounding error is because that tiff is a 16-bit half-float, and that conversion is getting rounded. Maybe the tiff 16-bit float spec is different than the one I'm using, which is from the EXR implementation. I'll have to dig deeper into the TIFF spec.

[brendan-duncan]

I tried another halfToFloat conversion function and got the same -1000 value, so my guess is the error is in the 16-bit tiff.

[moovida]

toFloatArray() is significantly slower than getFloat, since it allocates a Float32List and does a for-loop over all the pixels of the image to fill it. For 32-bit float images, image.red.data will be a Float32List with the pixel values so you can access them directly as an array without the extra pixel-to-index code in getFloat.

Ah, ok, thanks for this!

The other channels being filled for a single channel hdr image was done out of laziness and not for any other reason. I should clean that up, it would save a lot of wasted memory.

There's nothing about an image that would indicate "physical" values. If I fix up the "don't waste memory by filling the other channels" issue, then at least you'd be able to tell how many channels the image has.

Thanks a ton. I am not sure if I can help here, but if you point me in the right direction I could try to see if I understand what is going on and maybe make a pull request.

I didn't add 64-bit float to HdrImage, so if that's something you'll need, I'd need to do that. I don't think it would be too much work.

It sure might be a good addition if that unlocks certain physical data. For me at the moment this library is a beautiful black box, so I am not sure how I can help.

I'll take a look at the "rounding" issue. Definitely strange.

Remapping the image and using the color table on load is quite a specific need for a generic library, it's a quick way to end up with a lot of spaghetti code. But maybe there's something I could add, like a custom callback for setting the pixel on load or something. I'm not entirely sure, but I'll think about it. Performance is defiantly a concern for Dart code, it's not the fastest language for CPU heavy tasks. It's justifiably the most common complaint for this library. A lot of people will write native libraries to do CPU heavy functions.

Thanks for thinking about it.

One thing to note is when I wrote this library years ago, it was for a personal project...I hadn't intended on writing a general purpose image processing library, and there are many design decisions in here I would have done very differently if it were done again as a proper image library. I also have the excuse of writing it before Dart 1.0 even came out, and the language has changes considerably since it was first written. I just had fun and got a bit carried away figuring out image formats. Oh well, c'est la vie, maybe one day myself or someone else will clean it up. In the mean time, at least I can help fix things up as I can.

Well, I think this library is much appreciated by many. Mine are really just questions and by no means complaints. I was already wondering that so much could already be done for a "scientific" purpose. Many many thanks.

[attaching the other comments]

The rounding error is because that tiff is a 16-bit half-float, and that conversion is getting rounded. Maybe the tiff 16-bit float spec is different than the one I'm using, which is from the EXR implementation. I'll have to dig deeper into the TIFF spec.
I tried another halfToFloat conversion function and got the same -1000 value, so my guess is the error is in the 16-bit tiff.

For what I know from other libraries the tiff specs/metadata are a beast. I will try to look if I can find this in the geotools java library, maybe there are some hints about it.

[brendan-duncan]

I don't mind the questions at all, happy to help.

From what I've seen in the TIFF specs, https://www.adobe.io/open/standards/TIFF.html, I couldn't even find 16-bit float mentioned. TIFF is a kitchen sink of a format, everyone has thrown their junk into it, so sometimes figuring things out is a challenge. The code I'm currently using to do the half->float conversion is in lib/src/hdr/half.dart, and is derived from the OpenEXR library. I have also tried this code, https://www.asmail.be/msg0054931905.html, which produced the same results as my half.dart. I'll try a C++ version soon, rule out something in the Dart translation being the issue.

The code that needs to be fixed for decoding single channel images is lib/src/formats/tiff/tiff_image.dart. decodeHdr is always creating an HdrImage with four channels, and _decodeTile is always setting all four channels. I can probably get that cleaned up soon, but I'm always happy to get pull requests if it's something you're interested in doing.

For 64-bit float images, I'll need to add a FLOAT64 type to lib/src/hdr/hdr_image.dart, update lib/src/hdr/hdr_slice.dart to include Float64List, and add bitsPerSample == 64 checks in TiffImage._decodeTile. Shoudln't be too much work.

[brendan-duncan]

I pushed 64-bit float tiff support to github, but I haven't tested it. I don't actually know how to create a 64-bit float tiff for testing it with.

[brendan-duncan]

I also noticed in the same way I was lazily always creating 4 channels, I was also always lazily making the HdrImage 16-bit half-float. I suspect that's where the rounding error in your test is coming from. I'll fix that while I'm in there.

[brendan-duncan]

I pushed the fix for the floating-point precision, so the HdrImage will not be converted to 16-bit floats for every image. This fixes the -9999.0 => -10000 problem. I also fixed it so it doesn't create the wasted image channels for single channel images. You can use HdrImage.numChannels to see how many channels there are. Single channel images will be stored in the red channel.

I'll push an update soon, but noticed I must have broken CMYK TIFF images at some point, so off to fix that.

[brendan-duncan]

The updates are in 2.1.18

[moovida]

Outstanding! Thank you @brendan-duncan .
I will create a small project and see if I can produce different sample size tiff files to use in testcases.

I will keep you posted about results. This is exciting :-D

[moovida]

Hi @brendan-duncan , I created a small repository to test different single band data samples I created for this purpose.
Here it is: https://github.com/moovida/hortonmachine_dart
Feel free to take whatever from there if you find it useful.

I created 4 tiffs (the only ones I have been able to for now):

• 64float
• 32float
• 32int
• 16int

Then I created a testcase that checks all 4 sets.Each testcase contains also a dump of the tiff metadata as for example the 32float metadata are here: https://github.com/moovida/hortonmachine_dart/blob/master/test/hortonmachine_test.dart#L10-L68
These metadata come from a very mature library (gdal.org) so they can be assumed to be right.

My int readings fail due to the inner format being double. Is it possible that also the int files are read as doubles?

That said, I have a few questions again:

• if the image is single band, can I always assume that the red channel is used?
• when having a look at the metadata dump, is there something I can access from those? You might have noticed that there are a lot of data in there. As you said tiff is a kitchen sink and in my case I am handling geographic data and the metadata/tags (not even sure which naming is right in this case) contain vital information to properly place the image in the right "world" position.

[moovida]

One small correction to the above:

• the 32bit int case seems to be reading doubles and breaks with the exception:

type 'double' is not a subtype of type 'int' in type cast
package:image/src/hdr/hdr_slice.dart 81:21                                   HdrSlice.getInt

• the 16bit int case instead doesn't complain, but the numbers are read out wrong.

[moovida]

I digged a bit into the reader and see that the TiffImage has a list of TiffEntry tags. I assume those are used to fill the properties of the TiffImage. I do not see the possibility to get to the origin and resolution values, as ex for the test tiffs I placed:

        Origin = (1640650.000000000000000,5140020.000000000000000)
        Pixel Size = (30.000000000000000,-30.000000000000000)

These are the minimum required to properly scale and place the pixels on a map and do physical processing on them.
I guess they are unreachable, but I will ask anyways :-) Do you know if there is a way to get them from the tiff with the current library status?
Thank you

[moovida]

And last question: do you have some part of tiff writing available or was that format never planned? :-) I just ported a geomorphologic algorithm fro java to dart and noticed at the time of dumping the file to disk that tiff is a readonly format. Guess I have not been very careful due to the excitement :-D

[brendan-duncan]

• if the image is single band, can I always assume that the red channel is used?
  Yes, that's the way it's currently written. If you have another opinion on that, happy to hear it, I just picked something I could do quickly without too many changes.

• I haven't done much with the int formats, I think I might only have UInt16 in there. I'll add the others, which will be easier having your test images to validate against.

• Regarding origin/pixelSize properties and other tag data, I'm not actually sure, I don't remember off hand how I had written that TiffEntry list thing. If you can't access it now to get the info you need, then that's something that needs to be fixed. I'll check it out.

• For a TIFF encoder, I hadn't planned it, but then again I hadn't planned the TIFF decoder until someone had asked, so I'll see what I can do.

I'm getting a bit more time to work on this stuff recently since my wife has been busy working late :-). Might as well do something useful, and I enjoy it :-).

[brendan-duncan]

The HdrImage class was pretty limited. I originally wrote it when I needed the EXR format for playing around with some graphics programs I was working on, and it was very much tuned to just that usage. All of these formats (int, uint, float) and bit depths (8, 16, 32, 64) really require the HdrImage class to be refactored. I made a branch, https://github.com/brendan-duncan/image/tree/tiff_hdr_refactor, where I'm started doing that. So far it seems to be working well but I'll need to do a lot more testing and cleaning up. The only format I probably won't be able to do is int64 and uint64, since that's more limited with Dart, due to not being available in Javascript for web compilation.

[brendan-duncan]

I pushed a change into the branch where the tiff tag entries can be readable after decoding. I added a test,

      final bytes = File('test/res/tiff/dtm64float.tif').readAsBytesSync();
      final decoder = TiffDecoder();
      final image = decoder.decodeHdrImage(bytes);
      expect(image.numChannels, equals(1));
      final tags = decoder.info.images[0].tags;
      for (var tag in tags.keys) {
        final entry = tags[tag];
        if (entry.type == TiffEntry.TYPE_ASCII) {
          print('dtm64float TAG $tag: ${entry.readString()}');
        } else {
          print('dtm64float TAG $tag: ${entry.read()}');
        }
      }

After decoding, TiffDecoder.info will still have the extra data it used during decoding. Knowing the tag you're interested in, you can get it via:
List tagValues = decoder.info.images[0].tags[34264].read();

In your test images, I saw the values you mentioned in tag 34264, which has the values: [30.0, 0.0, 0.0, 1640650.0, 0.0, -30.0, 0.0, 5140020.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0]. Looking that tag up in the tiff tag database, https://www.loc.gov/preservation/digital/formats/content/tiff_tags.shtml, that tag is ModelTransformationTag, so it's a 4x4 transformation matrix. If you're just interested in the translation and scale, and you know there's no rotation in the transformation, you can just pull those values directly from the matrix.

[brendan-duncan]

I've started the encoder. We'll see how far I get while I have some more time this evening.

[brendan-duncan]

Progress on the encoder, but it'll take a couple more days.

[moovida]

@brendan-duncan , this is amazing! Your issue emails I find in the morning are something incredible :-D
Given the fact that the transformation matrix is accessible, there are almost all pieces to create something called geoimage. I assume that is out of scope for the image library itself, so if you think it is ok, I would like to start a geoimage project, which would basically:

• expose geographic information of the image, if available
• allow for physical data, if the format supports it

Regarding the red band being the one used by default for single band rasters, I have nothing against it. If there was another channel that could better represent it I would use that, but there is none, right? What does depth stand for?

Regarding the tiff tags being accessible, that helps a ton. I hope I will find also the projection string inside there somewhere. That would bring all the pieces together. I will check later when I can have some consequent time to work on it :-) It is something that starts with:

PROJCRS["WGS 84 / UTM zone 32N",
            BASEGEOGCRS["WGS 84",
                DATUM["World Geodetic System 1984",
                    ELLIPSOID["WGS 84",6378137,298.257223563,
                        LENGTHUNIT["metre",1]]],
...

PS: for now I will head to use the branch https://github.com/brendan-duncan/image/tree/tiff_hdr_refactor , right?
PS2: there is a format used a lot in GIS, because it is ascii and helps a ton when debugging. I will make use of it for conversion testcases and similar stuff. It is called esri ascii grid: https://en.wikipedia.org/wiki/Esri_grid
I will write and encoder/decoder. It is out of scope for the image library right? Or could it somehow fit in the HDR part?

[moovida]

Sorry, one more question, more a curiosity and out of scope here: did you ever try the library with dart2native? Is it supported? I have no idea but it feels like it could be a solution if more performance is needed?

[brendan-duncan]

There's a projection matrix tag was in there, tags[34735] = [1, 1, 2, 3, 1024, 0, 1, 1, 1025, 0, 1, 1, 3072, 0, 1, 32632]. I'm not sure I see a string, if it was in the test images.

Doing a new package specifically for geoimage would be a great way of doing it. That way you could target it for your specific needs with a cleaner API.

depth channel is used for rendering graphics, representing a z-buffer, one of the use cases for EXR images in rendering. I was using it for a rendering project I was playing with a couple years ago, https://github.com/brendan-duncan/dartray.

Yes, you'll need to point your pubspec config to use the github branch until I push the new version, which will still be a little bit as I try and get the encoder finished.

I think the esri_grid codec would be outside the scope of the image library. Seems like it would be a natural fit for the geoimage library, though.

I haven't tested it with dart2native. I haven't actually done very much Dart programming in a while, other than the couple of libraries I had written and try to maintain, and when I was doing more dart programming I mostly did web stuff. The last time I checked there were some things dart2native was less efficient at, but I don't know what the current state is. If you're targeting mobile, Dart does allow you to integrate native code plug-ins, so if something is too slow that's another option. Definitely worth trying dart2native.

[moovida]

There's a projection matrix tag was in there, tags[34735] = [1, 1, 2, 3, 1024, 0, 1, 1, 1025, 0, 1, 1, 3072, 0, 1, 32632]. I'm not sure I see a string, if it was in the test images.

I still have to understand what the first numbers are, but the last is the right EPSG code of the file's projection ( https://www.spatialreference.org/ref/epsg/32632/ )

Doing a new package specifically for geoimage would be a great way of doing it. That way you could target it for your specific needs with a cleaner API.

yes, agreed. I will definitely go in that direction after some first tests.

depth channel is used for rendering graphics, representing a z-buffer, one of the use cases for EXR images in rendering. I was using it for a rendering project I was playing with a couple years ago, https://github.com/brendan-duncan/dartray.

Is this a 3D rendering engine for dart? Would that allow to visualize LiDAR point cloud data?

Yes, you'll need to point your pubspec config to use the github branch until I push the new version, which will still be a little bit as I try and get the encoder finished.

That is perfectly fine. I am experimenting and will need some time to understand what is possible to support.

I think the esri_grid codec would be outside the scope of the image library. Seems like it would be a natural fit for the geoimage library, though.

Yes, I just wonder what I should take as blueprint from the image library in order to have the same behaviour as for tiffs for example. Maybe it is a push though. I will do some tests.

I haven't tested it with dart2native. I haven't actually done very much Dart programming in a while, other than the couple of libraries I had written and try to maintain, and when I was doing more dart programming I mostly did web stuff. The last time I checked there were some things dart2native was less efficient at, but I don't know what the current state is. If you're targeting mobile, Dart does allow you to integrate native code plug-ins, so if something is too slow that's another option. Definitely worth trying dart2native.

Ok thank you.

And as usual a closing question. Do you have support for affinetransformations in the image library? I was wondering if it would be possible to do simple reprojection of geographic rasters.

[brendan-duncan]

Is this a 3D rendering engine for dart? Would that allow to visualize LiDAR point cloud data?

It's an off-line slow raytrace renderer, more typically used for art, not a real-time 3D engine, which would be more suitable for visualizing LiDAR point clouds. That would be pretty straight forward to do with WebGL, but last time I checked there was no 3D plans for Dart/Flutter, which is the main reason I haven't been very interested in Flutter. There are some good Javascript/WebGL examples of this, http://potree.org/potree/examples/showcase/sorvilier.html http://lidarview.com https://sandcastle.cesium.com/?src=3D%20Tiles%20Point%20Cloud.html

Do you have support for affinetransformations in the image library? I was wondering if it would be possible to do simple reprojection of geographic rasters.

Not currently

[brendan-duncan]

Question: do you need encoding TIFF images to be compressed? I'll have the uncompressed export working relatively soon, but might take longer to add compression. I'll also need to add the ability to add custom tags to the TIFF.

[moovida]

Question: do you need encoding TIFF images to be compressed? I'll have the uncompressed export working relatively soon, but might take longer to add compression. I'll also need to add the ability to add custom tags to the TIFF.

Uncompressed would be already gold and allow me to to some consistency tests using different tiff image formats.

Yes, true, I assume without custom tags I would not be able to add geotiff mandatory tags, right?

[brendan-duncan]

Great, uncompressed is easier :-). The basic minimum encoder is mostly done now then, just have to add user defined tag data. Maybe something like:

final geoTags = [ TIFFField(TIFF_TAG_ModelTransformation, TIFF_FLOAT, [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]) ];
final tiff = TIFFEncoder(tags: geoTags).encodeHdrImage(hdr);

I don't know, I'll play around with different ideas for it tomorrow.

[moovida]

Just for your information. I have been playing with tif tags and samples from the geotools java project in order to have some solid test in place.

Right now the tests look good, apart of:

• the int datatypes (but we talked about this already)
• compressed data. I have really no knowledge in that, so I just made some testcases on the datasets, in the hope they might help you fi you want to check them: https://github.com/moovida/hortonmachine_dart/blob/master/test/hortonmachine_test.dart#L568-L640

BTW, even nodata values are read properly as nan in one dataset. Nice nice.

[brendan-duncan]

I'll have to get back to this in a little bit, regular work is starting to pile up. It's probably close to done other than the tag entries. Hopefully I can get through my work tasks soon.

[moovida]

I'll have to get back to this in a little bit, regular work is starting to pile up. It's probably close to done other than the tag entries. Hopefully I can get through my work tasks soon.

@brendan-duncan , don't know how to thank you for everything done so far. I already have enough to start a small geoimage project that can be used in mapping environments to load tiff and png/jpgs with worldfile definitions. Same as for you, I am lagging behind with regular work, so it might be good for me also to slow down a bit :-D

I might be opening issues with questions if that is ok. They are not by no means urgent, it is just to keep track of them.