How to proper set or copy EXIF tags for GTiff images

[bigopensky]

Expected behaviour and actual behaviour.

I work on toolkit that automatically classifies data from a bunch of aerial cameras. Each camera uses geo-tagging and stores position and time meta-data in exif tags. I use GDAL version 2.0.2 on a device running debian 8.

When I use GDAL to copy the dataset via GDALCreateCopy(...) for further work all additional tags from the EXIF domain are lost. When I use,
GDALSetMetadataItem(h_dataset, key, value, "EXIF"),
as proposed in the GDAL-GeoTiff documentation section Metadata, to copy the tags from the source file, the EXIF domain is empty. What is the right procedure to copy / set these tags into the new file?

Steps to reproduce the problem.

To reproduce the issue I've written a small C test program that takes a source GTiff,

1. make a copy of it with the help of GDALCreateCopy(...)
2. list the content of metadata domains of the source image
3. try to copy the tags of the EXIF domain with 4 different settings
4. list the content of metadata domains of the resulting image

The program has the following parameter:

Usage: store-md infile outfile N|Y N|Y

1. infile the geotiff input file
2. outfile the copy of the file
3. N|Y parameter to remove the EXIT_ prefix from the tag N - don do it, Y - yes remov it
4. N|Y parameter to choose the metatdata domain for storag
   N - use GDALSetMetadataItem(h_dataset, key, value, NULL)
   Y - use GDALSetMetadataItem(h_dataset, key, value,"EXIF")

In the source file are the domains IMAGE_STRUCTURE and EXIF present. To have an GDAL independent point of view, I use Phil Harveys exiftool.

Nature of the source.tif using store-md

# -------------------------------------------
#  INVESTIGATE SOURCE 
# -------------------------------------------
..open GTiff file source.tif
.OK open

list metadata domains of source.tif
DOMAIN.01: IMAGE_STRUCTURE 
  TAG.01: COMPRESSION=LZW 
  TAG.02: INTERLEAVE=BAND 

DOMAIN.02:     EXIF 
  TAG.01: EXIF_GPSAltitude=(309.77) 
  TAG.02: EXIF_GPSAltitudeRef=00 
  TAG.03: EXIF_GPSDateStamp=2018.05.28 
  TAG.04: EXIF_GPSLatitude=(48) (19) (36.461) 
  TAG.05: EXIF_GPSLatitudeRef=N 
  TAG.06: EXIF_GPSLongitude=(15) (53) (26.604) 
  TAG.07: EXIF_GPSLongitudeRef=E 
  TAG.08: EXIF_GPSMapDatum=WGS-84 
  TAG.09: EXIF_GPSTimeStamp=(15) (23) (34) 

Nature of the source.tif using gdalinfo

$ gdalinfo -mdd EXIF source.tif
Driver: GTiff/GeoTIFF
Files: source.tif
Size is 640, 512
Coordinate System is `'
Metadata (EXIF):
  EXIF_GPSAltitude=(309.77)
  EXIF_GPSAltitudeRef=00
  EXIF_GPSDateStamp=2018.05.28
  EXIF_GPSLatitude=(48) (19) (36.461)
  EXIF_GPSLatitudeRef=N
  EXIF_GPSLongitude=(15) (53) (26.604)
  EXIF_GPSLongitudeRef=E
  EXIF_GPSMapDatum=WGS-84
  EXIF_GPSTimeStamp=(15) (23) (34)
Image Structure Metadata:
  COMPRESSION=LZW
  INTERLEAVE=BAND
Corner Coordinates:
Upper Left  (    0.0,    0.0)
Lower Left  (    0.0,  512.0)
Upper Right (  640.0,    0.0)
Lower Right (  640.0,  512.0)
Center      (  320.0,  256.0)
Band 1 Block=640x6 Type=UInt16, ColorInterp=Gray

Nature of the source.tif using exiftool

$ exiftool source.tif
ExifTool Version Number         : 10.20
File Name                       : source.tif
Directory                       : .
File Size                       : 274 kB
File Modification Date/Time     : 2018:08:03 13:52:55+02:00
File Access Date/Time           : 2018:08:03 13:53:15+02:00
File Inode Change Date/Time     : 2018:08:03 13:52:55+02:00
File Permissions                : rw-------
File Type                       : TIFF
File Type Extension             : tif
MIME Type                       : image/tiff
Exif Byte Order                 : Little-endian (Intel, II)
Image Width                     : 640
Image Height                    : 512
Bits Per Sample                 : 16
Compression                     : LZW
Photometric Interpretation      : BlackIsZero
Strip Offsets                   : (Binary data 564 bytes, use -b option to extract)
Samples Per Pixel               : 1
Rows Per Strip                  : 6
Strip Byte Counts               : (Binary data 429 bytes, use -b option to extract)
Planar Configuration            : Chunky
Predictor                       : Horizontal differencing
GPS Latitude Ref                : North
GPS Longitude Ref               : East
GPS Altitude Ref                : Above Sea Level
GPS Time Stamp                  : 15:23:34
GPS Map Datum                   : WGS-84
GPS Date Stamp                  : 2018:05:28
GPS Altitude                    : 309.7 m Above Sea Level
GPS Date/Time                   : 2018:05:28 15:23:34Z
GPS Latitude                    : 48 deg 19' 36.46" N
GPS Longitude                   : 15 deg 53' 26.60" E
GPS Position                    : 48 deg 19' 36.46" N, 15 deg 53' 26.60" E
Image Size                      : 640x512
Megapixels                      : 0.328

The transfer of meta-data

The copy attempt finds all of these the tags an tries to write the data with different strategies into the new image. The copy strategy is given by the switches strip_it (Parameter 3) and tag_it (Parameter 4). The copy procedure looks like this:

 printf("# -------------------------------------------\n");
 printf("#  CREATE COPY \n");
 printf("# -------------------------------------------\n");
 printf("..copy %s file %s to %s\n", format, ifile, ofile);
 GDALDatasetH h_ddst = GDALCreateCopy( h_drv, ofile, h_dsrc, 0,
                                       NULL, NULL, NULL);
 if( h_ddst == NULL ) {error_exit(10, "Unable to copy dataset %s!\n", ofile); }
 printf(".OK open\n\n");

 printf(".. copy exif data from %s to %s \n", ifile, ofile);
 char** lst_exif = GDALGetMetadata(h_dsrc, dmn_exif);
 printf("..create copy %s from %s\n", ofile, ifile);
 if (tag_it)
 {
     printf("..use %s the metadata domain\n",dmn_exif);
 }
 else
 {
     printf("..use the default domain NULL\n");
 }

 if (strip_it)
 {
     printf("..remove the prefix EXIF_ from the tags\n");
 }

 if (lst_exif != NULL)
 {

     int tlen = CSLCount(lst_exif);

     for (int ix=0; ix < tlen; ix++ )
     {
         char *tkn  = strdup(CSLGetField(lst_exif, ix));
         char *key  = strtok(tkn, "=");
         char *value  = strtok(NULL, "");
         // DONT WORK
         // value = CPLParseNameValue( CSLGetField(lst_tags, ix), key );
         if ( (value != NULL) && (key!=NULL) )
         {
             if (strip_it) str_strip(key, "EXIF_");
             printf ("  TAG.%02d: %s=%s\n",ix, key, value);
             if (tag_it)
             {
                 GDALSetMetadataItem(h_ddst, key, value, dmn_exif);
             }
             else
             {
                 GDALSetMetadataItem(h_ddst, key, value, NULL);
             }
         }
         free(tkn);
     }
}

and produces the following output:

# -------------------------------------------
#  CREATE COPY 
# -------------------------------------------
..copy GTiff file source.tif to dest.tif
.OK open

.. copy exif data from source.tif to dest.tif 
..create copy dest.tif from source.tif
..use EXIF the metadata domain
  TAG.00: EXIF_GPSAltitude=(309.77)
  TAG.01: EXIF_GPSAltitudeRef=00
  TAG.02: EXIF_GPSDateStamp=2018.05.28
  TAG.03: EXIF_GPSLatitude=(48) (19) (36.461)
  TAG.04: EXIF_GPSLatitudeRef=N
  TAG.05: EXIF_GPSLongitude=(15) (53) (26.604)
  TAG.06: EXIF_GPSLongitudeRef=E
  TAG.07: EXIF_GPSMapDatum=WGS-84
  TAG.08: EXIF_GPSTimeStamp=(15) (23) (34)

Results ./store-md source.tif dest.tif N N

As expected due to the function call GDALSetMetadataItem(h_ddst, key, value, NULL); a new empty domain was created and gdalinfo reports them as an EXIF domain but the exiftool finds them in a XML string not in the EXIF-IFD.

GDALINFO

gdalinfo -mdd EXIF dest.tif
Driver: GTiff/GeoTIFF
Files: dest.tif
Size is 640, 512
Coordinate System is `'
Metadata:
  EXIF_GPSAltitude=(309.77)
  EXIF_GPSAltitudeRef=00
  EXIF_GPSDateStamp=2018.05.28
  EXIF_GPSLatitude=(48) (19) (36.461)
  EXIF_GPSLatitudeRef=N
  EXIF_GPSLongitude=(15) (53) (26.604)
  EXIF_GPSLongitudeRef=E
  EXIF_GPSMapDatum=WGS-84
  EXIF_GPSTimeStamp=(15) (23) (34)
Image Structure Metadata:
  INTERLEAVE=BAND
Corner Coordinates:
Upper Left  (    0.0,    0.0)
Lower Left  (    0.0,  512.0)
Upper Right (  640.0,    0.0)
Lower Right (  640.0,  512.0)
Center      (  320.0,  256.0)
Band 1 Block=640x6 Type=UInt16, ColorInterp=Gray

EXIFTOOL

ExifTool Version Number         : 10.20
File Name                       : dest.tif
Directory                       : .
File Size                       : 642 kB
File Modification Date/Time     : 2018:08:03 17:39:11+02:00
File Access Date/Time           : 2018:08:03 17:39:55+02:00
File Inode Change Date/Time     : 2018:08:03 17:39:11+02:00
File Permissions                : rw-r--r--
File Type                       : TIFF
File Type Extension             : tif
MIME Type                       : image/tiff
Exif Byte Order                 : Little-endian (Intel, II)
Image Width                     : 640
Image Height                    : 512
Bits Per Sample                 : 16
Compression                     : Uncompressed
Photometric Interpretation      : BlackIsZero
Strip Offsets                   : (Binary data 585 bytes, use -b option to extract)
Samples Per Pixel               : 1
Rows Per Strip                  : 6
Strip Byte Counts               : (Binary data 429 bytes, use -b option to extract)
Planar Configuration            : Chunky
Sample Format                   : Unsigned
GDAL Metadata                   : <GDALMetadata>.  <Item name="EXIF_GPSAltitude">(309.77)</Item>.  <Item name="EXIF_GPSAltitudeRef">00</Item>.  <Item name="EXIF_GPSDateStamp">2018.05.28</Item>.  <Item name="EXIF_GPSLatitude">(48) (19) (36.461)</Item>.  <Item name="EXIF_GPSLatitudeRef">N</Item>.  <Item name="EXIF_GPSLongitude">(15) (53) (26.604)</Item>.  <Item name="EXIF_GPSLongitudeRef">E</Item>.  <Item name="EXIF_GPSMapDatum">WGS-84</Item>.  <Item name="EXIF_GPSTimeStamp">(15) (23) (34)</Item>.</GDALMetadata>.
Image Size                      : 640x512
Megapixels                      : 0.328

Results ./store-md source.tif dest.tif N Y

Now the function call GDALSetMetadataItem(h_ddst, key, value, "EXIF"); used. A new empty domain was created and the tags and gdalinfo reports them neither in the EXIF domain nor in the default domain.

 gdalinfo -mdd EXIF dest.tif
Driver: GTiff/GeoTIFF
Files: dest.tif
Size is 640, 512
Coordinate System is `'
Image Structure Metadata:
  INTERLEAVE=BAND
Corner Coordinates:
Upper Left  (    0.0,    0.0)
Lower Left  (    0.0,  512.0)
Upper Right (  640.0,    0.0)
Lower Right (  640.0,  512.0)
Center      (  320.0,  256.0)
Band 1 Block=640x6 Type=UInt16, ColorInterp=Gray

Results ./store-md source.tif dest.tif Y Y

To check, if the prefix EXIF_ like in EXIF_GPSAltitude produces invalid tags for the EXIF-IFD, I've removed this prefix and try to store the data in the EXIF domain. But nothing is written to the EXIF-IFD.

Conclusion

At the moment I can not reproduce the transfer of EXIF tags to the EXIF-IFD from a proper source GTiff to a copy of that image.

Operating system

Linux: 3.16.0-6-amd64 #1 SMP Debian 3.16.57-2 (2018-07-14) x86_64 GNU/Linux
Debian 8 - Jessie

GDAL version and provenance

GDAL Version 2.0.2 version compiled from source

[rouault]

Perhaps you got confused by the formulation of the doc, but there's no write support for EXIF in the GeoTIFF driver. This is read-only for now.

[bigopensky]

But what is meant with the terminus "Starting with GDAL 1.10, EXIF metadata can be extracted from the file, and will be stored the EXIF metadata domain", not the EXIF-IFD?

[rouault]

That the EXIF metadata read from the file is exposed in the EXIF metadata domain. Is "exposed" less ambiguous than "stored" ? (I see there was a typo and the 'in' between 'stored' and 'the' was missing)

$ gdalinfo exif_and_gps.tif -mdd EXIF
Driver: GTiff/GeoTIFF
Files: exif_and_gps.tif
Size is 1, 1
Coordinate System is `'
Metadata:
  TIFFTAG_RESOLUTIONUNIT=2 (pixels/inch)
  TIFFTAG_XRESOLUTION=72
  TIFFTAG_YRESOLUTION=72
Metadata (EXIF):
  EXIF_ComponentsConfiguration=0x01 0x02 0x03 0x00
  EXIF_ExifVersion=0220
  EXIF_GPSLatitude=(77) (5) (60)
  EXIF_GPSLatitudeRef=S
  EXIF_GPSLongitude=(34) (12) (0)
  EXIF_GPSLongitudeRef=E
  EXIF_GPSVersionID=0x02 0x02 0x00 0x00
  EXIF_SpectralSensitivity=EXIF Spectral Sensitivity
Image Structure Metadata:
  INTERLEAVE=PIXEL
Corner Coordinates:
Upper Left  (    0.0,    0.0)
Lower Left  (    0.0,    1.0)
Upper Right (    1.0,    0.0)
Lower Right (    1.0,    1.0)
Center      (    0.5,    0.5)
Band 1 Block=1x1 Type=Byte, ColorInterp=Red
Band 2 Block=1x1 Type=Byte, ColorInterp=Green
Band 3 Block=1x1 Type=Byte, ColorInterp=Blue

[bigopensky]

Hm, for my understanding "EXIF metadata domain" in storage terms addresses always something in the GDAL Metadata tag coded with

<GDALMetadata>.  
  <Item name="EXIF_...">VALUE</Item>
</GDALMetadata>

right?

I think no subsequent running program can handle that format. Is there a work around to address the EXIF IFD directly?

[rouault]

right?

no. The EXIF metadata domain is completely independent from the default "" metadata domain.

[bigopensky]

Hm, Is there a docmentation, that goes beyond the gdal docu and class description. At the moment I'm screening the sources and look how it was implemented (gdal_translate and the -mo switch or the implentation of the GTiff datasource). But cannot see where in the "big picture" **metadata domain ** belongs to. So at least, what is addressed with the terminus metadata domain, any kind of an existing and documented IFD or GDAL internal coded schemas.

[rouault]

See https://gdal.org/gdal_datamodel.html , https://github.com/OSGeo/gdal/blob/master/gdal/gcore/gdalmultidomainmetadata.cpp

[bigopensky]

Ok, I suppose it handles metadata in the gdal coded XML schema addressing the domain ""

int GDALMultiDomainMetadata::XMLInit( CPLXMLNode *psTree, int /* bMerge */ ) {
 ...
}

CPLXMLNode *GDALMultiDomainMetadata::Serialize() {
...
   CPLXMLNode *psMD = CPLCreateXMLNode( nullptr, CXT_Element, "Metadata" );
...
}

and that means, metadata are more or less one-way-in-stuff. If you want write something to the EXIF domain in a certain standard manner (EXIF IFD / GPS IFD), it has to be ported on driver level (beside the abstraction level). So I think, the terminus metadata is ambiguous because it is not clear what the (abstracted/generalized diver) addresses or where the info is stored, if you use the procedure
GDALSetMetadataItem(h_dataset, key, value,"SOME IFD NAME").

May be it is a GDAL feature request, because the processing of aerial data is often "pinned down" to industrial standards containing EXIF and GPS IFD capabilities..

But this does not belongs into an issue list.

Thank you

[rouault]

See also the GTiffDataset::SetMetadata() and SetMetadataItem() methods of the GTiff driver

[bigopensky]

Yes, the but test program shows that GTiffDataset::SetMetadata() and SetMetadataItem() cannot address the EXIF IFD. Sorry for bothering and for the miss understandings. I will go on with a workaround using Phil Harveys exiftool.

Regards huck

[AndreasLuckert]

Dear community,

I'm using Linux Lubuntu 18.04 with exiftool command line in order to modify TIFF-Tags of a land cover classification GeoTIFF-file.

As the Python libraries

import exifread
import PIL
import tifffile as tf
from skimage.external import tifffile as sk_tf

don't even list the GeoTiff tags, as they are "blind on that eye", I found that exiftool lists, but cannot change them.

The output of my example file, using exiftool in the terminal, is:
Input:
exiftool -D -G -a -u -U -f "newfile.tif"
Output:

[ExifTool]          - ExifTool Version Number         : 10.80
[File]              - File Name                       : newfile.tif
[File]              - Directory                       : .
[File]              - File Size                       : 1503 kB
[File]              - File Modification Date/Time     : 2019:12:19 17:32:17+01:00
[File]              - File Access Date/Time           : 2019:12:19 17:32:17+01:00
[File]              - File Inode Change Date/Time     : 2019:12:19 17:32:17+01:00
[File]              - File Permissions                : rw-rw-r--
[File]              - File Type                       : TIFF
[File]              - File Type Extension             : tif
[File]              - MIME Type                       : image/tiff
[File]              - Exif Byte Order                 : Little-endian (Intel, II)
[File]              - Current IPTC Digest             : 79ffcf282ca6974ff99640a7421b40b7
[EXIF]            256 Image Width                     : 1148
[EXIF]            257 Image Height                    : 1337
[EXIF]            258 Bits Per Sample                 : 8
[EXIF]            259 Compression                     : Uncompressed
[EXIF]            262 Photometric Interpretation      : RGB Palette
[EXIF]            273 Strip Offsets                   : (Binary data 1390 bytes, use -b option to extract)
[EXIF]            274 Orientation                     : Horizontal (normal)
[EXIF]            277 Samples Per Pixel               : 1
[EXIF]            278 Rows Per Strip                  : 7
[EXIF]            279 Strip Byte Counts               : (Binary data 954 bytes, use -b option to extract)
[EXIF]            282 X Resolution                    : 1
[EXIF]            283 Y Resolution                    : 1
[EXIF]            284 Planar Configuration            : Chunky
[EXIF]            296 Resolution Unit                 : None
[EXIF]            305 Software                        : IMAGINE TIFF Support.Copyright 1991 - 1999 by ERDAS, Inc. All Rights Reserved.@(#)$RCSfile: etif.c $ $Revision: 1.11 $ $Date$
[EXIF]            320 Color Map                       : (Binary data 1536 bytes, use -b option to extract)
[EXIF]            339 Sample Format                   : Unsigned
[EXIF]          33550 Pixel Scale                     : 30 30 0
[EXIF]          33922 Model Tie Point                 : 0 0 0 1514925 1583985 0
[IPTC]             25 Keywords                        : word
[IPTC]              0 Application Record Version      : 4
[GeoTiff]           1 Geo Tiff Version                : 1.1.0
[GeoTiff]        1024 GT Model Type                   : Projected
[GeoTiff]        1025 GT Raster Type                  : Pixel Is Area
[GeoTiff]        1026 GT Citation                     : IMAGINE GeoTIFF Support.Copyright 1991 - 2001 by ERDAS, Inc. All Rights Reserved.@(#)$RCSfile: egtf.c $ $Revision: 1.11.2.3 $ $Date: 2004/11/24 09:12:56EST $.Projection Name = USA_Contiguous_Albers_Equal_Area_Conic_USGS_version.Units = meters.GeoTIFF Units = meters
[GeoTiff]        2048 Geographic Type                 : NAD83
[GeoTiff]        3072 Projected CS Type               : User Defined
[GeoTiff]        3073 PCS Citation                    : IMAGINE GeoTIFF Support.Copyright 1991 - 2001 by ERDAS, Inc. All Rights Reserved.@(#)$RCSfile: egtf.c $ $Revision: 1.11.2.3 $ $Date: 2004/11/24 09:12:56EST $.Projection = Albers Conical Equal Area
[GeoTiff]        3074 Projection                      : User Defined
[GeoTiff]        3075 Proj Coord Trans                : Albers Equal Area
[GeoTiff]        3076 Proj Linear Units               : Linear Meter
[GeoTiff]        3078 Proj Std Parallel 1             : 29.5
[GeoTiff]        3079 Proj Std Parallel 2             : 45.5
[GeoTiff]        3081 Proj Nat Origin Lat             : 23
[GeoTiff]        3082 Proj False Easting              : 0
[GeoTiff]        3083 Proj False Northing             : 0
[GeoTiff]        3088 Proj Center Long                : -96
[Composite]         - Image Size                      : 1148x1337
[Composite]         - Megapixels                      : 1.5

Now, I'd like to change/delete the GeoTiff-TAG "Projection", which throws the following warning message without changing anything:
Input:
exiftool "-Projection=" "newfile.tif"
Output:
Warning: Sorry, Projection is not writable. Nothing to do.

Next, I found the following documentation of this very webpage:
https://exiftool.org/TagNames/GeoTiff.html

It states, among other details, the following:

GeoTIFF tags are not writable individually, but they may be copied en mass via the block tags GeoTiffDirectory, GeoTiffDoubleParams and GeoTiffAsciiParams.

Is there any possibility to change/rename/delete a GeoTiff-TAG of choice via command line exiftools (as stated above in my attempt), or gdal etc.?
I'm quite desparate already, since I've invested a lot of time in finding a possiblity, but so far no avail.

Thanks in advance for helping me out.
Kind regards,
Andreas