Add ability to (over)write CRS for an existing raster file e.g. to geotif or to .aux.xml file for jpg + jgw

[qgib]

Author Name: Alister Hood (@AlisterH)
Original Redmine Issue: 4236

Redmine category:rasters

---

PROBLEM AND SUGGESTED SOLUTION

• .prj or .qpj files are not supported for raster formats

• Some raster formats (e.g. jpeg, bitmap) cannot include coordinate or projection information. These formats are typically used with world files to provide the coordinate information, in which case QGIS prompts for the CRS when the layer is added.

• as well as a world file, some raster files georeferenced by ArcGIS include projection information in a PAM (.aux.xml) file. QGIS supports this information (QGIS does not prompt for CRS if it is present).

In the case where:

1. a raster file did not come with projection information, or
2. a raster came with incorrect projection information, which the user has overriden in QGIS by setting another CRS,
   it would be good if QGIS could write the CRS to the file itself (if supported, e.g. for geotif), or to a .aux.xml file.

Possibly the best solution would be for QGIS to write the CRS information whenever the CRS is set for the layer in QGIS (so immediately after the layer is loaded in the first case). If this happened automatically it would be undesirable (as per comment 8 below), especially so if the "CRS for new layers" option is not set to "Prompt for CRS". But it would probably be fine if the user was prompted e.g. "Do you want to write this CRS to file?", or if there was a button in the "Set layer CRS" dialog to "write selected CRS to file". Personally I think a button in the "Set layer CRS" dialog would be best.

I also think it would be worth somehow indicating in the layers panel that layers have had their CRS set in QGIS but not written to file, although I am aware of the issues around "gui clutter" in the layers panel.

ASSOCIATED PROBLEMS WHICH COULD BE AVOIDED

I suspect there are quite a lot of features and tools which don't work correctly using layers which do not come with the correct projection information. At least some of these don't produce a helpful error message.

e.g. there are a number of places on the internet where people have encountered the following error when trying to clip a raster which comes with a world file but no projection information, using the gdaltools "clip raster by mask layer" option to "Crop the extent of the target dataset to take the nodata value from input":

GDAL command output:
ERROR 1: Cannot compute bounding box of cutline

In some cases people have figured out that they can work around the problem by saving to a new geotif first. In other cases they have given up on the "clip raster by mask layer" tool and looked for alternatives or changed their workflow. Often a number of other people have wasted their time trying to work out what is going out.

I realise the error message comes from upstream, but this kind of problem where a simple task fails without a clear error message leaves the impression that QGIS is unreliable i.e. try something and it might fail randomly, but if you are lucky it will work!

The suggested feature could prevent a lot of these problems and avoid the associated perception of QGIS being unreliable.

KNOWN WORKAROUNDS

See comment #7 for how to write the CRS information for a file using gdal_edit.py

[qgib]

Author Name: Alister Hood (@AlisterH)

---

Just for information, here is an example .aux.xml which defines CRS:

  <Metadata domain="xml:ESRI" format="xml">
    <GeodataXform xsi:type="typens:IdentityXform" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:typens="http://www.esri.com/schemas/ArcGIS/9.2">
      <SpatialReference xsi:type="typens:ProjectedCoordinateSystem">
        <WKT>PROJCS[&quot;GD_1949_New_Zealand_Map_Grid&quot;,GEOGCS[&quot;GCS_New_Zealand_1949&quot;,DATUM[&quot;D_New_Zealand_1949&quot;,SPHEROID[&quot;International_1924&quot;,6378388.0,297.0]],PRIMEM[&quot;Greenwich&quot;,0.0],UNIT[&quot;Degree&quot;,0.0174532925199433]],PROJECTION[&quot;New_Zealand_Map_Grid&quot;],PARAMETER[&quot;False_Easting&quot;,2510000.0],PARAMETER[&quot;False_Northing&quot;,6023150.0],PARAMETER[&quot;Central_Meridian&quot;,173.0],PARAMETER[&quot;Latitude_Of_Origin&quot;,-41.0],UNIT[&quot;Meter&quot;,1.0]]</WKT>
        <XOrigin>-149973.520044524</XOrigin>
        <YOrigin>-339656.819984179</YOrigin>
        <XYScale>1052125299.09217</XYScale>
        <ZOrigin>0</ZOrigin>
        <ZScale>8192</ZScale>
        <MOrigin>-100000</MOrigin>
        <MScale>10000</MScale>
        <XYTolerance>0.001</XYTolerance>
        <ZTolerance>2</ZTolerance>
        <MTolerance>2</MTolerance>
        <HighPrecision>true</HighPrecision>
      </SpatialReference>
    </GeodataXform>
  </Metadata>
</PAMDataset>

[qgib]

Author Name: Giovanni Manghi (@gioman)

---

• fixed_version_id was configured as Version 1.7.4

[qgib]

Author Name: Giovanni Manghi (@gioman)

---

• fixed_version_id was changed from Version 1.7.4 to Version 2.0.0

[qgib]

Author Name: Pirmin Kalberer (Pirmin Kalberer)

---

• fixed_version_id was changed from Version 2.0.0 to Future Release - Nice to have

[qgib]

Author Name: Miroslav Umlauf (@mirouml)

---

It actually would be nice to have. And it shouldn't be that hard. When I am creating JPGs from TIF with -co WORDFILE=YES it actually creates *.wld file plus .aux.xml with:

<PAMDataset>
  <SRS>PROJCS["WGS 84 / Pseudo-Mercator",...]</SRS>
</PAMDataset>

JPGs created this way are opened by QGIS straight (QGIS does not prompt for CRS).

[qgib]

Author Name: Giovanni Manghi (@gioman)

---

• easy_fix was configured as 0

[qgib]

Author Name: Alister Hood (@AlisterH)

---

Miroslav Umlauf wrote:

And it shouldn't be that hard.

Correct - e.g. you can write the CRS information like this:

@gdal_edit.py -ro -a_srs EPSG:2105 "C:\Processing\1.jpg"@

A complicating factor in this case is that the jpeg driver requires -ro:

@gdal_edit.py -stats "C:\Processing\1.jpg"
ERROR 6: The JPEG driver does not support update access to existing datasets.@

But there may be files that don't require -ro, and in those cases I guess it would be best not to use -ro. (By default the gdal driver normally writes statistics when you open a raster in QGIS (or at least used to) - are these always written to a .aux.xml file? I can't figure out how to enable it to test - I thought I'd disabled it by setting GDAL_PAM_ENABLED=NO in the QGIS options, but it isn't showing up there.)

---

• description was changed from - .prj or .qpj files are not supported for raster formats

• Some raster formats (e.g. jpeg, bitmap) cannot include coordinate or projection information. These formats are typically used with world files to provide the coordinate information, in which case QGIS prompts for the CRS when the layer is added.

• as well as a world file, some raster files georeferenced by ArcGIS include projection information in a PAM (.aux.xml) file. QGIS supports this information (QGIS does not prompt for CRS if it is present).

It would be good if QGIS could create .aux.xml files containing CRS information, probably after prompting for CRS when a raster layer is added.

N.B. I guess maybe this functionality would require improvements to Gdal. to - .prj or .qpj files are not supported for raster formats

• Some raster formats (e.g. jpeg, bitmap) cannot include coordinate or projection information. These formats are typically used with world files to provide the coordinate information, in which case QGIS prompts for the CRS when the layer is added.

• as well as a world file, some raster files georeferenced by ArcGIS include projection information in a PAM (.aux.xml) file. QGIS supports this information (QGIS does not prompt for CRS if it is present).

It would be good if QGIS could create .aux.xml files containing CRS information, probably after prompting for CRS when a raster layer is added.

N.B. I guess maybe this functionality would require improvements to Gdal.

[qgib]

Author Name: Alister Hood (@AlisterH)

---

Alister Hood wrote:

It would be good if QGIS could create .aux.xml files containing CRS information, probably after prompting for CRS when a raster layer is added.

Thinking about it, writing it when loading a layer would be bad, as people probably often guess a CRS, to see if the raster shows up in the right place, and they wouldn't want it written if they'd guessed the wrong CRS.
So I guess it would be better to provide an option to "Save CRS with layer", either from the right-click menu in the TOC (I'm sure this wouldn't be accepted, on the basis that the menu is busy), or in the coordinate reference system selector.

[qgib]

Author Name: Alister Hood (@AlisterH)

---

Also see #14783 and #14082

[qgib]

Author Name: Alister Hood (@AlisterH)

---

Alister Hood wrote:

So I guess it would be better to provide an option to "Save CRS with layer"

To be clear, I meant to suggest a button, rather than a checkbox or something.

[qgib]

Author Name: Alister Hood (@AlisterH)

---

Alister Hood wrote:

But there may be files that don't require -ro, and in those cases I guess it would be best not to use -ro. (By default the gdal driver normally writes statistics when you open a raster in QGIS (or at least used to) - are these always written to a .aux.xml file? I can't figure out how to enable it to test - I thought I'd disabled it by setting GDAL_PAM_ENABLED=NO in the QGIS options, but it isn't showing up there.)

Ah, of course - I don't have .aux.xml generation disabled currently, I'd just forgotten that they aren't actually written until the layer is removed or the project closed.

Testing opening a tiff without embedded statistics in QGIS, I see the statistics are written to a .aux.xml file. This seems sub-optimal if I can write statistics to the actual tiff using gdal_edit, but it does mean that if we always wrote CRS to a .aux.xml we would be consistent.

[qgib]

Author Name: Alister Hood (@AlisterH)

---

Incidentally, @gdal_edit -ro -a_srs ...@ doesn't do anything for a tiff, and if the tiff file is read-only @ gdal_edit -ro -a_srs...@ fails silently.

[qgib]

Author Name: Alister Hood (@AlisterH)

---

• description was changed from - .prj or .qpj files are not supported for raster formats

• Some raster formats (e.g. jpeg, bitmap) cannot include coordinate or projection information. These formats are typically used with world files to provide the coordinate information, in which case QGIS prompts for the CRS when the layer is added.

• as well as a world file, some raster files georeferenced by ArcGIS include projection information in a PAM (.aux.xml) file. QGIS supports this information (QGIS does not prompt for CRS if it is present).

It would be good if QGIS could create .aux.xml files containing CRS information, probably after prompting for CRS when a raster layer is added.

N.B. I guess maybe this functionality would require improvements to Gdal. to PROBLEM AND SUGGESTED SOLUTION

• .prj or .qpj files are not supported for raster formats

• Some raster formats (e.g. jpeg, bitmap) cannot include coordinate or projection information. These formats are typically used with world files to provide the coordinate information, in which case QGIS prompts for the CRS when the layer is added.

• as well as a world file, some raster files georeferenced by ArcGIS include projection information in a PAM (.aux.xml) file. QGIS supports this information (QGIS does not prompt for CRS if it is present).

In the case where:

1. a raster file did not come with projection information, or
2. a raster came with incorrect projection information, which the user has overriden in QGIS by setting another CRS,
   it would be good if QGIS could write the CRS to the file itself (if supported, e.g. for geotif), or to a .aux.xml file.

Possibly the best solution would be for QGIS to write the CRS information whenever the CRS is set for the layer in QGIS (so immediately after the layer is loaded in the first case). If this happened automatically it would be undesirable (as per comment 8 below), especially so if the "CRS for new layers" option is not set to "Prompt for CRS". But it would probably be fine if the user was prompted e.g. "Do you want to write this CRS to file?", or if there was a button in the "Set layer CRS" dialog to "write selected CRS to file". Personally I think a button in the "Set layer CRS" dialog would be best.

I also think it would be worth somehow indicating in the layers panel that layers have had their CRS set in QGIS but not written to file, although I am aware of the issues around "gui clutter" in the layers panel.

ASSOCIATED PROBLEMS WHICH COULD BE AVOIDED

I suspect there are quite a lot of features and tools which don't work correctly using layers which do not come with the correct projection information. At least some of these don't produce a helpful error message.

e.g. there are a number of places on the internet where people have encountered the following error when trying to clip a raster which comes with a world file but no projection information, using the gdaltools "clip raster by mask layer" option to "Crop the extent of the target dataset to take the nodata value from input":

GDAL command output:
ERROR 1: Cannot compute bounding box of cutline

In some cases people have figured out that they can work around the problem by saving to a new geotif first. In other cases they have given up on the "clip raster by mask layer" tool and looked for alternatives or changed their workflow. Often a number of other people have wasted their time trying to work out what is going out.

I realise the error message comes from upstream, but this kind of problem where a simple task fails without a clear error message leaves the impression that QGIS is unreliable i.e. try something and it might fail randomly, but if you are lucky it will work!

The suggested feature could prevent a lot of these problems and avoid the associated perception of QGIS being unreliable.

KNOWN WORKAROUNDS

See comment #7 for how to write the CRS information for a file using gdal_edit.py

• subject was changed from Write CRS to .aux.xml file when adding raster with no embedded CRS information to Add ability to (over)write CRS for an existing raster file e.g. to geotif or to .aux.xml file for jpg + jgw

[qgib]

Author Name: Alister Hood (@AlisterH)

---

I have rewritten the description including adding an example of the less obvious benefits of this feature i.e. one of the frustrating problems it would help to avoid.

[qgib]

Author Name: Alister Hood (@AlisterH)

---

Re the suggested workaround to use gdal_edit, I see that this is now available in Processing, as "Assign Projection". Unfortunately it doesn't allow editing the command to use -ro, so it is no use for jpeg layers.

[alexbruy]

Possible with Processing tool "Assign Projection".

[AlisterH]

Yes, but as I said, it doesn't allow using the -ro option for jpeg or any other formats that require it.

[AlisterH]

Also, it requires you to manually choose the CRS again - it would be good to have a tool that will assign the CRS that you have already set in QGIS.

[mirouml]

Oh my, so long outstanding issues like this make me want advance my QGIS experience to the level I am able to fix it myself, and return more to community around QGIS. I am probably capable to put together some very lame plugin for this.

@AlisterH Maybe you can ask on gis stackexchange if anyone has something like this already?

[AlisterH]

Feel free to ask around, but I imagine if someone had written a plugin for it they would have made it available in the plugin repository. Please note, this is the key point:

it would be good to have a tool that will assign the CRS that you have already set in QGIS