QGIS fails to read undefined projection from user datum in shape.prj file

[qgib]

Author Name: neteler-itc-it - (neteler-itc-it -)
Original Redmine Issue: 418

Redmine category:projection_support
Assignee: Magnus Homann

---

Hi,

when opening Italian maps, we usually have to define a "user" datum since PROJ4 doesn't contain the definitions (as there are 3 datums for the same EPSC code in Italy, depending on where you are). The [[QgsSpatialRefSys]] detection fails, however:

qgis ammprvBL.shp
Warning: [[QgsSpatialRefSys]]::getRecord failed : select * from tbl_srs where parameters='+proj=tmerc +lat_0=0 +lon_0=9 +k=0.999600 +x_0=1500000 +y_0=0 +ellps=intl +towgs84=-104.1,-49.1,-9.9,0.971,-2.917,0.714,-11.68 +units=m +no_defs'

If I take out the towgs84 part from the .prj file, it works.
It would be great to make QGIS tolerant for user defined towgs84 parameters which is rather the standard in Europe (many countries, many datums).

Proposal as general solution:
Maybe the towgs84 part could be temporarily taken out for the SQL search and re-added after the getRecord request was done.

Best,
Markus

PS: Attached the .prj file content

---

• ammprvBL.prj (neteler-itc-it -)

[qgib]

Author Name: Martin Dobias (@wonder-sk)

---

This is not that trivial as it looked like. Let's explain what happens when QGIS tries to match a projection with its spatial reference database:

• check whether projection's name is in user's or global database; if not
• check whether the whole proj4 string is in user's or global database; if not
• try to find heuristically by traversing database and using OGRSpatialReference::IsSame()

In your case first two options don't find equivalent in database, but third does and tells QGIS that it's the same projection as "+proj=tmerc +lat_0=0 +lon_0=9 +k=0.999600 +x_0=1500000 +y_0=0 +ellps=intl +units=m +no_defs". The problem is that OGR's [IsSame] function is not robust enough to make a difference between this and the specified ones. (See ogrspatialreference.cpp in OGR if interested)

To get it working correctly you have to add it as a custom projection (menu Settings -> Custom projection). If [IsSame] worked correctly, it wouldn't recognize any projection and you would be asked to select one from database or a default one would be used (depending on settings).

Thus I'm closing this ticket as it seems that we're unable to fix it. ... probably this should be reported to OGR project...?

---

• resolution was changed from to wontfix
• status_id was changed from Open to Closed

[qgib]

Author Name: neteler-itc-it - (neteler-itc-it -)

---

You said that "The problem is that OGR's [IsSame] function is not robust enough to make a difference between this and the specified ones." In fact, there is close to no difference besides the towgs84 part which would "just" be needed to be transferred to the projection found by QGIS in it's DB.

I don't know if failing on that completely and not recognize any projection
is the best solution here. I think that the [IsSame] heuristics work quite
well. Substracting the input and the output (from [IsSame]) projection string will show just towgs84 as difference which is perfect.

Best,
Markus

[qgib]

Author Name: Martin Dobias (@wonder-sk)

---

Well, it seems that towgs84 parameter is quite important. Maybe in this case there's really small difference. But when looking to the QGIS's database of projections, there are (many) projections that specify towgs84 parameter, so I guess that it might be quite important - removing it temporarily would be incorrect.

A solution might be to use original proj string even if another projection has been declared as the same. But I don't know if this won't break other things or create some inconsistencies.

[qgib]

Author Name: neteler-itc-it - (neteler-itc-it -)

---

Hi,

I think we have a missunderstanding here. I meant to say
that the towgs84 parameter should be taken out while
parsing the string (to match the QGIS SRS DB), then
reinserted, after having found the matching EPSG (which usually lacks the towgs84 parameter.

In case that a towgs84 parameter is already present in the QGIS SRS DB, it should use the user provided one (maybe pop up a notice that it was doing so).

Hope that's clarified now what I tried to say.

Markus

[qgib]

Author Name: Martin Dobias (@wonder-sk)

---

Replying to [comment:5 neteler@itc.it]:

I think we have a missunderstanding here. I meant to say
that the towgs84 parameter should be taken out while
parsing the string (to match the QGIS SRS DB), then
reinserted, after having found the matching EPSG (which usually lacks the towgs84 parameter.

I understand your idea. But the problem is in the way how QGIS copes with spatial references. Once proj4 string is read, QGIS searches for a SRS ID in SRS database. SRS ID is unique for every projection. Thus even if two projection are nearly same (e.g. with or without towgs84) they must have different SRS ID to enable QGIS understand the difference.

That's why taking out towgs84 temporiraly makes no sense - another SRS ID would be chosen instead.

Rescue from this situation is to add custom projection. Once added, your projection has its own SRS ID and will be detected correctly.

Martin

[qgib]

Author Name: neteler-itc-it - (neteler-itc-it -)

---

Hi Martin,

I am again falling into this trap:

Warning: [[QgsSpatialRefSys]]::getRecord failed : select * from tbl_srs where parameters='+proj=utm +zone=32 +ellps=intl +towgs84=-87.000,-98.000,-121.000 +units=m +no_defs'

Displaying these two maps with on-the-fly projection activated, they are shifted by around 100m:

PROJCS[[UTM Zone 32 Northern Hemisphere"GEOGCS["international"DATUM["D_European_1950"SPHEROID["International_1924"6378388297]TOWGS84[-87000-98000-121000]]PRIMEM["Greenwich"0]UNIT["Degree"0017453292519943295]]PROJECTION["Transverse_Mercator"]PARAMETER["latitude_of_origin"0]PARAMETER["central_meridian"9]PARAMETER["scale_factor"09996]PARAMETER["false_easting"500000]PARAMETER["false_northing"0]UNIT["Meter]]

PROJCS[[Transverse Mercator"GEOGCS["international"DATUM["D_Monte_Mario"SPHEROID["international"6378388297]]PRIMEM["Greenwich"0]UNIT["Degree"0017453292519943295]]PROJECTION["Transverse_Mercator"]PARAMETER["latitude_of_origin"0]PARAMETER["central_meridian"9]PARAMETER["scale_factor"09996]PARAMETER["false_easting"1500000]PARAMETER["false_northing"0]UNIT["Meter]]

In my opinion at least a visible warning should pop up that reading the .proj file failed.

thanks,
Markus

[qgib]

Author Name: Redmine Admin (Redmine Admin)

---

Hi

Please consider Markus request. QGIS silent failure to assign a datum shift is really bad. If, instead, a warning would pop up with info that datum was not recognized and the user is asked to define the CRS manually himself to workaround it, this would be a great help.

I just had this issue too, working with archival topo maps of Germany, for which the 584.8,67.0,400.3,0.105,0.013,-2.378,10.29 datum shift from Bessel 1841 to WGS84 was not recognized by QGIS and it silently fell to no datum shift at all. This has resulted in an over 250m error. For an unexperienced user this might mean real trouble.

Maciek

---

• resolution was changed from wontfix to
• status_id was changed from Closed to Feedback

[qgib]

Author Name: Magnus Homann (@homann)

---

This can't be done unless we re-do the entire projections system, earliest at 0.9.

[qgib]

Author Name: Markus Neteler (Markus Neteler)

---

In current SVN trunk i(pre 1.0.0) t continues to fail:

qgis viapri.shp
Python support ENABLED :-)
Loading python plugins
Loaded : Plugin installer (package: plugin_installer)
Critical: [[QgsSpatialRefSys]]::createFromWkt -- theWkt is uninitialised, operation failed
Warning: [[QgsSpatialRefSys]]::getRecord failed :  select * from tbl_srs where parameters='+proj=tmerc +lat_0=0 +lon_0=9 +k                                               =0.9996 +x_0=1500000 +y_0=0 +ellps=intl +towgs84=-104.1,-49.1,-9.9,0.971,-2.917,0.714,-11.68 +units=m +no_defs'
Warning: [[QgsSpatialRefSys]]::findMatchingProj will only work if prj acr ellipsoid acr and proj4string are set!...

It works when I manually remove the TOWGS84 string from the WKT (.prj) file.

Suggestion: Ignore the TOWGS84 string when matching the projection, and issue a warning (not a complete failure) that TOWGS84 differs.

Note that most (in theory all) Italian QGIS users have this problem. Likewise for other countries.

thanks,
Markus

[qgib]

Author Name: Magnus Homann (@homann)

---

Is this error still present? Another towgs bug has been fixed, I believe.

---

• status_id was changed from Feedback to Open

[qgib]

Author Name: Magnus Homann (@homann)

---

Talked to timlinux, and the idea with have is if the specfied projection does not have an exact match (including +towgs84= )in the database of projections, it will ask the user if he/she wants to create a custom projection.

Do you think that will solve the issue? This wont happen until 1.1.0 I'm afraid, needs GUI changes and has potential for large changes of projection code needed.

---

• status_id was changed from Open to In Progress

[qgib]

Author Name: Markus Neteler (Markus Neteler)

---

Replying to [comment:14 homann]:

Talked to timlinux, and the idea with have is if the specfied projection does not have an exact match (including +towgs84= )in the database of projections, it will ask the user if he/she wants to create a custom projection.

Do you think that will solve the issue?

I think so. But:

This wont happen until 1.1.0 I'm afraid, needs GUI changes and has potential for large changes of projection code needed.

Understood. But as suggested above: Please issue at least a warning that TOWGS84 differs (if not happening already).

Markus

[qgib]

Author Name: Magnus Homann (@homann)

---

I'm sorry, but the warning is still a GUI change. That towgs differs is just a special case of the fact that the projections differs. As soons as GUI freeze is over i'll try to fix this.

[qgib]

Author Name: Paolo Cavallini (@pcav)

---

There is a problem in PROJ4: there are several datum for EPSG 3003 (and
others), but in PROJ4 the epsg file does not allow a relation 1:n with
present format (thanks Markus for clarifying this).
Currently, users need to define their special SRS, which is cumbersome
and scary for newbies. GRASS solved it recently:
http://grass.osgeo.org/screenshots/gui.php#wxpython
-> Location wizard (geodetic datum selection for EPSG 3003 - Italy mainland)
Would it be feasible to copy the approach and port it to QGIS?

[qgib]

Author Name: Magnus Homann (@homann)

---

Automatic addition of a custom CRS for unknown to QGIS - but valid - CRS:es from e.g WKT in 93d5bd2 (SVN r11367). If there are any different problems from Italian qgissers, please open a new ticket.

---

• resolution was changed from to fixed
• status_id was changed from In Progress to Closed