Allow custom proj operation definitions when transforming between two CRSes

[MichelRuijter]

I am trying to define a custom CRS. I have tested this CRS in OSGEO and I am able to put this CRS in the adapted CRS screen. It works for the test coordinates. When I press "OK" Qgis replies that the CRS is not valid: See also attached screenshot

cct +proj=pipeline +step +proj=cart +ellps=WGS84 +step +proj=helmert +convention=coordinate_frame +x=194.245 +y=-154.53 +z=102.676 +rx=351.5046 +ry=-701.841 +rz=251.9889 +s=6.296557 +exact +step +proj=cart +inv +ellps=intl +step +proj=tmerc +units=m +lat_0=17.63621319 +lon_0=-63.23752013 +x_0=4714.87 +y_0=1967.19 +k_0=1.0

Describe the bug

How to Reproduce

QGIS and OS versions

Additional context

[nyalldawson]

Can you confirm your PROJ library version number?

[MichelRuijter]

Nyall, I use QGIS 3.6.1 and as far as I can see QGIS uses a similar (identical) share\proj files as OSGEO. Kind regards, Michel <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> Virusvrij. www.avg.com <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> Op di 6 aug. 2019 om 08:34 schreef Nyall Dawson <notifications@github.com>:

…

Can you confirm your PROJ library version number? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#31106?email_source=notifications&email_token=AMTW4UEZWGUC7P5EQCZVKQDQDELPRA5CNFSM4IJS6CAKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3UBFXY#issuecomment-518525663>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AMTW4UDDNAJVFRH6T3XADGDQDELPRANCNFSM4IJS6CAA> .

-- Michel Ruijter MNRengineering de Kempenaerstraat 82, 2341GP, Oegstgeest mob: +31638944484, tel: +31718897137

[nyalldawson]

You'll need 3.8.1 and a build based off Proj 6.1 in order to use pipeline based CRSes.

[MichelRuijter]

Nyall, Thanks I am going to try that. Michel <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> Virusvrij. www.avg.com <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> Op di 6 aug. 2019 om 08:50 schreef Nyall Dawson <notifications@github.com>:

…

You'll need 3.8.1 and a build based off Proj 6.1 in order to use pipeline based CRSes. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#31106?email_source=notifications&email_token=AMTW4UDT4MP37MTYGCLMHCLQDENJ5A5CNFSM4IJS6CAKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3UCNHI#issuecomment-518530717>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AMTW4UA7V47KZDSLSZOQVGTQDENJ5ANCNFSM4IJS6CAA> .

-- Michel Ruijter MNRengineering de Kempenaerstraat 82, 2341GP, Oegstgeest mob: +31638944484, tel: +31718897137

[MichelRuijter]

Nyall, I just installed "zanzibar" 64 bit version but I still got the proj 4 message. How do I activate proj6? Kind regards, Michel [image: image.png] <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> Virusvrij. www.avg.com <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> Op di 6 aug. 2019 om 08:50 schreef Nyall Dawson <notifications@github.com>:

…

You'll need 3.8.1 and a build based off Proj 6.1 in order to use pipeline based CRSes. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#31106?email_source=notifications&email_token=AMTW4UDT4MP37MTYGCLMHCLQDENJ5A5CNFSM4IJS6CAKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3UCNHI#issuecomment-518530717>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AMTW4UA7V47KZDSLSZOQVGTQDENJ5ANCNFSM4IJS6CAA> .

-- Michel Ruijter MNRengineering de Kempenaerstraat 82, 2341GP, Oegstgeest mob: +31638944484, tel: +31718897137

[nyalldawson]

The windows builds aren't currently using it -- you need to do a custom build for now.

[MichelRuijter]

Nyall, Does that mean that I should start with the OSgeo4W installation? is there a procedure which I should follow? Kind regards, Michel <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> Virusvrij. www.avg.com <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> Op di 6 aug. 2019 om 09:31 schreef Nyall Dawson <notifications@github.com>:

…

The windows builds aren't currently using it -- you need to do a custom build for now. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#31106?email_source=notifications&email_token=AMTW4UGK367SNU27T6ZUWBTQDESFFA5CNFSM4IJS6CAKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3UF64A#issuecomment-518545264>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AMTW4UBFSQBQVEZTL52Q7KLQDESFFANCNFSM4IJS6CAA> .

-- Michel Ruijter MNRengineering de Kempenaerstraat 82, 2341GP, Oegstgeest mob: +31638944484, tel: +31718897137

[nyalldawson]

Windows builds are very tricky, and it's going to be made much trickier because you'd also need to build proj 6. I'd suggest waiting till the official builds upgrade, at which stage this will be supported. In the meantime, it's more or less impossible on Windows.

[jef-n]

Hi Nyall,

On Tue, 06. Aug 2019 at 00:56:00 -0700, Nyall Dawson wrote: Windows builds are very tricky, and it's going to be made much trickier because you'd also need to build proj 6.

There are nightly builds of PROJ in OSGeo4W that can be used to do this (also nightlies of GDAL that depend on the PROJ nightly). But as GRASS doesn't support PROJ 6 yet, so the upgrade didn't take place yet. Jürgen

…

-- Jürgen E. Fischer norBIT GmbH Tel. +49-4931-918175-31 Dipl.-Inf. (FH) Rheinstraße 13 Fax. +49-4931-918175-50 Software Engineer D-26506 Norden https://www.norbit.de QGIS release manager (PSC) Germany IRC: jef on FreeNode norBIT Gesellschaft fuer Unternehmensberatung und Informationssysteme mbH Rheinstrasse 13, 26506 Norden GF: Juergen Fischer, Nils Kutscher HR: Amtsgericht Aurich HRB 100827 Datenschutzerklaerung: https://www.norbit.de/83/

[nyalldawson]

@jef-n

But as GRASS doesn't support PROJ 6 yet, so the upgrade didn't take place yet.

I saw this in another thread, but didn't understand it. PROJ 6 doesn't drop the deprecated proj 4 API, so it should be able to build just by defining the ACCEPT_USE_OF_DEPRECATED_PROJ_API_H macro

[MichelRuijter]

Jurgen, Can you explain to me how I should proceed? Just wait or is there an option to include the proj 6.. options in QGIS? Kind regards, Michel <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> Virusvrij. www.avg.com <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> Op di 6 aug. 2019 om 11:03 schreef Jürgen Fischer <notifications@github.com

…

: Hi Nyall, On Tue, 06. Aug 2019 at 00:56:00 -0700, Nyall Dawson wrote: > Windows builds are very tricky, and it's going to be made much trickier > because you'd also need to build proj 6. There are nightly builds of PROJ in OSGeo4W that can be used to do this (also nightlies of GDAL that depend on the PROJ nightly). But as GRASS doesn't support PROJ 6 yet, so the upgrade didn't take place yet. Jürgen -- Jürgen E. Fischer norBIT GmbH Tel. +49-4931-918175-31 Dipl.-Inf. (FH) Rheinstraße 13 Fax. +49-4931-918175-50 Software Engineer D-26506 Norden https://www.norbit.de QGIS release manager (PSC) Germany IRC: jef on FreeNode norBIT Gesellschaft fuer Unternehmensberatung und Informationssysteme mbH Rheinstrasse 13, 26506 Norden GF: Juergen Fischer, Nils Kutscher HR: Amtsgericht Aurich HRB 100827 Datenschutzerklaerung: https://www.norbit.de/83/ — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#31106?email_source=notifications&email_token=AMTW4UA7GZDXON33P2RY64TQDE465A5CNFSM4IJS6CAKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3UOUWY#issuecomment-518580827>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AMTW4UHZDUFZSN4N4G4PNNDQDE465ANCNFSM4IJS6CAA> .

-- Michel Ruijter MNRengineering de Kempenaerstraat 82, 2341GP, Oegstgeest mob: +31638944484, tel: +31718897137

[MichelRuijter]

Nyall, A couple of month's ago you explained that the proj6 definition would become available within QGIS. I have downloaded since then every new version of QGIS but I still get a CRS error indicating that my CRS is not confirming the proj4 definition. Do I forget something or is proj 6 still not available within the new QGIS? Kind regards, Michel Ruijter Op di 6 aug. 2019 om 09:56 schreef Nyall Dawson <notifications@github.com>:

…

Windows builds are very tricky, and it's going to be made much trickier because you'd also need to build proj 6. I'd suggest waiting till the official builds upgrade, at which stage this will be supported. In the meantime, it's more or less impossible on Windows. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#31106?email_source=notifications&email_token=AMTW4UCGIG42IV2T2NNRFO3QDEVB5A5CNFSM4IJS6CAKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3UIHOY#issuecomment-518554555>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AMTW4UB5BD7DSCKIYX3N77LQDEVB5ANCNFSM4IJS6CAA> .

-- Michel Ruijter MNRengineering de Kempenaerstraat 82, 2341GP, Oegstgeest mob: +31638944484, tel: +31718897137

[nyalldawson]

@MichelRuijter is your build based on proj 6? You can check via the About dialog

[MichelRuijter]

Nyall, I use the win 10 version of QGis 3.10, Coruna and it uses the Proj version 5.2.0. Is there a way (for me) to compile Qgis with the proj6 version? Kind regards, Michel Op di 19 nov. 2019 om 08:18 schreef Nyall Dawson <notifications@github.com>:

…

@MichelRuijter <https://github.com/MichelRuijter> is your build based on proj 6? You can check via the About dialog — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#31106?email_source=notifications&email_token=AMTW4UEX56YWGEZX4LLHCWDQUOHMBA5CNFSM4IJS6CAKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEENEBQQ#issuecomment-555368642>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMTW4UCFULOVTWACNWSI3WLQUOHMBANCNFSM4IJS6CAA> .

-- Michel Ruijter MNRengineering de Kempenaerstraat 82, 2341GP, Oegstgeest mob: +31638944484, tel: +31718897137

[roya0045]

@MichelRuijter Update QGIS, it should come with proj6.

[MichelRuijter]

Alex, Thanks very much the latest version is compiled against proj 6..2.1 I tried within OSGEO 4 the following string: echo -63.25156367 17.61578594 -41.194 |cct +proj=pipeline +step +proj=cart +ellps=WGS84 +step +proj=helmert +convention=coordinate_frame +x=194.245 +y=-154.53 +z=102.676 +rx=351.5046 +ry=-701.841 +rz=251.9889 +s=6.296557 +exact +step +proj=cart +inv +ellps=intl +step +proj=tmerc +units=m +lat_0=17.63621319 +lon_0=-63.23752013 +x_0=4714.87 +y_0=1967.19 +k_0=1.0 and I got the proper answer: 3215.43 (E) and -287.55 (N) If I include the following string in the CRS in the new Qgis +proj=pipeline +step +proj=cart +ellps=WGS84 +step +proj=helmert +convention=coordinate_frame +x=194.245 +y=-154.53 +z=102.676 +rx=351.5046 +ry=-701.841 +rz=251.9889 +s=6.296557 +exact +units=m +no_defs +step +proj=cart +inv +ellps=intl +step +proj=tmerc +units=m +lat_0=17.63621319 +lon_0=-63.23752013 +x_0=4714.87 +y_0=1967.19 +k_0=1.0 and try to use "lon -63.25156367 lat 17.61578594) I got the message that the proj string is invalid and the message that the sting is not proj4 In the older versions of QGIS The same string gave proper values in the test environment and only when you click on "OK" the above message. Do you have an idea how to proceed? Kind regards, Michel Op di 19 nov. 2019 om 17:44 schreef Alex <notifications@github.com>:

…

@MichelRuijter <https://github.com/MichelRuijter> Update QGIS, it should come with proj6. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#31106?email_source=notifications&email_token=AMTW4UELY32EMFMWM7R2ERDQUQJWLA5CNFSM4IJS6CAKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEEO3Z5A#issuecomment-555597044>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMTW4UAEQR3U4X4MBYMJRTTQUQJWLANCNFSM4IJS6CAA> .

-- Michel Ruijter MNRengineering de Kempenaerstraat 82, 2341GP, Oegstgeest mob: +31638944484, tel: +31718897137

[rouault]

I tried within OSGEO 4 the following string:

This is a PROJ coordinate operation / pipeline string, not a PROJ CRS string, which is what QGIS / GDAL expects. In your use case, you could try +proj=tmerc +units=m +lat_0=17.63621319 +lon_0=-63.23752013 +x_0=4714.87 +y_0=1967.19 +k_0=1.0 +ellps=intl +towgs84=194.245,-154.53,102.676 ,-351.5046,701.841,-251.9889,6.296557 (note the negated signs of the rotation terms to pass from coordinate_frame convention to the implied position_vector by +towgs84)

[MichelRuijter]

Even, Thanks, I will try this (again) but, as far as I know the "towgs" statement uses approximations for the rotation part of the transformation, in the case provided the rotations are large and towgs does not work. Nyall has worked on the integration of Proj6 within Qgis and Proj6 should have the option to include more complex coordinate transformations including large rotations. Kind regards, Michel Thanks Op di 19 nov. 2019 19:55 schreef Even Rouault <notifications@github.com>:

…

I tried within OSGEO 4 the following string: This is a PROJ coordinate operation / pipeline string, not a PROJ CRS string, which is what QGIS / GDAL expects. In your use case, you could try +proj=tmerc +units=m +lat_0=17.63621319 +lon_0=-63.23752013 +x_0=4714.87 +y_0=1967.19 +k_0=1.0 +ellps=intl +towgs84=194.245,-154.53,102.676 ,-351.5046,701.841,-251.9889,6.296557 (note the negated signs of the rotation terms to pass from coordinate_frame convention to the implied position_vector by +towgs84) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#31106?email_source=notifications&email_token=AMTW4UCSXGANUDYCPYHNU7TQUQZB5A5CNFSM4IJS6CAKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEEPJ6CQ#issuecomment-555654922>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMTW4UHM55T4EXUXVORKHBLQUQZB5ANCNFSM4IJS6CAA> .

[rouault]

Proj6 should have the option to include more complex coordinate transformations including large rotations.

It does. But not in the CRS definition. Actually any Helmert-based transformation has nothing to do with the CRS definition, but is related how to the CRS transforms to another one. I'm not sure if QGIS has currently an option to define a custom PROJ pipeline between 2 CRS but I'm not sure. There is a Datum preference tab in Projects/Settings or Preferences/Options where you can select a datum transformation, but I couldn't find a way to override to add a PROJ pipeline.
Another possibility might be to tune the proj.db to define your own transformation, but that's a bit more involved.

[MichelRuijter]

Even, What still confuses me is that the pipeline structure as send in a previous mail works within QGIS in the "CRS test window" and I get the proper coordinates, but once I press OK that QGIS replies that the pipeline structure (the CRS) is not a valid proj 4 string. In the new QGIS that uses proj6 I was not even able to get the proper coordinates in the CRS Test window. Michel Op wo 20 nov. 2019 om 11:01 schreef Even Rouault <notifications@github.com>:

…

Proj6 should have the option to include more complex coordinate transformations including large rotations. It does. But not in the CRS definition. Actually any Helmert-based transformation has nothing to do with the CRS definition, but is related how to the CRS transforms to another one. I'm not sure if QGIS has currently an option to define a custom PROJ pipeline between 2 CRS but I'm not sure. There is a Datum preference tab in Projects/Settings or Preferences/Options where you can select a datum transformation, but I couldn't find a way to override to add a PROJ pipeline. Another possibility might be to tune the proj.db to define your own transformation, but that's a bit more involved. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#31106?email_source=notifications&email_token=AMTW4UG4XQAQRQDCGJ24OA3QUUDHHA5CNFSM4IJS6CAKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEERNMQQ#issuecomment-555931202>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMTW4UF2FLB5ODPZ4EYJLILQUUDHHANCNFSM4IJS6CAA> .

-- Michel Ruijter MNRengineering de Kempenaerstraat 82, 2341GP, Oegstgeest mob: +31638944484, tel: +31718897137

[MichelRuijter]

Even, The CRS definition, seems not to accept proj6 strings, however in the choice of reference systems the "pipeline" option is widely used, so how can I edit the underlying proj database to include a custom made projection/transformation Kind regards, Michel Op wo 20 nov. 2019 om 11:01 schreef Even Rouault <notifications@github.com>:

…

Proj6 should have the option to include more complex coordinate transformations including large rotations. It does. But not in the CRS definition. Actually any Helmert-based transformation has nothing to do with the CRS definition, but is related how to the CRS transforms to another one. I'm not sure if QGIS has currently an option to define a custom PROJ pipeline between 2 CRS but I'm not sure. There is a Datum preference tab in Projects/Settings or Preferences/Options where you can select a datum transformation, but I couldn't find a way to override to add a PROJ pipeline. Another possibility might be to tune the proj.db to define your own transformation, but that's a bit more involved. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#31106?email_source=notifications&email_token=AMTW4UG4XQAQRQDCGJ24OA3QUUDHHA5CNFSM4IJS6CAKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEERNMQQ#issuecomment-555931202>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMTW4UF2FLB5ODPZ4EYJLILQUUDHHANCNFSM4IJS6CAA> .

-- Michel Ruijter MNRengineering de Kempenaerstraat 82, 2341GP, Oegstgeest mob: +31638944484, tel: +31718897137

[nyalldawson]

@MichelRuijter

Sorry, I missed that your definition was actually defining an operation and not a CRS in my earlier response.

This isn't yet possible to do via the QGIS ui (but the required backend capabilities should all be present). What we'd need to do is extent the transformation selection dialog (the dialog which allows you to pick from known transformation pathways between a source and destination crs pair) to also allow users to enter a custom proj operation string, alongside the existing choice of known operations.

This would then allow QGIS to use your custom operation when transforming between the source/destination CRSes.

There's a bit of development work required to do this, mainly just revamping the transformation selection dialog to allow this kind of manual entry. Let me know if you'd like to sponsor this change!

[Ognian]

Having exact the same problem, I'm wondering if there is another way to do it, since it is quite a bit of time passed since this was opened...

[otofoto]

@Ognian
If you would like to test new pipeline, you must edit proj.db, add the new transform and a new usage record for it. Only after that you select my custom transform there. However, this approach is a bit too complicated, as described above.