Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test failures on FreeBSD 12 #1644

Closed
pramsey opened this issue Sep 26, 2019 · 26 comments
Closed

Test failures on FreeBSD 12 #1644

pramsey opened this issue Sep 26, 2019 · 26 comments

Comments

@pramsey
Copy link

pramsey commented Sep 26, 2019

I'm tracking down this PostGIS ticket 4500 and found that Proj 6.1.1 has unit failures on the FreeBSD 12 platform.

The following tests FAILED:
	 30 - testvarious (Failed)
	 31 - testprojinfo (Failed)
	 38 - proj_test_cpp_api (Failed)

The OS version is

FreeBSD bazinga.localdomain 12.0-RELEASE FreeBSD 12.0-RELEASE r341666 GENERIC  amd64

I ran build and test with cmake.

(Side issue, I could not complete a build using the ./configure method, it died on the build of proj.db. If autotools are deprecated and not maintained, maybe they should be fully removed.)

@rouault
Copy link
Member

rouault commented Sep 26, 2019

You need likely to do:
PROJ_LIB=data ctest

Automatically setting PROJ_LIB for ctest has been fixed just a few days ago in master

Autotools work fine and are tested on Travis-CI.

We also have a FreeBSD CI: https://cirrus-ci.com/github/OSGeo/PROJ/master , and it uses autotools. You need to use "gmake". And the build requirements are "pkg install -y autoconf automake libtool pkgconf sqlite3 unzip gmake". See https://github.com/OSGeo/PROJ/blob/master/.cirrus.yml

@pramsey
Copy link
Author

pramsey commented Sep 26, 2019

Using PROJ_LIB=data ctest causes very many more tests to fail, so it's something else. Is there an easy place to look for the diffs in the testvarious, testprojinfo modules, etc?

@rouault
Copy link
Member

rouault commented Sep 26, 2019

Is there an easy place to look for the diffs in the testvarious, testprojinfo modules, etc?

I use autotools 99% of the time. You'll get verbose output with them. I guess you could with ctest also (ctest -V ?)

@pramsey
Copy link
Author

pramsey commented Sep 26, 2019

Looks like test failures are associated with the same problem that is blocking the autotools build: proj.db is not being created.

30: doing tests into file tv_out, please wait
30: proj_create_operation_factory_context: Cannot find proj.db
30: pj_obj_create: Cannot find proj.db

The cmake build just fails to create proj.db silently, while autotools comes to a halt.

@pramsey
Copy link
Author

pramsey commented Sep 26, 2019

For the record, here's the stop in autotools

Making all in data
Make proj.db
proj.db
make[1]: exec(proj.db) failed (No such file or directory)
*** Error code 1

@rouault
Copy link
Member

rouault commented Sep 26, 2019

Do you use gmake (gnu make) with autotools ?

@pramsey
Copy link
Author

pramsey commented Sep 26, 2019

No, I used just make which is not GNU. Using gmakethe error message is (marginally) less opaque,

Make proj.db
rm -f proj.db
Error: unable to open database "proj.db": unable to open database file
Build of proj.db failed
gmake[1]: *** [Makefile:575: proj.db] Error 1

@rouault
Copy link
Member

rouault commented Sep 26, 2019

Do you have the sqlite3 binary installed ? Did you look at https://github.com/OSGeo/PROJ/blob/master/.cirrus.yml ?

@pramsey
Copy link
Author

pramsey commented Sep 27, 2019

I do have sqlite3, and that CI script looks very much like what I'm doing. I will ape it and see if anything else falls out, but this is certainly curious.

@rouault
Copy link
Member

rouault commented Sep 27, 2019

There seems to be a PROJ 6.1.1 FreeBSD package: http://www1.fr.freebsd.org/ports/graphics.html#proj-6.1.1

@pramsey
Copy link
Author

pramsey commented Sep 27, 2019

Yep, I'm just trying to track down https://trac.osgeo.org/postgis/ticket/4500 and figured build-and-regress-proj was the first step. I'll probably try the package and see if I can reproduce the ticket. It really looks like a failure to find the dictionary to me.

@robe2
Copy link

robe2 commented Sep 28, 2019

proj works fine on PostGIS 2.5 for freebsd 12 and I confirmed sqlite is installed.
It gives the error only on PostGIS 3.0.

Unfortunately I'm using pkg install (which uses binaries) and proj 6.1.1 is not yet available on that it seems.

@robe2
Copy link

robe2 commented Sep 28, 2019

Sadly installing Proj 6.1.1 did not help. To get Proj 6.1.1 I had to pull it from portsnap.
I have the same issue with the FreeBSD 12 32-bit bot (that one I kept at Proj 6.1.0 since the 6.1.1 which took a lot of time (cause I chose to reinstall GDAL from ports as well which required compiling many other packages).

PostgreSQL 11.4 on amd64-portbld-freebsd12.0, compiled by FreeBSD clang version 6.0.1 (tags/RELEASE_601/final 335540) (based on LLVM 6.0.1), 64-bit
Postgis 3.0.0alpha5dev - r17827 - 2019-09-28 19:13:43
scripts 3.0.0alpha5dev r17827
GEOS: 3.7.2-CAPI-1.11.2 b55d2125
PROJ: 6.1.1

Running tests

../loader/Point .............. ok
../loader/PointM .............. ok
../loader/PointZ .............. ok
../loader/MultiPoint .............. ok
../loader/MultiPointM .............. ok
../loader/MultiPointZ .............. ok
../loader/Arc .............. ok
../loader/ArcM .............. ok
../loader/ArcZ .............. ok
../loader/Polygon .............. ok
../loader/PolygonM .............. ok
../loader/PolygonZ .............. ok
../loader/TSTPolygon ......... ok
../loader/TSIPolygon ......... ok
../loader/TSTIPolygon ......... ok
../loader/PointWithSchema ..... ok
../loader/NoTransPoint ......... ok
../loader/NotReallyMultiPoint ......... ok
../loader/MultiToSinglePoint ......... ok
../loader/ReprojectPts ..... failed (wkt testdiff expected obtained: /home/jenkins/tmp/pgis_reg_/test_20_diff)

--- ../loader/ReprojectPts-w.select.expected 2019-09-28 10:42:46.255801000 +0000
+++ /home/jenkins/tmp/pgis_reg_/test_20_out 2019-09-28 19:16:48.016016000 +0000
@@ -1,66 +1,66 @@
-SRID=4326;POINT(-74.17782492 41.08263361)
-SRID=4326;POINT(-74.44881286 40.50133744)
+SRID=4326;POINT(-74.17782443 41.08263361)
+SRID=4326;POINT(-74.4488128 40.50133699)
SRID=4326;POINT(-74.00376358 40.28149688)
SRID=4326;POINT(-75.12863742 39.7797389)
-SRID=4326;POINT(-74.57629014 40.85597595)
-SRID=4326;POINT(-74.47160401 40.52369066)
+SRID=4326;POINT(-74.57628962 40.85597621)
+SRID=4326;POINT(-74.47160381 40.52369026)
SRID=4326;POINT(-75.46891683 39.69688334)
SRID=4326;POINT(-75.11458618 39.70739231)
SRID=4326;POINT(-74.22643701 40.09726563)
-SRID=4326;POINT(-74.26766926 40.83522615)
-SRID=4326;POINT(-74.42152037 40.76232181)
-SRID=4326;POINT(-74.18666598 40.89980341)
-SRID=4326;POINT(-74.20201874 40.94448827)
-SRID=4326;POINT(-74.31866663 40.6680465)
-SRID=4326;POINT(-74.83205963 40.84912898)
+SRID=4326;POINT(-74.26766872 40.8352264)
+SRID=4326;POINT(-74.42151978 40.76232209)
+SRID=4326;POINT(-74.1866655 40.89980354)
+SRID=4326;POINT(-74.20201825 40.94448845)
+SRID=4326;POINT(-74.31866643 40.66804657)
+SRID=4326;POINT(-74.83205905 40.84912911)
SRID=4326;POINT(-74.64402101 39.96633708)
SRID=4326;POINT(-74.22194028 40.09559148)
-SRID=4326;POINT(-74.60375255 40.75504208)
-SRID=4326;POINT(-74.09376018 40.86569336)
-SRID=4326;POINT(-74.4430374 40.77797967)
+SRID=4326;POINT(-74.60375193 40.75504234)
+SRID=4326;POINT(-74.09375981 40.86569326)
+SRID=4326;POINT(-74.44303683 40.77797995)
SRID=4326;POINT(-74.76841703 40.22038455)
-SRID=4326;POINT(-74.19078182 40.73914574)
-SRID=4326;POINT(-74.19628444 40.79591416)
-SRID=4326;POINT(-74.19130306 40.74330253)
-SRID=4326;POINT(-74.17636308 40.73783123)
+SRID=4326;POINT(-74.19078139 40.73914578)
+SRID=4326;POINT(-74.19628396 40.79591425)
+SRID=4326;POINT(-74.19130262 40.74330258)
+SRID=4326;POINT(-74.17636268 40.73783123)
SRID=4326;POINT(-74.53148731 39.49029456)
-SRID=4326;POINT(-74.16618054 40.73634864)
-SRID=4326;POINT(-74.35732607 40.80076793)
-SRID=4326;POINT(-74.17573811 40.73901418)
+SRID=4326;POINT(-74.16618015 40.7363486)
+SRID=4326;POINT(-74.35732552 40.80076819)
+SRID=4326;POINT(-74.1757377 40.73901418)
SRID=4326;POINT(-74.66491581 40.34572735)
-SRID=4326;POINT(-74.36625323 40.51061374)
-SRID=4326;POINT(-74.17631876 40.74329159)
-SRID=4326;POINT(-74.4544664 40.52427239)
-SRID=4326;POINT(-74.02836656 40.89756584)
+SRID=4326;POINT(-74.36625351 40.51061338)
+SRID=4326;POINT(-74.17631833 40.74329159)
+SRID=4326;POINT(-74.45446627 40.524272)
+SRID=4326;POINT(-74.02836625 40.89756567)
SRID=4326;POINT(-75.00833975 39.82895026)
SRID=4326;POINT(-74.13132221 40.33161528)
SRID=4326;POINT(-74.67999522 39.46203859)
-SRID=4326;POINT(-74.08904806 40.9515804)
+SRID=4326;POINT(-74.08904769 40.95158042)
SRID=4326;POINT(-75.12091068 39.94826917)
-SRID=4326;POINT(-74.08628025 40.70929009)
+SRID=4326;POINT(-74.08628002 40.70928983)
SRID=4326;POINT(-74.73270242 40.27825159)

@rouault
Copy link
Member

rouault commented Sep 29, 2019

Isn't it related to the lack of some grids (but in that case I would expect all point coordinates to differ) ?
Try to isolate & replicate with PROJ command line utilities, like cs2cs

@pramsey
Copy link
Author

pramsey commented Sep 30, 2019

Confirmed without using PostGIS:

FreeBSD environment is FreeBSD 12 with the Proj 6.1.1 pkg installed.

cs2cs --version
Rel. 6.1.1, July 1st, 2019

echo 613346.499961959 691449.747538779 | cs2cs -f "%.8f" '+init=epsg:2260' '+init=epsg:4326'
-74.06257592	40.73086029 0.00000000

echo 506358.893822162 607539.257299624 | cs2cs -f "%.8f" '+init=epsg:2260' '+init=epsg:4326'
-74.44881280	40.50133699 0.00000000

OSX environment is 10.14.6 with Proj 6.1.0 from source.

cs2cs --version
Rel. 6.1.0, September 1st, 2019

echo 613346.499961959 691449.747538779 | cs2cs -f "%.8f" '+init=epsg:2260' '+init=epsg:4326'
-74.06257620	40.73086062 0.00000000

echo 506358.893822162 607539.257299624 | cs2cs -f "%.8f" '+init=epsg:2260' '+init=epsg:4326'
-74.44881286	40.50133744 0.00000000

The results from the OSX run match the results in the regression tests which are generating fine on all our other platforms, so something is awry in the FreeBSD world (probably the packaging).

@kbevers
Copy link
Member

kbevers commented Sep 30, 2019

Do you get the same thing if you don't use the +init syntax? Here's what PROJ 6.1.1 produces on my Mac:

$ echo 613346.499961959 691449.747538779 | cs2cs -f "%.8f" EPSG:2260 EPSG:4326
40.73086062	-74.06257620 0.00000000

$ echo 506358.893822162 607539.257299624 | cs2cs -f "%.8f" EPSG:2260 EPSG:4326
40.50133744	-74.44881286 0.00000000

Obviously the axis order is switched, but otherwise the numbers are the same as the OSX result above. I am wondering if an old epsg file is mistakenly picked up by PROJ on the FreeBSD install.

@pramsey
Copy link
Author

pramsey commented Sep 30, 2019

Your supposition is correct: when using explicit notation, the results on FreeBSD now match OSX.

echo 613346.499961959 691449.747538779 | cs2cs -f "%.8f" '+proj=tmerc +lat_0=38.83333333333334 +lon_0=-74.5 +k=0.9999 +x_0=150000 +y_0=0 +datum=NAD83 +units=us-ft' '+proj=longlat +datum=WGS84'
-74.06257620	40.73086062 0.00000000

echo 506358.893822162 607539.257299624 | cs2cs -f "%.8f" '+proj=tmerc +lat_0=38.83333333333334 +lon_0=-74.5 +k=0.9999 +x_0=150000 +y_0=0 +datum=NAD83 +units=us-ft' '+proj=longlat +datum=WGS84'
-74.44881286	40.50133744 0.00000000

So that leaves the question: why/where is FreeBSD picking up some different definition? (I got those explicit notations from the PostGIS spatial_ref_sys table, so they aren't exactly fresh'n'spicy themselves.

Update: Using EPSG notation provides FreeBSD results that are the same as the init string results

echo 613346.499961959 691449.747538779 | cs2cs -f "%.8f" EPSG:2260 EPSG:4326
40.73086029	-74.06257592 0.00000000

echo 506358.893822162 607539.257299624 | cs2cs -f "%.8f" EPSG:2260 EPSG:4326
40.50133699	-74.44881280 0.00000000

Summary so far:

# Input Coordinates
613346.499961959 691449.747538779
# FreeBSD w/ Init Proj String
-74.06257592 40.73086029
# FreeBSD w/ Full Proj String
-74.06257620 40.73086062
# FreeBSD w/ EPSG Code
-74.06257592 40.73086029	
# OSX w/ ESPG Code
-74.06257620 40.73086062	

@kbevers
Copy link
Member

kbevers commented Sep 30, 2019

What does ls `echo $PROJ_LIB` return?

@pramsey
Copy link
Author

pramsey commented Sep 30, 2019

[vagrant@bazinga ~]$ echo $PROJ_LIB

empty string

@kbevers
Copy link
Member

kbevers commented Sep 30, 2019

Not what I expected.... Try running the above commands with a high debug level:

echo 506358.893822162 607539.257299624 | PROJ_DEBUG=3 cs2cs -f "%.8f" '+init=epsg:2260' '+init=epsg:4326'

@pramsey
Copy link
Author

pramsey commented Sep 30, 2019

[vagrant@bazinga ~]$ echo 506358.893822162 607539.257299624 | PROJ_DEBUG=3 cs2cs -f "%.8f" '+init=epsg:2260' '+init=epsg:4326' 
pj_open_lib(proj.db): call fopen(/usr/local/share/proj/proj.db) - succeeded
pj_open_lib(epsg): call fopen(/usr/local/share/proj/epsg) - failed
pj_open_lib(epsg): call fopen(/usr/local/share/proj/epsg) - failed
pj_open_lib(epsg): call fopen(/usr/local/share/proj/epsg) - failed
pj_open_lib(epsg): call fopen(/usr/local/share/proj/epsg) - failed
pj_open_lib(NAD83-98.gsb): call fopen(/usr/local/share/proj/NAD83-98.gsb) - failed
pj_open_lib(AB_CSRS.DAC): call fopen(/usr/local/share/proj/AB_CSRS.DAC) - failed
pj_open_lib(SK83-98.gsb): call fopen(/usr/local/share/proj/SK83-98.gsb) - failed
pj_open_lib(ethpgn.gsb): call fopen(/usr/local/share/proj/ethpgn.gsb) - succeeded
pj_open_lib(wthpgn.gsb): call fopen(/usr/local/share/proj/wthpgn.gsb) - succeeded
pj_open_lib(FL): call fopen(/usr/local/share/proj/FL) - succeeded
pj_open_lib(emhpgn.gsb): call fopen(/usr/local/share/proj/emhpgn.gsb) - succeeded
pj_open_lib(WO): call fopen(/usr/local/share/proj/WO) - succeeded
pj_open_lib(mihpgn.gsb): call fopen(/usr/local/share/proj/mihpgn.gsb) - succeeded
pj_open_lib(cnhpgn.gsb): call fopen(/usr/local/share/proj/cnhpgn.gsb) - succeeded
pj_open_lib(nvhpgn.gsb): call fopen(/usr/local/share/proj/nvhpgn.gsb) - succeeded
pj_open_lib(mnhpgn.gsb): call fopen(/usr/local/share/proj/mnhpgn.gsb) - succeeded
pj_open_lib(nmhpgn.gsb): call fopen(/usr/local/share/proj/nmhpgn.gsb) - succeeded
pj_open_lib(azhpgn.gsb): call fopen(/usr/local/share/proj/azhpgn.gsb) - succeeded
pj_open_lib(nyhpgn.gsb): call fopen(/usr/local/share/proj/nyhpgn.gsb) - succeeded
pj_open_lib(cshpgn.gsb): call fopen(/usr/local/share/proj/cshpgn.gsb) - failed
pj_open_lib(mohpgn.gsb): call fopen(/usr/local/share/proj/mohpgn.gsb) - succeeded
pj_open_lib(okhpgn.gsb): call fopen(/usr/local/share/proj/okhpgn.gsb) - succeeded
pj_open_lib(WI): call fopen(/usr/local/share/proj/WI) - succeeded
pj_open_lib(cohpgn.gsb): call fopen(/usr/local/share/proj/cohpgn.gsb) - succeeded
pj_open_lib(wmhpgn.gsb): call fopen(/usr/local/share/proj/wmhpgn.gsb) - succeeded
pj_open_lib(wyhpgn.gsb): call fopen(/usr/local/share/proj/wyhpgn.gsb) - succeeded
pj_open_lib(nchpgn.gsb): call fopen(/usr/local/share/proj/nchpgn.gsb) - succeeded
pj_open_lib(nbhpgn.gsb): call fopen(/usr/local/share/proj/nbhpgn.gsb) - succeeded
pj_open_lib(uthpgn.gsb): call fopen(/usr/local/share/proj/uthpgn.gsb) - succeeded
pj_open_lib(vahpgn.gsb): call fopen(/usr/local/share/proj/vahpgn.gsb) - succeeded
pj_open_lib(ilhpgn.gsb): call fopen(/usr/local/share/proj/ilhpgn.gsb) - succeeded
pj_open_lib(sdhpgn.gsb): call fopen(/usr/local/share/proj/sdhpgn.gsb) - succeeded
pj_open_lib(lahpgn.gsb): call fopen(/usr/local/share/proj/lahpgn.gsb) - succeeded
pj_open_lib(gahpgn.gsb): call fopen(/usr/local/share/proj/gahpgn.gsb) - succeeded
pj_open_lib(hihpgn.gsb): call fopen(/usr/local/share/proj/hihpgn.gsb) - succeeded
pj_open_lib(kshpgn.gsb): call fopen(/usr/local/share/proj/kshpgn.gsb) - failed
pj_open_lib(kyhpgn.gsb): call fopen(/usr/local/share/proj/kyhpgn.gsb) - succeeded
pj_open_lib(ndhpgn.gsb): call fopen(/usr/local/share/proj/ndhpgn.gsb) - succeeded
pj_open_lib(iahpgn.gsb): call fopen(/usr/local/share/proj/iahpgn.gsb) - succeeded
pj_open_lib(mshpgn.gsb): call fopen(/usr/local/share/proj/mshpgn.gsb) - failed
pj_open_lib(alhpgn.gsb): call fopen(/usr/local/share/proj/alhpgn.gsb) - succeeded
pj_open_lib(arhpgn.gsb): call fopen(/usr/local/share/proj/arhpgn.gsb) - succeeded
pj_open_lib(wvhpgn.gsb): call fopen(/usr/local/share/proj/wvhpgn.gsb) - succeeded
pj_open_lib(mehpgn.gsb): call fopen(/usr/local/share/proj/mehpgn.gsb) - succeeded
pj_open_lib(ohhpgn.gsb): call fopen(/usr/local/share/proj/ohhpgn.gsb) - succeeded
pj_open_lib(schpgn.gsb): call fopen(/usr/local/share/proj/schpgn.gsb) - succeeded
pj_open_lib(pahpgn.gsb): call fopen(/usr/local/share/proj/pahpgn.gsb) - succeeded
pj_open_lib(nehpgn.gsb): call fopen(/usr/local/share/proj/nehpgn.gsb) - succeeded
pj_open_lib(TN): call fopen(/usr/local/share/proj/TN) - succeeded
pj_open_lib(inhpgn.gsb): call fopen(/usr/local/share/proj/inhpgn.gsb) - succeeded
pj_open_lib(MD): call fopen(/usr/local/share/proj/MD) - succeeded
pj_open_lib(njhpgn.gsb): call fopen(/usr/local/share/proj/njhpgn.gsb) - succeeded
pj_open_lib(pvhpgn.gsb): call fopen(/usr/local/share/proj/pvhpgn.gsb) - succeeded
pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
pj_ellipsoid - final:    ellps=GRS80
pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
pj_ellipsoid - final:    
Pipeline: Building arg list for step no. 0
Pipeline: init - proj=unitconvert, 5
    xy_in=us-ft
    z_in=us-ft
    xy_out=m
    z_out=m
pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
pj_ellipsoid - final:    ellps=GRS80
xy_in unit: U.S. Surveyor's Foot
xy_out unit: Meter
z_in unit: U.S. Surveyor's Foot
z_out unit: Meter
Pipeline: Step 0 (proj=unitconvert) at 0x800c6d080
Pipeline at [0x800c6cd00]:    step at [0x800c6d080] (proj=unitconvert) done
Pipeline: Building arg list for step no. 1
Pipeline: init - inv, 8
    proj=tmerc
    lat_0=38.8333333333333
    lon_0=-74.5
    k=0.9999
    x_0=150000
    y_0=0
    ellps=GRS80
pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
pj_ellipsoid - final:    ellps=GRS80
Pipeline: Step 1 (inv) at 0x800c6d400
Pipeline at [0x800c6cd00]:    step at [0x800c6d400] (inv) done
Pipeline: Building arg list for step no. 2
Pipeline: init - proj=hgridshift, 2
    grids=nyhpgn.gsb
pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
pj_ellipsoid - final:    ellps=GRS80
pj_open_lib(nyhpgn.gsb): call fopen(/usr/local/share/proj/nyhpgn.gsb) - succeeded
NTv2 ny       45x25: LL=(-81,40) UR=(-70,46)
Pipeline: Step 2 (proj=hgridshift) at 0x800c6d780
Pipeline at [0x800c6cd00]:    step at [0x800c6d780] (proj=hgridshift) done
Pipeline: Building arg list for step no. 3
Pipeline: init - proj=unitconvert, 3
    xy_in=rad
    xy_out=deg
pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
pj_ellipsoid - final:    ellps=GRS80
xy_in unit: Radian
xy_out unit: Degree
Pipeline: Step 3 (proj=unitconvert) at 0x800c6db00
Pipeline at [0x800c6cd00]:    step at [0x800c6db00] (proj=unitconvert) done
Pipeline: 4 steps built. Determining i/o characteristics
pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
pj_ellipsoid - final:    ellps=GRS80
pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
pj_ellipsoid - final:    
Pipeline: Building arg list for step no. 0
Pipeline: init - proj=unitconvert, 3
    xy_in=deg
    xy_out=rad
pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
pj_ellipsoid - final:    ellps=GRS80
xy_in unit: Degree
xy_out unit: Radian
Pipeline: Step 0 (proj=unitconvert) at 0x800c6de80
Pipeline at [0x800c6e900]:    step at [0x800c6de80] (proj=unitconvert) done
Pipeline: Building arg list for step no. 1
Pipeline: init - proj=tmerc, 7
    lat_0=38.8333333333333
    lon_0=-74.5
    k=0.9999
    x_0=150000
    y_0=0
    ellps=GRS80
pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
pj_ellipsoid - final:    ellps=GRS80
Pipeline: Step 1 (proj=tmerc) at 0x800c6e200
Pipeline at [0x800c6e900]:    step at [0x800c6e200] (proj=tmerc) done
Pipeline: Building arg list for step no. 2
Pipeline: init - proj=unitconvert, 5
    xy_in=m
    z_in=m
    xy_out=us-ft
    z_out=us-ft
pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
pj_ellipsoid - final:    ellps=GRS80
xy_in unit: Meter
xy_out unit: U.S. Surveyor's Foot
z_in unit: Meter
z_out unit: U.S. Surveyor's Foot
Pipeline: Step 2 (proj=unitconvert) at 0x800c6e580
Pipeline at [0x800c6e900]:    step at [0x800c6e580] (proj=unitconvert) done
Pipeline: 3 steps built. Determining i/o characteristics
pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
pj_ellipsoid - final:    ellps=GRS80
pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
pj_ellipsoid - final:    ellps=GRS80
pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
pj_ellipsoid - final:    
Pipeline: Building arg list for step no. 0
Pipeline: init - proj=unitconvert, 5
    xy_in=us-ft
    z_in=us-ft
    xy_out=m
    z_out=m
pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
pj_ellipsoid - final:    ellps=GRS80
xy_in unit: U.S. Surveyor's Foot
xy_out unit: Meter
z_in unit: U.S. Surveyor's Foot
z_out unit: Meter
Pipeline: Step 0 (proj=unitconvert) at 0x8017dc000
Pipeline at [0x800c6ec80]:    step at [0x8017dc000] (proj=unitconvert) done
Pipeline: Building arg list for step no. 1
Pipeline: init - inv, 8
    proj=tmerc
    lat_0=38.8333333333333
    lon_0=-74.5
    k=0.9999
    x_0=150000
    y_0=0
    ellps=GRS80
pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
pj_ellipsoid - final:    ellps=GRS80
Pipeline: Step 1 (inv) at 0x8017dc380
Pipeline at [0x800c6ec80]:    step at [0x8017dc380] (inv) done
Pipeline: Building arg list for step no. 2
Pipeline: init - proj=hgridshift, 2
    grids=nyhpgn.gsb
pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
pj_ellipsoid - final:    ellps=GRS80
Pipeline: Step 2 (proj=hgridshift) at 0x8017dca80
Pipeline at [0x800c6ec80]:    step at [0x8017dca80] (proj=hgridshift) done
Pipeline: Building arg list for step no. 3
Pipeline: init - proj=unitconvert, 3
    xy_in=rad
    xy_out=deg
pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
pj_ellipsoid - final:    ellps=GRS80
xy_in unit: Radian
xy_out unit: Degree
Pipeline: Step 3 (proj=unitconvert) at 0x8017dce00
Pipeline at [0x800c6ec80]:    step at [0x8017dce00] (proj=unitconvert) done
Pipeline: 4 steps built. Determining i/o characteristics
pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
pj_ellipsoid - final:    ellps=GRS80
pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
pj_ellipsoid - final:    
Pipeline: Building arg list for step no. 0
Pipeline: init - proj=unitconvert, 5
    xy_in=us-ft
    z_in=us-ft
    xy_out=m
    z_out=m
pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
pj_ellipsoid - final:    ellps=GRS80
xy_in unit: U.S. Surveyor's Foot
xy_out unit: Meter
z_in unit: U.S. Surveyor's Foot
z_out unit: Meter
Pipeline: Step 0 (proj=unitconvert) at 0x8017dd500
Pipeline at [0x8017dd180]:    step at [0x8017dd500] (proj=unitconvert) done
Pipeline: Building arg list for step no. 1
Pipeline: init - inv, 8
    proj=tmerc
    lat_0=38.8333333333333
    lon_0=-74.5
    k=0.9999
    x_0=150000
    y_0=0
    ellps=GRS80
pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
pj_ellipsoid - final:    ellps=GRS80
Pipeline: Step 1 (inv) at 0x8017dd880
Pipeline at [0x8017dd180]:    step at [0x8017dd880] (inv) done
Pipeline: Building arg list for step no. 2
Pipeline: init - proj=unitconvert, 3
    xy_in=rad
    xy_out=deg
pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
pj_ellipsoid - final:    ellps=GRS80
xy_in unit: Radian
xy_out unit: Degree
Pipeline: Step 2 (proj=unitconvert) at 0x8017ddc00
Pipeline at [0x8017dd180]:    step at [0x8017ddc00] (proj=unitconvert) done
Pipeline: 3 steps built. Determining i/o characteristics
pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
pj_ellipsoid - final:    ellps=GRS80
pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
pj_ellipsoid - final:    
Pipeline: Building arg list for step no. 0
Pipeline: init - proj=unitconvert, 5
    xy_in=us-ft
    z_in=us-ft
    xy_out=m
    z_out=m
pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
pj_ellipsoid - final:    ellps=GRS80
xy_in unit: U.S. Surveyor's Foot
xy_out unit: Meter
z_in unit: U.S. Surveyor's Foot
z_out unit: Meter
Pipeline: Step 0 (proj=unitconvert) at 0x8017de300
Pipeline at [0x8017ddf80]:    step at [0x8017de300] (proj=unitconvert) done
Pipeline: Building arg list for step no. 1
Pipeline: init - inv, 8
    proj=tmerc
    lat_0=38.8333333333333
    lon_0=-74.5
    k=0.9999
    x_0=150000
    y_0=0
    ellps=GRS80
pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
pj_ellipsoid - final:    ellps=GRS80
Pipeline: Step 1 (inv) at 0x8017de680
Pipeline at [0x8017ddf80]:    step at [0x8017de680] (inv) done
Pipeline: Building arg list for step no. 2
Pipeline: init - proj=hgridshift, 2
    grids=nehpgn.gsb
pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
pj_ellipsoid - final:    ellps=GRS80
pj_open_lib(nehpgn.gsb): call fopen(/usr/local/share/proj/nehpgn.gsb) - succeeded
NTv2 ne       25x25: LL=(-75,40) UR=(-69,46)
Pipeline: Step 2 (proj=hgridshift) at 0x8017dea00
Pipeline at [0x8017ddf80]:    step at [0x8017dea00] (proj=hgridshift) done
Pipeline: Building arg list for step no. 3
Pipeline: init - proj=unitconvert, 3
    xy_in=rad
    xy_out=deg
pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
pj_ellipsoid - final:    ellps=GRS80
xy_in unit: Radian
xy_out unit: Degree
Pipeline: Step 3 (proj=unitconvert) at 0x8017ded80
Pipeline at [0x8017ddf80]:    step at [0x8017ded80] (proj=unitconvert) done
Pipeline: 4 steps built. Determining i/o characteristics
pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
pj_ellipsoid - final:    ellps=GRS80
pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
pj_ellipsoid - final:    
Pipeline: Building arg list for step no. 0
Pipeline: init - proj=unitconvert, 5
    xy_in=us-ft
    z_in=us-ft
    xy_out=m
    z_out=m
pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
pj_ellipsoid - final:    ellps=GRS80
xy_in unit: U.S. Surveyor's Foot
xy_out unit: Meter
z_in unit: U.S. Surveyor's Foot
z_out unit: Meter
Pipeline: Step 0 (proj=unitconvert) at 0x8017df480
Pipeline at [0x8017df100]:    step at [0x8017df480] (proj=unitconvert) done
Pipeline: Building arg list for step no. 1
Pipeline: init - inv, 8
    proj=tmerc
    lat_0=38.8333333333333
    lon_0=-74.5
    k=0.9999
    x_0=150000
    y_0=0
    ellps=GRS80
pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
pj_ellipsoid - final:    ellps=GRS80
Pipeline: Step 1 (inv) at 0x8017df800
Pipeline at [0x8017df100]:    step at [0x8017df800] (inv) done
Pipeline: Building arg list for step no. 2
Pipeline: init - proj=hgridshift, 2
    grids=pahpgn.gsb
pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
pj_ellipsoid - final:    ellps=GRS80
pj_open_lib(pahpgn.gsb): call fopen(/usr/local/share/proj/pahpgn.gsb) - succeeded
NTv2 pa       33x21: LL=(-81.5,39) UR=(-73.5,44)
Pipeline: Step 2 (proj=hgridshift) at 0x8017dfb80
Pipeline at [0x8017df100]:    step at [0x8017dfb80] (proj=hgridshift) done
Pipeline: Building arg list for step no. 3
Pipeline: init - proj=unitconvert, 3
    xy_in=rad
    xy_out=deg
pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
pj_ellipsoid - final:    ellps=GRS80
xy_in unit: Radian
xy_out unit: Degree
Pipeline: Step 3 (proj=unitconvert) at 0x8017dff00
Pipeline at [0x8017df100]:    step at [0x8017dff00] (proj=unitconvert) done
Pipeline: 4 steps built. Determining i/o characteristics
pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
pj_ellipsoid - final:    ellps=GRS80
pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
pj_ellipsoid - final:    
Pipeline: Building arg list for step no. 0
Pipeline: init - proj=unitconvert, 5
    xy_in=us-ft
    z_in=us-ft
    xy_out=m
    z_out=m
pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
pj_ellipsoid - final:    ellps=GRS80
xy_in unit: U.S. Surveyor's Foot
xy_out unit: Meter
z_in unit: U.S. Surveyor's Foot
z_out unit: Meter
Pipeline: Step 0 (proj=unitconvert) at 0x8017e0600
Pipeline at [0x8017e0280]:    step at [0x8017e0600] (proj=unitconvert) done
Pipeline: Building arg list for step no. 1
Pipeline: init - inv, 8
    proj=tmerc
    lat_0=38.8333333333333
    lon_0=-74.5
    k=0.9999
    x_0=150000
    y_0=0
    ellps=GRS80
pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
pj_ellipsoid - final:    ellps=GRS80
Pipeline: Step 1 (inv) at 0x8017e0980
Pipeline at [0x8017e0280]:    step at [0x8017e0980] (inv) done
Pipeline: Building arg list for step no. 2
Pipeline: init - proj=hgridshift, 2
    grids=njhpgn.gsb
pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
pj_ellipsoid - final:    ellps=GRS80
pj_open_lib(njhpgn.gsb): call fopen(/usr/local/share/proj/njhpgn.gsb) - succeeded
NTv2 nj       25x25: LL=(-76,38) UR=(-70,44)
Pipeline: Step 2 (proj=hgridshift) at 0x8017e0d00
Pipeline at [0x8017e0280]:    step at [0x8017e0d00] (proj=hgridshift) done
Pipeline: Building arg list for step no. 3
Pipeline: init - proj=unitconvert, 3
    xy_in=rad
    xy_out=deg
pj_ellipsoid - final: a=6378137.000 f=1/298.257, errno=0
pj_ellipsoid - final:    ellps=GRS80
xy_in unit: Radian
xy_out unit: Degree
Pipeline: Step 3 (proj=unitconvert) at 0x8017e1080
Pipeline at [0x8017e0280]:    step at [0x8017e1080] (proj=unitconvert) done
Pipeline: 4 steps built. Determining i/o characteristics
Using coordinate operation Inverse of SPCS83 New York East zone (US Survey feet) + NAD83 to WGS 84 (31) + axis order change (2D)
NTv2 - loading grid ny      
pj_open_lib(nyhpgn.gsb): call fopen(/usr/local/share/proj/nyhpgn.gsb) - succeeded
-74.44881280	40.50133699 0.00000000

@kbevers
Copy link
Member

kbevers commented Sep 30, 2019

Right... So, initially I did this without having the proj-datum-north-america package installed and got the same result as you on OSX. With the package installed I reproduce your FreeBSD results. It doesn't matter if I use the init-syntax or not. So, this is probably not a bug, you are likely to just be picking up a better transformation when North American grids are available.

Some projinfoing might explain this in more detail.

@rouault
Copy link
Member

rouault commented Sep 30, 2019

The logs show that the New York grid is used

Using coordinate operation Inverse of SPCS83 New York East zone (US Survey feet) + NAD83 to WGS 84 (31) + axis order change (2D)
NTv2 - loading grid ny      
pj_open_lib(nyhpgn.gsb): call fopen(/usr/local/share/proj/nyhpgn.gsb) - succeeded

So there's no longer any mystery

@pramsey
Copy link
Author

pramsey commented Sep 30, 2019

So @robe2 I think what this means for us is we need to rework those tests to avoid any geodetic transformation. Stick with cartographic reprojection in the same GCS so that we aren't getting random fails depending on the completeness of the Proj installation.

@robe2
Copy link

robe2 commented Sep 30, 2019

@pramsey fine

@pramsey pramsey closed this as completed Sep 30, 2019
@pramsey
Copy link
Author

pramsey commented Sep 30, 2019

For those who stumble across this and are, the PostGIS resolution is here https://trac.osgeo.org/postgis/ticket/4500

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants