PROJ core dumps when setting data directory on specific context and it is not set on global context

[snowman2]

pyproj4/pyproj#419

This is a wierd edge case that has taken me some time to track down as the cause has been elusive for quite some time due to the core dumping. However, I believe the symptoms and the solution should help find a solution in PROJ I believe.

In the case where the data directory is not able to be determined by PROJ in the default locations on the global context, it will core dump when attempting to use proj_create with a perfectly valid custom context that has the proper data directory set. However, this does not happen in all instances. It only happens when using the +init= syntax.

For example:

>>> from pyproj import CRS
>>> cc = CRS("+proj=utm +zone=33 +ellps=WGS84 +datum=WGS84 +units=m +no_defs")
>>> cc = CRS(init="epsg:4326")
Segmentation fault (core dumped)

My guess is it has to do with:

PROJ/src/iso19111/c_api.cpp

Lines 437 to 440 in f059b90

	// Only connect to proj.db if needed
	if (strstr(text, "proj=") == nullptr || strstr(text, "init=") != nullptr) {
	getDBcontextNoException(ctx, __FUNCTION__);
	}

The fix I applied in pyproj was to set the data directory for the global PROJ context along with the custom context.

[snowman2]

The tricky part is to have the data directory in a place where PROJ cannot find it and then use proj_context_set_search_paths to set the data directory on a different context.

[snowman2]

Note: This issue began in PROJ 6.1 and still persists in 6.2

[snowman2]

This is also problematic for pyproj wheels when the PROJ_LIB is set to an old PROJ data directory. The global context finds the old data directory without the proj.db and proceeds to core dump even though the custom context has the data directory paths set properly..

[rouault]

I tried to reproduce, but couldn't manage. A bit confused by your explanations. Could you give an exact way to reproduce (possibly with a standalone C file) or perhaps display the stack trace where it crashes ?

gdb python
[ set args myscript.py ]
run
[ python code and wait for it to crash ]
bt

[kbevers]

possibly with a standalone C file

As a side note, I generally find it quite difficult to debug problems presented in the form of pyproj code. If the problem can be reproduced using PROJ apps or a bit of C code that it is preferable.

[snowman2]

I definitely see that the explanation is confusing. Code is definitely a good way to communicate. Hopefully this will help:

Step 1:

Move the PROJ data directory to a non-standard location where it cannot be auto detected and is not set by PROJ_LIB.

Step 2:

Create a new context and set the search paths without setting it in the global context. Then, attempt to create a CRS using the init= syntax:

std::string dirname = "/path/to/proj_data/share234/proj";
auto ctx = proj_context_create();

const char *path = dirname.c_str();
proj_context_set_search_paths(ctx, 1, &path);
proj_context_use_proj4_init_rules(ctx, 1);
auto P = proj_create(ctx, "+init=epsg:4326 +type=crs");

Updated to allow init rules ^^

[rouault]

I've tested your above (updated) script, and can't make it crash

$ PROJ_DEBUG=5 LD_LIBRARY_PATH=src/.libs ./mytest 
pj_open_lib(proj.db): call fopen(/path/to/proj_data/share234/proj/proj.db) - failed
internal_proj_create: Cannot find proj.db
pj_open_lib(proj.db): call fopen(/path/to/proj_data/share234/proj/proj.db) - failed
pj_open_lib(epsg): call fopen(/home/even/proj/install-proj-master/share/proj/epsg) - failed
internal_proj_create: no database context specified

The hardcoded data path is (/home/even/proj/install-proj-master/share/proj and I renamted it to (/home/even/proj/install-proj-master/share/proj.disabed

[snowman2]

One step closer. Can you try with:

std::string dirname = "/home/even/proj/install-proj-master/share/proj.disabed";

The reason is that the new context needs the path to the PROJ data directory in its moved form while the main/global PROJ context cannot find it.

[rouault]

One step closer. Can you try with:

std::string dirname = "/home/even/proj/install-proj-master/share/proj.disabed";

Still no crash. New output is

PROJ_DEBUG=5 LD_LIBRARY_PATH=src/.libs ./mytest 
pj_open_lib(proj.db): call fopen(/home/even/proj/install-proj-master/share/proj.dis/proj.db) - succeeded
pj_open_lib(epsg): call fopen(/home/even/proj/install-proj-master/share/proj/epsg) - failed

[snowman2]

Hmmm, strange. Wonder what the difference is on my end. That's where it core dumps in the pyproj code. I wonder if running as a C program instead of Cpp makes any difference?

[rouault]

I wonder if running as a C program instead of Cpp makes any difference?

I doubt so. Why not running your python code under gdb and displaying the stack trace when it crashes as I suggested above in #1574 (comment) ?

[snowman2]

The output is here: pyproj4/pyproj#368 (comment)

[snowman2]

Another one is here: pyproj4/pyproj#351