Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pipe the dump of the ogr2ogr command when using PgPool #215

Closed
giohappy opened this issue Nov 30, 2023 · 0 comments · Fixed by #216
Closed

Pipe the dump of the ogr2ogr command when using PgPool #215

giohappy opened this issue Nov 30, 2023 · 0 comments · Fixed by #216
Assignees

Comments

@giohappy
Copy link
Contributor

giohappy commented Nov 30, 2023

Introduction

In the end, it looks like the problem with the COPY command and PgPool is due to OGR.

Even Rouault confirms he was able to reproduce the issue and he made a commit to add a warning inside the page of the Postgresql OGR driver:

image

Solution

The solution is to dump the command generated by ogr2ogr to psql through /vsistdout/.

Example:
/usr/bin/ogr2ogr --config PG_USE_COPY YES -f PGDump /vsistdout/ "/usr/src/geonode/pgpool_tests/it_1km/it_1km.shp" -nln it_1km "it_1km" -lco precision=no -lco DIM=2 -lco GEOMETRY_NAME=geometry | psql -d geonode_data -h pgpool -U postgres -f -

The time it takes to complete is comparable to the original command executed directly on PostgreSQL.

By the way, since this is less efficient then piping directly from OGR, I would introduce a new setting OGR2OGR_COPY_WITH_DUMP that can be optionally set to True to switch the default (current) command.

I would also remove the PG_USE_COPY setting, and the option to switch to INSERT statements because it proved to be extremely slow and inefficient.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants