Failed to import SHP multipolygons into PostGIS working copy

[SrNetoChan]

Describe the bug
This error is returned while trying to import a shapefile containing multifeature into a PostGIS working copy.

C:\Users\testing\pdm_tests>kart import z:\Fix_PDM\multi_test.shp
Starting git-fast-import...
Importing 4 features from z:\Fix_PDM\multi_test.shp:multi_test to multi_test/ ...
Added 4 Features to index in 0.0s
Overall rate: 4000 features/s)
Closed in 0s
Joining 2 parallel-imported trees...
Joined trees in 0s
Updating postgresql://postgres@localhost/pdm_tests/pdm ...
Traceback (most recent call last):
  File "lib\site-packages\sqlalchemy\engine\base.py", line 1783, in _execute_context
  File "lib\site-packages\sqlalchemy\dialects\postgresql\psycopg2.py", line 957, in do_executemany
  File "lib\site-packages\psycopg2\extras.py", line 1292, in execute_values
psycopg2.errors.InvalidParameterValue: Geometry type (MultiPolygon) does not match column type (Polygon)


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "kart_cli.py", line 4, in <module>
  File "kart\cli.py", line 351, in entrypoint
  File "lib\site-packages\click\core.py", line 829, in __call__
  File "lib\site-packages\click\core.py", line 782, in main
  File "kart\cli.py", line 159, in invoke
  File "lib\site-packages\click\core.py", line 1259, in invoke
  File "lib\site-packages\click\core.py", line 1066, in invoke
  File "lib\site-packages\click\core.py", line 610, in invoke
  File "lib\site-packages\click\decorators.py", line 21, in new_func
  File "kart\init.py", line 367, in import_
  File "kart\init.py", line 60, in _add_datasets_to_working_copy
  File "kart\working_copy\base.py", line 943, in write_full
  File "lib\site-packages\sqlalchemy\orm\session.py", line 1692, in execute
  File "lib\site-packages\sqlalchemy\engine\base.py", line 1614, in _execute_20
  File "lib\site-packages\sqlalchemy\sql\elements.py", line 326, in _execute_on_connection
  File "lib\site-packages\sqlalchemy\engine\base.py", line 1491, in _execute_clauseelement
  File "lib\site-packages\sqlalchemy\engine\base.py", line 1846, in _execute_context
  File "lib\site-packages\sqlalchemy\engine\base.py", line 2027, in _handle_dbapi_exception
  File "lib\site-packages\sqlalchemy\util\compat.py", line 207, in raise_
  File "lib\site-packages\sqlalchemy\engine\base.py", line 1783, in _execute_context
  File "lib\site-packages\sqlalchemy\dialects\postgresql\psycopg2.py", line 957, in do_executemany
  File "lib\site-packages\psycopg2\extras.py", line 1292, in execute_values
sqlalchemy.exc.DataError: (psycopg2.errors.InvalidParameterValue) Geometry type (MultiPolygon) does not match column type (Polygon)

[SQL: INSERT INTO pdm.multi_test ("FID", geom, id) VALUES (%(FID)s, %(geom)s, %(id)s)]
[parameters: ({'FID': 0, 'geom': <psycopg2.extensions.Binary object at 0x000001D9CB094720>, 'id': None}, {'FID': 1, 'geom': <psycopg2.extensions.Binary object at 0x000001D9CB094750>, 'id': None}, {'FID': 2, 'geom': <psycopg2.extensions.Binary object at 0x000001D9CB094780>, 'id': None}, {'FID': 3, 'geom': <psycopg2.extensions.Binary object at 0x000001D9CB0947B0>, 'id': None})]
(Background on this error at: https://sqlalche.me/e/14/9h9h)
[3272] Failed to execute script kart_cli

The problem does not happen if the shape only has single polygons features. It also doesn't happen if the working copy is a geopackage.

To Reproduce
Steps to reproduce the behaviour:

1. Create a shapefile of polygons and create several single polygons and at least a multipolygon feature or use the following example file:

https://mega.nz/file/sAFTHQrB#XMWdLccbPlCWcCY0ypJ4irlyxw3XzZ7M_FNGp8dvclo

2. create a repository with a postgis working copy
3. use kart import to import the shapefile to the repository
4. See error

Expected behaviour
What did you expect to happen?

Output
Add the output you're seeing to help explain your problem.

**Version Info **

• OS: Windows 10
  Kart v0.11.1, Copyright (c) Kart Contributors
  » GDAL v3.3.2; PROJ v8.1.1
  » PyGit2 v1.9.0; Libgit2 v1.4.2; Git v2.34.0.windows.1.13.g93318cbc8d
  » SQLAlchemy v1.4.29; pysqlite3 v2.6.0/v3.31.1; SpatiaLite v5.0.0; Libpq v11.0.9
  » SpatialIndex v1.9.3

[olsen232]

Haven't actually gotten to this yet but from the output it's clear that the geometry column is being imported as type POLYGON which causes issues as the geometry column actually contains at least one MULTIPOLYGON, which PostGIS won't allow in a POLYGON column.
It's likely that importing the same data and creating GPKG working copy results in the same mix of types, but GPKG is less strict about such things.

As a workaround in the meantime, you could try to make sure that all geometries in your source file are of uniform type - all POLYGON, or all MULTIPOLYGON.

[SrNetoChan]

@olsen232 that's the thing. Shapefiles are by definition always multipolygons, even if they only contain single polygons. I think that the problem relies in the fact that kart assumes that the shapefile only contains single polygons (maybe by reading the first element?!) I haven't tried importing a shapefile only containing multipolygons to see what happens.

[olsen232]

Ah I see. We don't parse shapefiles directly but use osgeo.ogr to open them and adapt them to its common interface - I don't know off the top of my head if OGR reports the column-type by reading the appropriate column-metadata or by sampling the first element (or a mixture of both depending on whether the column-metadata contains all the relevant information).

If I understand correctly, this means the fix should be straight forward in this instance - if OGR says "POLYGON" then we should import as MULTIPOLYGON, same logic for POLYLINE. I'll test to see if POINT and MULTIPOINT work as expected since it seems shapefiles treat those as distinct.