Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unhandled exception when importing from a Postgres DB which doesn't have the postgis extension #728

Closed
pfw opened this issue Nov 7, 2022 · 2 comments
Assignees
Labels
good first issue Good for newcomers

Comments

@pfw
Copy link
Contributor

pfw commented Nov 7, 2022

Describe the bug

kart import dumps a stack trace rather than showing an error when importing from a Postgres DB which hasn't had `CREATE EXTENSION...' run.

To Reproduce
Steps to reproduce the behaviour:

  1. Create a new PG database
  2. Import some data eg. ogr2ogr -f PostgreSQL PG:dbname=data_gov_au mdb-boundaries.geojson
  3. In an existing Kart repo try to import that table kart import postgresql://pfw:@localhost/data_gov_au mdb_boundaries
  4. See error

If between step 1 and 2 create EXTENSION postgis; is run then it will complete correctly.

Expected behaviour
Kart should probably show an error on Postgis missing

Kart show allow importing (non-spatial) tables from PostgreSQL installs that don't have postGIS installed.

Output

Traceback (most recent call last):
  File "sqlalchemy/engine/base.py", line 1901, in _execute_context
  File "sqlalchemy/engine/default.py", line 732, in do_execute
psycopg2.errors.UndefinedFunction: function postgis_typmod_type(integer) does not exist
LINE 6:                 upper(postgis_typmod_type(A.atttypmod)) AS g...
                              ^
HINT:  No function matches the given name and argument types. You might need to add explicit type casts.


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "kart_cli.py", line 4, in <module>
  File "kart/cli.py", line 392, in entrypoint
  File "click/core.py", line 1130, in __call__
  File "click/core.py", line 1055, in main
  File "kart/cli_util.py", line 68, in invoke
  File "click/core.py", line 1657, in invoke
  File "click/core.py", line 1404, in invoke
  File "click/core.py", line 760, in invoke
  File "click/decorators.py", line 26, in new_func
  File "kart/tabular/import_.py", line 322, in import_
  File "kart/tabular/import_source.py", line 57, in check_valid
  File "kart/list_of_conflicts.py", line 91, in check_sources_are_importable
  File "kart/tabular/sqlalchemy_import_source.py", line 229, in meta_items
  File "kart/tabular/sqlalchemy_import_source.py", line 237, in meta_items_from_db
  File "kart/sqlalchemy/adapter/base.py", line 155, in all_v2_meta_items
  File "kart/utils.py", line 22, in wrapper
  File "kart/sqlalchemy/adapter/postgis.py", line 176, in all_v2_meta_items_including_empty
  File "sqlalchemy/orm/session.py", line 1712, in execute
  File "sqlalchemy/engine/base.py", line 1705, in _execute_20
  File "sqlalchemy/sql/elements.py", line 334, in _execute_on_connection
  File "sqlalchemy/engine/base.py", line 1582, in _execute_clauseelement
  File "sqlalchemy/engine/base.py", line 1944, in _execute_context
  File "sqlalchemy/engine/base.py", line 2125, in _handle_dbapi_exception
  File "sqlalchemy/util/compat.py", line 208, in raise_
  File "sqlalchemy/engine/base.py", line 1901, in _execute_context
  File "sqlalchemy/engine/default.py", line 732, in do_execute
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedFunction) function postgis_typmod_type(integer) does not exist
LINE 6:                 upper(postgis_typmod_type(A.atttypmod)) AS g...
                              ^
HINT:  No function matches the given name and argument types. You might need to add explicit type casts.

[SQL: 
            SELECT
                C.column_name, C.ordinal_position, C.data_type, C.udt_name,
                C.character_maximum_length, C.numeric_precision, C.numeric_scale,
                PK.ordinal_position AS pk_ordinal_position,
                upper(postgis_typmod_type(A.atttypmod)) AS geometry_type,
                postgis_typmod_srid(A.atttypmod) AS geometry_srid
            FROM information_schema.columns C
            LEFT OUTER JOIN (
            SELECT KCU.* FROM information_schema.key_column_usage KCU
            INNER JOIN information_schema.table_constraints TC
            ON KCU.constraint_schema = TC.constraint_schema
            AND KCU.constraint_name = TC.constraint_name
            WHERE TC.constraint_type = 'PRIMARY KEY'
        ) PK
            ON (PK.table_schema = C.table_schema)
            AND (PK.table_name = C.table_name)
            AND (PK.column_name = C.column_name)
            LEFT OUTER JOIN pg_attribute A
            ON (A.attname = C.column_name)
            AND (A.attrelid = (%(table_identifier)s)::regclass::oid)
            WHERE C.table_schema=%(table_schema)s AND C.table_name=%(table_name)s
            ORDER BY C.ordinal_position;
        ]
[parameters: {'table_identifier': 'public.mdb_boundaries', 'table_schema': 'public', 'table_name': 'mdb_boundaries'}]
(Background on this error at: https://sqlalche.me/e/14/f405)
[48283] Failed to execute script 'kart_cli' due to unhandled exception!

Version Info

❯ kart --version
Kart v0.11.5, Copyright (c) Kart Contributors
» GDAL v3.4.0; PROJ v6.3.0
» PyGit2 v1.9.0; Libgit2 v1.4.2; Git v2.33.GIT
» SQLAlchemy v1.4.40; pysqlite3 v2.6.0/v3.31.1; SpatiaLite v5.0.1; Libpq v14.0.5
» SpatialIndex v1.9.3
@pfw pfw changed the title Unhandled exception when importing from a Postgis DB which doesn't have the postgis extension Unhandled exception when importing from a Postgres DB which doesn't have the postgis extension Nov 7, 2022
@olsen232 olsen232 added the good first issue Good for newcomers label Feb 2, 2023
@jhunjhunwala1234
Copy link

Hi, I would like to work on this bug. it would be great could you please assign this issue to me

@rcoup
Copy link
Member

rcoup commented Feb 19, 2023

@jhunjhunwala1234 Great! Open a PR when you have something ready.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

5 participants