Provide the native OID of PG type in pg_type

[goldmedal]

Description

DuckDB provide the internal OID of a type in pg_type.oid. However, the type OID is a constant in the native Postgres. Some pg driver will keep a cache of type info, such as JDBC. To avoid incorrect type mapping, we should provide the native OID in pg_type.

What's Changed

• Create the mapping for the native PG type.
• Provide native pg name in typname if it can be mapped to the native pg type.
• Provide numeric type.
• Filter the duplicate record if its type_oid is null.

Sample Result

D select oid, typname from pg_type;
┌───────┬──────────────┐
│  oid  │   typname    │
│ int64 │   varchar    │
├───────┼──────────────┤
│    20 │ int8         │
│    17 │ bytea        │
│  1560 │ bit          │
│    16 │ bool         │
│  1043 │ varchar      │
│  1082 │ date         │
│  1114 │ timestamp    │
│  1700 │ numeric      │
│   701 │ float8       │
│       │ enum         │
│   700 │ float4       │
│  2950 │ uuid         │
│       │ hugeint      │
│    23 │ int4         │
│       │ tinyint      │
│    21 │ int2         │
│  1186 │ interval     │
│       │ list         │
│       │ map          │
│       │ null         │
│       │ struct       │
│  1083 │ time         │
│  1184 │ timestamptz  │
│       │ timestamp_ms │
│       │ timestamp_ns │
│       │ timestamp_s  │
│  1266 │ timetz       │
│       │ ubigint      │
│       │ uhugeint     │
│       │ usmallint    │
│       │ uinteger     │
│       │ utinyint     │
│       │ union        │
├───────┴──────────────┤
│ 33 rows    2 columns │
└──────────────────────┘

Other DuckDB types have the oid field set to null in pg_type.

If there are custom enums, they will be set to the internal OID in DuckDB.

D CREATE TYPE greeting AS ENUM('hi', 'bonjour', 'konnichiwa', 'howdy')
  ;
D select oid, typname from pg_type where oid is not null;
┌───────┬─────────────────────────────┐
│  oid  │           typname           │
│ int64 │           varchar           │
├───────┼─────────────────────────────┤
...
│  1369 │ greeting                    │
...
├───────┴─────────────────────────────┤
│ 17 rows                   2 columns │
└─────────────────────────────────────┘
D select * from pg_enum;
┌───────┬───────────┬───────────────┬────────────┐
│  oid  │ enumtypid │ enumsortorder │ enumlabel  │
│ int32 │   int64   │     int32     │  varchar   │
├───────┼───────────┼───────────────┼────────────┤
│       │      1369 │             4 │ howdy      │
│       │      1369 │             3 │ konnichiwa │
│       │      1369 │             2 │ bonjour    │
│       │      1369 │             1 │ hi         │
└───────┴───────────┴───────────────┴────────────┘

[Mytherin]

This is not robust - the oids in DuckDB are not guaranteed to be constants and might change over time. Could you rework this to use the names of the types instead?

[goldmedal]

Thanks for the comments. I have addressed it in the latest commits.

[Mytherin]

Thanks for the PR! I agree we should match Postgres on the oids here - but this needs to be done robustly, see my comment.

[goldmedal]

I found the extra 2 issue here:

• The typname isn't the native pg type.
  I think that we also provide native name here is better.
• numeric type is missing
  Previously, we filter duckdb_types() with type_size is not null. However, numeric should be provided actually. This condition only filter the two record, numeric and enum. I added extra condition to exclude enum, then added type_oid is not null to filter the duplicate record.

I also updated the PR description.

[Tishj]

In the PR you state:

Provide numeric type

But in the old code, this was already present?

logical_type='DECIMAL' then 'numeric'

[Tishj]

I think we should use ILIKE here, so this doesn't happen

D select pg_catalog.format_pg_type('DECIMAL', 'test');
┌─────────────────────────────┐
│ pg_type2('DECIMAL', 'test') │
│           varchar           │
├─────────────────────────────┤
│ numeric                     │
└─────────────────────────────┘
D select pg_catalog.format_pg_type('decimal', 'test');
┌─────────────────────────────┐
│ pg_type2('decimal', 'test') │
│           varchar           │
├─────────────────────────────┤
│ decimal                     │
└─────────────────────────────┘

[goldmedal]

In the PR you state:

Provide numeric type

But in the old code, this was already present?

logical_type='DECIMAL' then 'numeric'

It's weird. I tried the v0.10.2 version

v0.10.2 1601d94f94
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
D select oid, typname from pg_type where typname = 'numeric';
┌───────┬─────────┐
│  oid  │ typname │
│ int64 │ varchar │
├─────────────────┤
│     0 rows      │
└─────────────────┘

I can't find any result.

The definition of the original pg_type, it includes a filter

FROM duckdb_types() WHERE type_size IS NOT NULL

However, I found the type_size of DECIMAL is always null.

D select type_name, type_size from duckdb_types() where type_name = 'numeric';
┌───────────┬───────────┐
│ type_name │ type_size │
│  varchar  │   int64   │
├───────────┼───────────┤
│ numeric   │           │
│ numeric   │           │
│ numeric   │           │
└───────────┴───────────┘

I'm not sure about that but I guess it's a bug.

I think we should use ILIKE here, so this doesn't happen

@Tishj Many thanks! I'll fix them today.

[goldmedal]

I think we should use ILIKE here, so this doesn't happen

Instead of multiple WHEN logical_type ILIKE xxx THEN, I chose upper(logical_type to make the code cleaner.

In the PR you state:

Provide numeric type

But in the old code, this was already present?

logical_type='DECIMAL' then 'numeric'

I think I found why DuckDB handle DECIMAL here previously. In test/sql/pg_catalog/sqlalchemy.test, The # get_columns test will use format_type which invokes format_pg_type internally. The test case asserts that numeric should be provided. However, it doesn't query pg_type but get type from the table metadata. That's why DuckDB handled numeric but no one found it's missing in pg_type.

[Mytherin]

Thanks!