Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

[#3961] Enable cataloging of unlogged Postgres tables #3993

Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
### Fixes
- Add generic tests defined on sources to the manifest once, not twice ([#3347](https://github.com/dbt-labs/dbt/issues/3347), [#3880](https://github.com/dbt-labs/dbt/pull/3880))
- Skip partial parsing if certain macros have changed ([#3810](https://github.com/dbt-labs/dbt/issues/3810), [#3982](https://github.com/dbt-labs/dbt/pull/3892))
- Enable cataloging of unlogged Postgres tables ([3961](https://github.com/dbt-labs/dbt/issues/3961))
samlader marked this conversation as resolved.
Show resolved Hide resolved

### Under the hood

Expand Down
2 changes: 1 addition & 1 deletion plugins/postgres/dbt/include/postgres/macros/catalog.sql
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@
{%- endfor -%}
)
and not pg_is_other_temp_schema(sch.oid) -- not a temporary schema belonging to another session
and tbl.relpersistence = 'p' -- [p]ermanent table. Other values are [u]nlogged table, [t]emporary table
and tbl.relpersistence in ('p', 'u') -- [p]ermanent table or [u]nlogged table. Exclude [t]emporary tables
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

馃憤

and tbl.relkind in ('r', 'v', 'f', 'p') -- o[r]dinary table, [v]iew, [f]oreign table, [p]artitioned table. Other values are [i]ndex, [S]equence, [c]omposite type, [t]OAST table, [m]aterialized view
and col.attnum > 0 -- negative numbers are used for system columns such as oid
and not col.attisdropped -- column as not been dropped
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
version: 2
models:
- name: table_unlogged
description: "Unlogged table model"
columns:
- name: column_a
description: "Sample description"
quote: true
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{{ config(materialized = 'table', unlogged = True) }}

select 1 as column_a
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
import re
import json
from test.integration.base import DBTIntegrationTest, use_profile


class TestPostgresUnloggedTable(DBTIntegrationTest):
@property
def schema(self):
return "postgres_unlogged_074"

@property
def models(self):
return "models"

@property
def project_config(self):
return {
'config-version': 2,
'models': {
'test': {
'materialized': 'table',
'+persist_docs': {
"relation": True,
"columns": True,
},
}
}
}

@use_profile('postgres')
def test__postgres__unlogged__table__catalog(self):
table_name = 'table_unlogged'

results = self.run_dbt(['run', '--models', table_name])
self.assertEqual(len(results), 1)

assert self.get_table_persistence(table_name) == 'u'

self.run_dbt(['docs', 'generate'])

with open('target/catalog.json') as fp:
catalog_data = json.load(fp)

assert len(catalog_data['nodes']) == 1

table_node = catalog_data['nodes'][f'model.test.{table_name}']
assert 'column_a' in table_node['columns']

def get_table_persistence(self, table_name):
sql = """
SELECT
relpersistence
FROM pg_class
WHERE relname = '{table_name}'
"""
sql = sql.format(table_name=table_name, schema=self.unique_schema())
result, = self.run_sql(sql, fetch='one')
self.assertEqual(len(result), 1)
return result