PostgresMetadataExtractor doesn't discover Redshift late binding views

[nathanlawrence-asana]

Expected Behavior

We're heavy users of late binding views in Redshift (e.g. with no schema binding). I believe that those views should show up in Amundsen (non-late binding views seem to show up just fine), but are not currently showing up in Amundsen.

Possible Solution

The root issue is that views don't actually show up in pg_tables or in the information_schema tables.

The correct way to pull the metadata for these tables is by running the built-in function PG_GET_LATE_BINDING_VIEW_COLS() or by querying against svv_columns (note that unfortunately, neither of these seem to include description metadata).

See amundsen-io/amundsendatabuilder#356 as a possible solution.

Steps to Reproduce

1. Create a Redshift late binding view
2. Ingest the Redshift database using the PostgresMetadataExtractor
3. Verify that the late binding view wasn't ingested

Screenshots (if appropriate)

Context

A number of our late-binding views are very heavily used and so it's important that analysts and data scientists who are using Amundsen can use the tool to look up the table structure.

Your Environment

• Amunsen version used:
• Data warehouse stores: Redshift
• Deployment (k8s or native):
• Link to your fork or repository:

[jornh]

@nathanlawrence-asana I guess with amundsen-io/amundsendatabuilder#356 merged this issue is resolved, so you can close it?

[nathanlawrence-asana]

Yep! Thank you.

[nathanlawrence-asana]

@jornh or @feng-tao , do we need to update the Status: label as well? (I don't think I have permissions there).

[feng-tao]

thanks @nathanlawrence-asana @jornh , label is updated as well.