(fix)Exposures definitions and added ability to preserve columns #221
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR addresses a few issues with the exposures sync:
name
for exposures, which could contain incompatible characters: synced dbt exposures name need to be an identifier #207 and Exposure names contain invalid characters #177query_context
value until they are viewed in Explore, which could cause: syncing dbt exposures fail #206This PR also introduces a new flag to the dbt sync:
--preserve-columns
. By default, the sync would update existing datasets in Superset withoverride_columns=True
, which resets the column definition (groupby
andfilterable
configurations) and also sync columns from the dataset. This could potentially misconfigure existing columns in Superset, or bring back to the datasets columns that were intentionally excluded. This behavior will be kept by default (since the main use-case for the dbt integration is to have the dbt info as the source of truth), however if the--preserve-columns
flag is included in the dbt sync command, the update dataset API calls would includeoverride_columns=False
, preserving column configurations and avoiding a column sync.