Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] When derived_columns are provided, all source columns are transformed to uppercase #166

Closed
anouar-zh opened this issue Oct 24, 2022 · 1 comment
Assignees
Labels
bug Something isn't working

Comments

@anouar-zh
Copy link

Describe the bug
As soon as a use derived_columns: containing a couple of derived columns, suddenly all source columns are transformed to uppercase while I expect them to be unchanged.

Environment

dbt version: 1.3.0
dbtvault version: 0.9.0
Database/Platform: databricks

To Reproduce

The following results in source columns as they where: (lowercase)

{{config(materialized='view')}}

{% set yaml_metadata %}
source_model: raw_stores
{% endset %}  

{% set metadata_dict = fromyaml(yaml_metadata) %}

{% set source_model = metadata_dict['source_model'] %}

{{ dbtvault.stage(include_source_columns=true,
                  source_model=source_model,
                  derived_columns=derived_columns,
                  null_columns=null_columns,
                  hashed_columns=hashed_columns,
                  ranked_columns=ranked_columns) }}

The following code results in a view in which all columns are suddenly uppercase:

{{config(materialized='view')}}

{% set yaml_metadata %}
source_model: raw_stores
derived_columns: 
   RECORD_SOURCE: "!SOURCE"
   LOAD_DATE: DNAHeader_date
   EFFECTIVE_FROM: EffectiveDate

{% endset %}  

{% set metadata_dict = fromyaml(yaml_metadata) %}

{% set source_model = metadata_dict['source_model'] %}
{% set derived_columns = metadata_dict['derived_columns'] %}

{{ dbtvault.stage(include_source_columns=true,
                  source_model=source_model,
                  derived_columns=derived_columns,
                  null_columns=null_columns,
                  hashed_columns=hashed_columns,
                  ranked_columns=ranked_columns) }}

Expected behavior
I expect the source column names to be unchanged

@anouar-zh anouar-zh added the bug Something isn't working label Oct 24, 2022
@DVAlexHiggs DVAlexHiggs changed the title [BUG] when derived_columns is used as key in stage model all source columns are transformed to uppercase [BUG] When derived_columns are provided, all source columns are transformed to uppercase Oct 25, 2022
@DVAlexHiggs
Copy link
Member

0.9.1 Released which addresses this issue. Thanks! Please re-open if this continues

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants