[CT-343] partial parse conflicts with generate_schema_name changes #4850
Comments
github-actions
bot
changed the title
|
I have not been able to recreate this problem (though I don't have access to a Windows machine to test). Is there anything in the logs? Can you share your generate_schema_name macro? Could you try removing all files except the partial_parse.msgpack file from the target directory? Since the msgpack file has been loaded (or the error would be earlier), I'm wondering if it's one of the other files in the target directory that's the issue. |
|
I tried reproducing the issue on a fresh project and the error didn't arise. |
|
Thank you for opening this issue. This also happened to me. I was running DBT inside of a docker container:
running poetry with this The CI/CD pipeline broke because of this issue and I when I added the dbt clean and dbt deps commands to my workflow, it worked. |
|
Hi, I got the same error when creating my own custom "source" macro. The macro itself just calls the builtins.source and does nothing else. It works fine when first do a "dbt clean" and a "dbt deps" like described above. But when I change something in this file (might just be a comment) I get the output below and first need to do a clean and dpes again to get it working. I have the dbt_utils, spark_utils and dbt_external_tables packages installed. I use the dbt-databricks adapter and I get the error while running in Visual Studio Code on a WIN11 machine. Hope this helps with finding the issue. |
|
I hit this running DBT in a docker container whenever I changed the |
|
Getting the same error in a fresh project that uses macros for generating schema and aliases names. |
|
I have this issue now, updated the generate_schema_name macro, with a minor change. then it now fails with the above mentioned error msgs. But I am not able to run a dbt deps (for Cloud) as suggested here as the UI instantly fails and gives me this message: |
|
I've got a repro of this one (ensure partial parsing hasn't been accidentally disabled). # dbt_project.yml
name: "my_dbt_project"
version: "1.0.0"
config-version: 2
profile: "snowflake"
models:
my_dbt_project:
+materialized: table
+database: development-- models/foo.sql
select 1 as user_id-- macros/generate_database_name.sql
{% macro generate_database_name(custom_database_name, node) -%}
{%- set default_database = target.database -%}
{%- if target.name == 'dev' -%}
{{ default_database }}
{%- elif target.name == 'prod' -%}
{{ custom_database_name | trim }}
{%- else -%}
{{ default_database }}
{%- endif -%}
{%- endmacro %}Key to this is you want to add a file
-- macros/generate_database_name.sql
{% macro generate_database_name(custom_database_name, node) -%}
{%- set default_database = target.database -%}
{%- if target.name == 'dev' -%}
{{ default_database }}
{%- elif target.name == 'ci' -%}
{{ custom_database_name | trim }}
{%- else -%}
{{ default_database }}
{%- endif -%}
{%- endmacro %}
Good ol yaml trying to parse an empty on the second go around - but the first time is okay though (full parse) because we do not try and read |
|
This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days. |
|
Although we are closing this issue as stale, it's not gone forever. Issues can be reopened if there is renewed community interest. Just add a comment to notify the maintainers. |
1 similar comment
|
Although we are closing this issue as stale, it's not gone forever. Issues can be reopened if there is renewed community interest. Just add a comment to notify the maintainers. |
Is there an existing issue for this?
Current Behavior
We have a project with a custom generate_schema_name macro.
If I change the macro generate_schema_name, project compilation fails with error
a string or stream input is required.If I run a
dbt cleanbefore compiling, compilation completes without errors.Expected Behavior
Compilation should not depend on
dbt cleanSteps To Reproduce
dbt clean
dbt deps
dbt compile --> no errors
modify generate_schema_name macro (even adding a jinja comment):
dbt compile --> error
10:01:21 Running with dbt=1.0.3
10:01:22 Change detected to override macro used during parsing. Starting full parse.
10:01:34 Encountered an error:
a string or stream input is required
Relevant log output
No response
Environment
What database are you using dbt with?
bigquery
Additional Context
No response
The text was updated successfully, but these errors were encountered: