Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ADAP-435] [Bug] Cannot use begin/end scripting statement with snowflake_dml_explicit_transaction #556

Open
2 tasks done
OSalamaPret opened this issue Apr 10, 2023 · 6 comments · May be fixed by #887
Open
2 tasks done
Labels
bug Something isn't working good_first_issue Good for newcomers

Comments

@OSalamaPret
Copy link

OSalamaPret commented Apr 10, 2023

Is this a new bug in dbt-snowflake?

  • I believe this is a new bug in dbt-snowflake
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

I am writing a custom incremental materialization which calls {{ strategy_sql_macro_func(strategy_arg_dict) }} within a Snowflake scripting begin/end block.
Example:

  {%- call statement('main') -%}
    CREATE OR REPLACE TASK batch.test schedule = 'USING CRON 0 17 * * * Europe/London' AS
    EXECUTE IMMEDIATE
    $$
    BEGIN
      {{ snowflake__create_view_as_with_temp_flag(tmp_relation, compiled_code, True) }}
      {{ strategy_sql_macro_func(strategy_arg_dict) }}
    END;
    $$;
  {%- endcall -%}

When this runs, it generates the following error syntax error line 43 at position 7 unexpected ';'.

Expected Behavior

I expect that we should be able to mix Snowflake begin/end scripting blocks with transactions, as this is fully supported in Snowflake.

The issue is due to the snowflake_dml_explicit_transaction macro using begin to start the transaction, which, in this context, Snowflake is seeing as a begin/end block.
The fix is to use the explicit begin transaction instead of begin

I have overriden the snowflake_dml_explicit_transaction to use begin transaction and confirm that my issue is resolved.

Steps To Reproduce

Quickest route to reproduce:

  1. Modify the Snowflake incremental materialization and change the {%- call statement('main') -%} to following:
{%- call statement('main') -%}
    CREATE OR REPLACE TASK test_db.test_schema.test_task AS
    EXECUTE IMMEDIATE
    $$
    BEGIN
      {{ snowflake__create_view_as_with_temp_flag(tmp_relation, compiled_code, True) }}
      {{ strategy_sql_macro_func(strategy_arg_dict) }}
    END;
    $$;
  {%- endcall -%}
  1. Create a Snowflake incremental model, eg test_model.sql:
{{
    config(
        materialized='incremental'
    )
}}

select 'test' as test_query, '2023-04-10' as created_date 
{% if is_incremental() %}
  WHERE created_date > (select max(created_date) from {{ this }})
{% endif %}
  1. Run dbt run --select test_model.sql

Relevant log output

No response

Environment

- OS:Ubuntu 20.04 (running under Windows using WSL)
- Python: 3.10.5
- dbt-core: 1.4.5
- dbt-snowflake: 1.4.2

Additional Context

@OSalamaPret OSalamaPret added bug Something isn't working triage labels Apr 10, 2023
@github-actions github-actions bot changed the title [Bug] Cannot use begin/end scripting statement with snowflake_dml_explicit_transaction [ADAP-435] [Bug] Cannot use begin/end scripting statement with snowflake_dml_explicit_transaction Apr 10, 2023
@dbeatty10
Copy link
Contributor

Thanks for reaching out @OSalamaPret !

Would you be interested in raising a PR with the fix you proposed?

@OSalamaPret
Copy link
Author

Thanks for reaching out @OSalamaPret !

Would you be interested in raising a PR with the fix you proposed?

I tried, but not sure I'm able to sign the CLA on behalf of my company! I guess I could create a personal account and use that though. Will try it out when I get some free time.

@github-actions
Copy link
Contributor

github-actions bot commented Oct 8, 2023

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days.

@github-actions github-actions bot added the Stale label Oct 8, 2023
@dbeatty10 dbeatty10 added the good_first_issue Good for newcomers label Oct 9, 2023
@dbeatty10
Copy link
Contributor

The suggested fix is here, namely:

-     begin;
+     begin transaction;

I'm labeling this as a "good first issue" if anyone wants to pick it up.

@github-actions github-actions bot removed the Stale label Oct 10, 2023
@shreyadatar
Copy link

@dbeatty10 I'd be happy to pick this up. Will start going through the contributing guidelines!

@lokofoko
Copy link

Any updates here? Issue is still persist and it is impossible to create procedure in snowflake from dbt.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working good_first_issue Good for newcomers
Projects
None yet
4 participants