Use tmp table in static insert overwrite

[github-christophe-oudar]

Resolves #427
Resolves #556

Description

As described in #427, static insert overwrite, when involving on_schema_change != 'ignore' would create a temp table to check for a schema change anyway.
It fixes as well:

• In case, on_schema_change = 'ignore' with dynamic insert overwrite where the SQL header would not be added
• In case, we would have static insert overwrite with on_schema_change != 'ignore' where the temp table would not be deleted

Checklist

• I have read the contributing guide and understand what's expected of me
• I have signed the CLA
• I have run this code in development and it appears to resolve the stated issue
• This PR includes tests, or tests are not required/relevant for this PR
• I have opened an issue to add/update docs, or docs changes are not required/relevant for this PR
• I have run changie new to create a changelog entry

[Fleid]

Lining this up for 1.6.0, sadly I don't think we'll have time to review it before 1.5.0 ships.

[github-christophe-oudar]

Align the parameter naming on the call site variable name

[github-christophe-oudar]

I think it should work as intended, should we move forward on that one for 1.6 release?

[Fleid]

I'm raising priority on our side, but there is still a ton of other tickets to review higher on the list :/
I'm sorry @github-christophe-oudar

[mikealfare]

@Kayrnt We just merged in a bunch of smaller dependabot type changes. Now that that's all done, do you mind updating your branch?

[Kayrnt]

I just rebased, let's see if it passes the tests

[colin-rogers-dbt]

Local testing successful and it looks like we're now using the temp table as expected.

[colin-rogers-dbt]

Last request: can we add a test fixture that covers this to test_incremental_strategies.py

Suggested fixture:

{% set partitions_to_replace = [
  "date_sub(current_date, interval 1 day)",
  "date_sub(current_date, interval 2 day)"
] %}

{{
    config(
        materialized="incremental",
        incremental_strategy="insert_overwrite",
        cluster_by="id",
        partition_by={
            "field": "date_time",
            "data_type": "datetime",
            "granularity": "day"
        },
        partitions=partitions_to_replace,
        on_schema_change="sync_all_columns"
    )
}}


with data as (

    {% if not is_incremental() %}

        select 1 as id, cast('2020-01-01' as datetime) as date_time union all
        select 2 as id, cast('2020-01-01' as datetime) as date_time union all
        select 3 as id, cast('2020-01-01' as datetime) as date_time union all
        select 4 as id, cast('2020-01-01' as datetime) as date_time

    {% else %}

        -- we want to overwrite the 4 records in the 2020-01-01 partition
        -- with the 2 records below, but add two more in the 2020-01-02 partition
        select 10 as id, cast('2020-01-01' as datetime) as date_time union all
        select 20 as id, cast('2020-01-01' as datetime) as date_time union all
        select 30 as id, cast('2020-01-02' as datetime) as date_time union all
        select 40 as id, cast('2020-01-02' as datetime) as date_time

    {% endif %}

)

select * from data

[Kayrnt]

@colin-rogers-dbt I added the test case, we should be fine 🚀

[dbeatty10]

@github-christophe-oudar Could you bring it up-to-date with the base branch?

We might need to keep asking you to do an update like that until this PR is merged 😅

Alternatively, if you allow edits from maintainers (recommended!), then we can handle that type of thing as-needed.

[Kayrnt]

You're merging too many things 😅
I tried to find the option but couldn't... I wonder if it's because I created the PR from another account than the one owning the fork

[colin-rogers-dbt]

if we can do one more update I can merge this

[cla-bot]

Thanks for your pull request, and welcome to our community! We require contributors to sign our Contributor License Agreement and we don't seem to have your signature on file. Check out this article for more information on why we have a CLA.

In order for us to review and merge your code, please submit the Individual Contributor License Agreement form attached above above. If you have questions about the CLA, or if you believe you've received this message in error, please reach out through a comment on this PR.

CLA has not been signed by users: @github-christophe-oudar

[github-christophe-oudar]

Done 👍

[McKnight-42]

@cla-bot check

[cla-bot]

Thanks for your pull request, and welcome to our community! We require contributors to sign our Contributor License Agreement and we don't seem to have your signature on file. Check out this article for more information on why we have a CLA.

In order for us to review and merge your code, please submit the Individual Contributor License Agreement form attached above above. If you have questions about the CLA, or if you believe you've received this message in error, please reach out through a comment on this PR.

CLA has not been signed by users: @github-christophe-oudar

[cla-bot]

The cla-bot has been summoned, and re-checked this pull request!

[Kayrnt]

@McKnight-42 @colin-rogers-dbt I rebased with my "cla:yes" account (my personal one) so that it should be OK.
I initially rebased earlier today because with my other account as I was logged in on it.