update of macro for postgres/redshift use of unique_key as a list

[McKnight-42]

#4738 update of macro macro get_delete_insert_merge_sql to check if unique key being passed is a list or string

Description

updates

{% if unique_key is not none %}
    delete from {{ target }}
    where ({{ unique_key }}) in (
        select ({{ unique_key }})
        from {{ source }}
    );
    {% endif %}

to

 {% if unique_key is not none and unique_key != [] %}
        {% if unique_key is sequence and unique_key is not mapping and unique_key is not string %}
            delete from {{target }}
            using {{ source }}
            where (
                {% for key in unique_key %}
                    {{ source }}.{{ key }} = {{ target }}.{{ key }}
                    {{ "and " if not loop.last }}
                {% endfor %}
            );
        {% else %}
            delete from {{ target }}
            where (
                {{ unique_key }}) in (
                select ({{ unique_key }})
                from {{ source }}
            );

        {% endif %}
        {% endif %}

To pair with changes made in #4618 to extend capabilities to postgres and redshift adapters

• co-author @jtcohen6

Checklist

• I have signed the CLA
• I have run this code in development and it appears to resolve the stated issue
• This PR includes tests, or tests are not required/relevant for this PR
• I have added information about my change to be included in the CHANGELOG.

[github-actions]

Thank you for your pull request! We could not find a changelog entry for this change. For details on how to document a change, see the contributing guide.

[jtcohen6]

@McKnight-42 and I had a chance to pair this afternoon, and made good progress!

• Our "standard" tests (copied from plugins) are passing on Postgres here
• The exact same tests pass against Redshift in local testing
• And against Snowflake, using the delete+insert incremental strategy

Next steps look like:

• Rewriting these tests to use the new pytest framework
• Make them truly cross-database compatible (I think by turning expected models into seeds)
• Add them to the inheritable adapter tests, and turn them on for Redshift + Snowflake + BigQuery + Spark. Delete the copy-pasted versions present in those repos :)

I'd really like to include this change ahead of cutting v1.1.0-b1 on Thursday, and I don't think those next steps need to block merging this PR. If we do merge with next steps still open, we should open issues for the remaining work.

[jtcohen6]

I was worried that this may be a breaking change for any users who have defined a unique key named "False". Putting aside how confusing that would be — we tested it out, and this logic should still work.

[nathaniel-may]

also an empty string

[jtcohen6]

This logic needs to be database-agnostic. The dbt-snowflake tests expect a highly specific error message, which is specific to Snowflake. The test xfails correctly on other databases, but then those databases use their own words to tell us that the column is missing. No good.

The move here is to:

• Only check for the missing column name (thisisnotacolumn) in the error message, since any database worth its salt should include the column name in its error message
• Lowercase the whole error message, since some databases (Snowflake) will return the column name in uppercase

Turns out, this is also what we need to get the tests passing on Snowflake with delete+insert incremental strategy, since the error message is slightly different (different column alias)

[McKnight-42]

I have opened up a new issue to continue the conversation here #4926

[jtcohen6]

Thanks @McKnight-42! I should been clearer in my statement above: The logic in this PR is database-agnostic. It isn't in some of the other plugins (since we copy-pasted-edited the test cases). We'll want to sort that out when we go to solve #4882

[emmyoop]

One small question on your changes to the macro.

But I think the bigger thing is the new tests you added should be added in the new pattern of testing instead of the old integration tests pattern.

[emmyoop]

@McKnight-42 I just saw @jtcohen6's comment on converting the tests not being a blocker to getting this into the beta. That's fine but can you create an issue now to resolve those last steps so it's not lost?

Next steps look like:

• Rewriting these tests to use the new pytest framework
• Make them truly cross-database compatible (I think by turning expected models into seeds)
• Add them to the inheritable adapter tests, and turn them on for Redshift + Snowflake + BigQuery + Spark. Delete the copy-pasted versions present in those repos :)

[McKnight-42]

@emmyoop new issue created #4882

[VersusFacit]

Minor suggestions for readability but the logic looks solid.

[emmyoop]

Looks good now!

[iknox-fa]

This LGTM with the exception of the cross-database concerns @jtcohen6 pointed out.

I wouldn't be opposed to merge as is for now since we still only run integration tests with postgres as long as we have a follow-on ticket to make it agnostic.

The feedback has been addressed so unblocking this PR

[gshank]

Looks good!