New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
🐛 Destination snowflake: Handle namespace/stream name/columns containing ${
#30416
Conversation
Before Merging a Connector Pull RequestWow! What a great pull request you have here! 🎉 To merge this PR, ensure the following has been done/considered for each connector added or updated:
If the checklist is complete, but the CI check is failing,
|
${
${
|
If it's not difficult, yes I'd say to remove the final |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There probably should be a test for a column with a ${}
too
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
@evantahler I added some commits which are individually reviewable, PTAL |
destination-snowflake test report (commit
|
Step | Result |
---|---|
Java Connector Unit Tests | ✅ |
Build connector tar | ✅ |
Build destination-snowflake docker image for platform linux/x86_64 | ✅ |
Java Connector Integration Tests | ✅ |
Validate airbyte-integrations/connectors/destination-snowflake/metadata.yaml | ✅ |
Connector version semver check | ✅ |
Connector version increment check | ✅ |
QA checks | ✅ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=destination-snowflake test
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
✔️
closes #30391
The underlying problem is that
EXECUTE IMMEDIATE
throws an error if you have a string or identifier containing the${
bigram in your script block. Make two changes to support this:${
is now replaced by__
(double underscore) in table, schema, and column namesFor the original issue, this means we'll end up with a column
"__meta_ads}"
... do we want to also turn that final}
into an underscore?Notably: there's now a distinction between
escapeJsonIdentifier
andescapeSqlIdentifier
, because we still need to query the raw data as"_airbyte_data":"${foo}"
. This will be a merge conflict with @cynthiaxyin 's 9b1d1f5#diff-e6c24d3a3e554cd6197d2bc046dcf7b1926e25d01fdae9108b7a56fe306d8e91 :(