Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: snowflake appendmode #3745

Merged
merged 24 commits into from
Aug 25, 2023
Merged

feat: snowflake appendmode #3745

merged 24 commits into from
Aug 25, 2023

Conversation

fracasula
Copy link
Collaborator

@fracasula fracasula commented Aug 14, 2023

Description

Adding APPEND mode for Snowflake.

Changelog

  • DRYing MERGE INTO query
  • Adding new loadTableStrategy feature flag
  • Adding APPEND support to LoadTable with integration tests
  • Adding APPEND support to LoadUserTables with integration tests

Linear Ticket

< Linear Link >

Security

  • The code changed/added as part of this pull request won't create any security issues with how the software is being used.

@linear
Copy link

linear bot commented Aug 14, 2023

@codecov
Copy link

codecov bot commented Aug 14, 2023

Codecov Report

Patch coverage: 82.32% and project coverage change: +0.03% 🎉

Comparison is base (fd30711) 68.69% compared to head (2be8bfc) 68.73%.
Report is 5 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3745      +/-   ##
==========================================
+ Coverage   68.69%   68.73%   +0.03%     
==========================================
  Files         345      345              
  Lines       51652    51601      -51     
==========================================
- Hits        35484    35468      -16     
+ Misses      13881    13846      -35     
  Partials     2287     2287              
Files Changed Coverage Δ
warehouse/integrations/snowflake/snowflake.go 64.78% <79.19%> (-0.15%) ⬇️
warehouse/integrations/testhelper/verify.go 89.64% <90.15%> (-0.53%) ⬇️
warehouse/integrations/manager/manager.go 92.85% <100.00%> (ø)
warehouse/integrations/testhelper/setup.go 98.67% <100.00%> (ø)

... and 15 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Comment on lines 314 to 323
logFields := []interface{}{
lf.SourceID, sf.Warehouse.Source.ID,
lf.SourceType, sf.Warehouse.Source.SourceDefinition.Name,
lf.DestinationID, sf.Warehouse.Destination.ID,
lf.DestinationType, sf.Warehouse.Destination.DestinationDefinition.Name,
lf.WorkspaceID, sf.Warehouse.WorkspaceID,
lf.Namespace, sf.Namespace,
lf.TableName, tableName,
lf.LoadTableStrategy, sf.config.loadTableStrategy,
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❤️ It definitely cleaned up a lot of duplicate code.

Copy link
Member

@lvrach lvrach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved, with minor comments

warehouse/integrations/snowflake/snowflake.go Outdated Show resolved Hide resolved
warehouse/integrations/snowflake/snowflake.go Outdated Show resolved Hide resolved
warehouse/integrations/snowflake/snowflake.go Outdated Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants