-
Notifications
You must be signed in to change notification settings - Fork 311
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(warehouse): added duplicates stats for snowflake #3097
Conversation
fc69486
to
debbee9
Compare
…t.snowflake-load-updates
// Skip loading user tables if identifies table schema is not present | ||
if identifiesSchema := job.GetTableSchemaInUpload(warehouseutils.IdentifiesTable); len(identifiesSchema) == 0 { | ||
return []error{}, nil | ||
} | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Instead of keeping this check at the integration level, moving this out.
WHEN MATCHED THEN | ||
UPDATE SET original.received_at = original.received_at; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Adding a dummy update block. We are just updating a dummy column with its original values, not the staging values.
Codecov ReportPatch coverage:
Additional details and impacted files@@ Coverage Diff @@
## master #3097 +/- ##
==========================================
- Coverage 53.79% 53.42% -0.37%
==========================================
Files 348 348
Lines 53846 54182 +336
==========================================
- Hits 28965 28947 -18
- Misses 23240 23592 +352
- Partials 1641 1643 +2
... and 6 files with indirect coverage changes Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. ☔ View full report in Codecov by Sentry. |
if dbHandle == nil { | ||
dbHandle, err = Connect(sf.getConnectionCredentials(optionalCreds{schemaName: sf.Namespace})) | ||
if err != nil { | ||
sf.Logger.Errorf("SF: Error establishing connection for copying table:%s: %v\n", tableName, err) | ||
return | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Currently, dbHandle is always set as nil. Removing this condition.
Description
Notion Ticket
https://www.notion.so/rudderstacks/dedup-metrics-for-snowflake-2ff3ff988a9f4b01bd1815ecc70849e6?pvs=4
Security