feat(warehouse): added duplicates stats for snowflake #3097
Conversation
achettyiitr
force-pushed
the
feat.snowflake-load-updates
branch
from
fc69486
to
debbee9
Compare
…t.snowflake-load-updates
| // Skip loading user tables if identifies table schema is not present | ||
| if identifiesSchema := job.GetTableSchemaInUpload(warehouseutils.IdentifiesTable); len(identifiesSchema) == 0 { | ||
| return []error{}, nil | ||
| } | ||
|
|
There was a problem hiding this comment.
Instead of keeping this check at the integration level, moving this out.
| WHEN MATCHED THEN | ||
| UPDATE SET original.received_at = original.received_at; |
There was a problem hiding this comment.
Adding a dummy update block. We are just updating a dummy column with its original values, not the staging values.
Codecov ReportPatch coverage:
Additional details and impacted files@@ Coverage Diff @@
## master #3097 +/- ##
==========================================
- Coverage 53.79% 53.42% -0.37%
==========================================
Files 348 348
Lines 53846 54182 +336
==========================================
- Hits 28965 28947 -18
- Misses 23240 23592 +352
- Partials 1641 1643 +2
... and 6 files with indirect coverage changes Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. ☔ View full report in Codecov by Sentry. |
| if dbHandle == nil { | ||
| dbHandle, err = Connect(sf.getConnectionCredentials(optionalCreds{schemaName: sf.Namespace})) | ||
| if err != nil { | ||
| sf.Logger.Errorf("SF: Error establishing connection for copying table:%s: %v\n", tableName, err) | ||
| return | ||
| } |
There was a problem hiding this comment.
Currently, dbHandle is always set as nil. Removing this condition.
Description
Notion Ticket
https://www.notion.so/rudderstacks/dedup-metrics-for-snowflake-2ff3ff988a9f4b01bd1815ecc70849e6?pvs=4
Security