Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: snowflake appendmode #3745

Merged
merged 24 commits into from
Aug 25, 2023
Merged

feat: snowflake appendmode #3745

Show file tree
Hide file tree
Changes from 22 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
5da62e0
chore: dry merge query
fracasula Aug 14, 2023
3c66e34
chore: loadTableStrategy feature flag
fracasula Aug 14, 2023
d236cb1
chore: DRYing log entries
fracasula Aug 14, 2023
634c7a0
chore: loadTable append mode
fracasula Aug 14, 2023
02b9dcf
chore: imports
fracasula Aug 14, 2023
570958c
chore: renaming imports
fracasula Aug 14, 2023
77201f0
chore: DRYing test
fracasula Aug 14, 2023
88d6b74
chore: decreasing interval
fracasula Aug 14, 2023
76529a4
chore: - COALESCE
fracasula Aug 16, 2023
6c10066
chore: cleaning up verify
fracasula Aug 16, 2023
f01e916
chore: typo in env var
fracasula Aug 16, 2023
e7c6894
chore: snowflake append mode test
fracasula Aug 16, 2023
e7cfb58
chore: adding table
fracasula Aug 16, 2023
5dd3462
chore: cleaning up LoadIdentityMergeRulesTable
fracasula Aug 16, 2023
872ed03
chore: cleaning up LoadIdentityMappingsTable
fracasula Aug 16, 2023
aeaa27d
chore: cleaning up LoadUserTables
fracasula Aug 16, 2023
91dec4e
chore: test for LoadUserTables append
fracasula Aug 17, 2023
96efea4
chore: using insert with overwrite
fracasula Aug 17, 2023
feb979c
chore: snowflake constructor error
fracasula Aug 17, 2023
d650b02
chore: return correct table
fracasula Aug 17, 2023
06d73b1
chore: merging traits for users
fracasula Aug 17, 2023
c511406
chore: finalizing append mode
fracasula Aug 21, 2023
2be8bfc
chore: using *testing.T from correct scope
fracasula Aug 23, 2023
d04e07f
chore: sorting out logs
fracasula Aug 25, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 8 additions & 11 deletions warehouse/integrations/manager/manager.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,12 @@ import (
"fmt"
"time"

"github.com/rudderlabs/rudder-go-kit/config"
"github.com/rudderlabs/rudder-go-kit/logger"
"github.com/rudderlabs/rudder-go-kit/stats"

"github.com/rudderlabs/rudder-server/warehouse/internal/model"

"github.com/rudderlabs/rudder-server/warehouse/integrations/azure-synapse"
"github.com/rudderlabs/rudder-server/utils/misc"
"github.com/rudderlabs/rudder-server/warehouse/client"
azuresynapse "github.com/rudderlabs/rudder-server/warehouse/integrations/azure-synapse"
"github.com/rudderlabs/rudder-server/warehouse/integrations/bigquery"
"github.com/rudderlabs/rudder-server/warehouse/integrations/clickhouse"
"github.com/rudderlabs/rudder-server/warehouse/integrations/datalake"
Expand All @@ -19,11 +19,8 @@ import (
"github.com/rudderlabs/rudder-server/warehouse/integrations/postgres"
"github.com/rudderlabs/rudder-server/warehouse/integrations/redshift"
"github.com/rudderlabs/rudder-server/warehouse/integrations/snowflake"

"github.com/rudderlabs/rudder-go-kit/config"
"github.com/rudderlabs/rudder-server/utils/misc"
"github.com/rudderlabs/rudder-server/warehouse/client"
"github.com/rudderlabs/rudder-server/warehouse/utils"
"github.com/rudderlabs/rudder-server/warehouse/internal/model"
warehouseutils "github.com/rudderlabs/rudder-server/warehouse/utils"
)

type Manager interface {
Expand Down Expand Up @@ -67,7 +64,7 @@ func New(destType string, conf *config.Config, logger logger.Logger, stats stats
case warehouseutils.BQ:
return bigquery.New(conf, logger), nil
case warehouseutils.SNOWFLAKE:
return snowflake.New(conf, logger, stats), nil
return snowflake.New(conf, logger, stats)
case warehouseutils.POSTGRES:
return postgres.New(conf, logger, stats), nil
case warehouseutils.CLICKHOUSE:
Expand All @@ -92,7 +89,7 @@ func NewWarehouseOperations(destType string, conf *config.Config, logger logger.
case warehouseutils.BQ:
return bigquery.New(conf, logger), nil
case warehouseutils.SNOWFLAKE:
return snowflake.New(conf, logger, stats), nil
return snowflake.New(conf, logger, stats)
case warehouseutils.POSTGRES:
return postgres.New(conf, logger, stats), nil
case warehouseutils.CLICKHOUSE:
Expand Down
Loading
Loading