Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(warehouse): added additional datatypes for warehouse integrations #2971

Closed
wants to merge 13 commits into from

Conversation

achettyiitr
Copy link
Member

@achettyiitr achettyiitr commented Feb 14, 2023

Description

  • Additional datatypes for warehouse destination. These are collected from warehouse_rudder_missing_datatype stats generated during the fetching of the schema from the warehouse.
  • Some datatypes got missed as part of the alias supported by the warehouses.

Corresponding documentation links:

  1. BQ
  2. CLICKHOUSE
  3. DELTALAKE, ODBC
  4. MSSQL
  5. POSTGRES, bytea
  6. RS
  7. S3_DATALAKE
  8. SNOWFLAKE

Notion Ticket

https://www.notion.so/rudderstacks/Additional-datatypes-for-warehouse-Integrations-a765dda055db40589f0b2eedddce56ad?pvs=4

Security

  • The code changed/added as part of this pull request won't create any security issues with how the software is being used.

@codecov
Copy link

codecov bot commented Feb 14, 2023

Codecov Report

Patch coverage has no change and project coverage change: +0.08 🎉

Comparison is base (0d9af35) 53.51% compared to head (7890603) 53.59%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2971      +/-   ##
==========================================
+ Coverage   53.51%   53.59%   +0.08%     
==========================================
  Files         350      350              
  Lines       54452    54452              
==========================================
+ Hits        29139    29184      +45     
+ Misses      23649    23612      -37     
+ Partials     1664     1656       -8     
Impacted Files Coverage Δ
...ehouse/integrations/azure-synapse/azure-synapse.go 0.31% <ø> (ø)
warehouse/integrations/bigquery/bigquery.go 2.08% <ø> (ø)
warehouse/integrations/clickhouse/clickhouse.go 74.56% <ø> (ø)
...ns/datalake/schema-repository/schema_repository.go 77.27% <ø> (ø)
warehouse/integrations/deltalake/deltalake.go 65.17% <ø> (ø)
warehouse/integrations/mssql/mssql.go 1.62% <ø> (ø)
warehouse/integrations/postgres/postgres.go 0.53% <ø> (ø)
warehouse/integrations/redshift/redshift.go 10.34% <ø> (ø)
warehouse/integrations/snowflake/snowflake.go 1.02% <ø> (ø)

... and 5 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@achettyiitr achettyiitr marked this pull request as draft February 14, 2023 10:11
@achettyiitr achettyiitr force-pushed the feat.integration-additional-datatypes branch from c8f241e to b289470 Compare February 20, 2023 18:50
@achettyiitr
Copy link
Member Author

#2636

@achettyiitr
Copy link
Member Author

DeepSource issues are resolved in this #2964

@achettyiitr achettyiitr reopened this Mar 6, 2023
@achettyiitr achettyiitr marked this pull request as ready for review March 6, 2023 18:49
@github-actions
Copy link

github-actions bot commented Apr 6, 2023

This PR is considered to be stale. It has been open 20 days with no further activity thus it is going to be closed in 5 days. To avoid such a case please consider removing the stale label manually or add a comment to the PR.

@github-actions github-actions bot added the Stale label Apr 6, 2023
@github-actions github-actions bot closed this Apr 13, 2023
Comment on lines +60 to +96
"bigint": "int",
"int": "int",
"smallint": "int",
"tinyint": "int",
"integer": "int",

"numeric": "float",
"decimal": "float",
"money": "float",
"small money": "float",
"real": "float",
"float": "float",
"double precision": "float",

"char": "string",
"nchar": "string",
"varchar": "string",
"nvarchar": "string",
"text": "string",
"ntext": "string",
"binary": "string",
"image": "string",
"varbinary": "string",
"uniqueidentifier": "string",

"date": "datetime",
"datetimeoffset": "datetime",
"datetime2": "datetime",
"smalldatetime": "datetime",
"datetime": "datetime",
"time": "datetime",
"timestamp with time zone": "datetime",
"timestamp": "datetime",
"jsonb": "json",
"bit": "boolean",

"bit": "boolean",

"jsonb": "json",
Copy link
Member Author

@achettyiitr achettyiitr Apr 25, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These datatypes we have added:
int, money, small money, uniqueidentifier

@github-actions github-actions bot deleted the feat.integration-additional-datatypes branch May 17, 2023 02:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants