Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: redshift staging table creation #3180

Merged
merged 3 commits into from
Apr 12, 2023
Merged

Conversation

achettyiitr
Copy link
Member

@achettyiitr achettyiitr commented Apr 6, 2023

Description

While creating a staging table, it should be essentially a clone of the main table. Use CREATE TABLE staging (LIKE main) to create a staging table.
If we are issuing a CREATE TABLE IF NOT EXISTS statement, we are mapping strings with varchar(65535) while the main table can have varchar(512). COPY to staging table will succeed but insert into the main table will fail with the following error inserting into original table: pq: Value too long for character type

Notion Ticket

https://www.notion.so/rudderstacks/Redshift-create-temporary-table-with-main-table-fe7a74f2f9984034aadd577f77ab86d6?pvs=4

Security

  • The code changed/added as part of this pull request won't create any security issues with how the software is being used.

@codecov
Copy link

codecov bot commented Apr 6, 2023

Codecov Report

Patch coverage: 75.55% and project coverage change: +0.17 🎉

Comparison is base (7d82354) 53.26% compared to head (00d822d) 53.43%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3180      +/-   ##
==========================================
+ Coverage   53.26%   53.43%   +0.17%     
==========================================
  Files         315      315              
  Lines       51757    51782      +25     
==========================================
+ Hits        27569    27672     +103     
+ Misses      22519    22443      -76     
+ Partials     1669     1667       -2     
Impacted Files Coverage Δ
warehouse/integrations/redshift/redshift.go 10.30% <0.00%> (-0.05%) ⬇️
services/streammanager/kafka/kafkamanager.go 76.81% <82.14%> (+0.06%) ⬆️
services/streammanager/kafka/client/producer.go 84.03% <100.00%> (+4.21%) ⬆️

... and 8 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@achettyiitr achettyiitr merged commit 8ec737d into master Apr 12, 2023
16 checks passed
@achettyiitr achettyiitr deleted the fix.redshift-create-table branch April 12, 2023 07:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants