Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: uploader can append #3805

Merged
merged 15 commits into from
Sep 14, 2023
Merged

chore: uploader can append #3805

merged 15 commits into from
Sep 14, 2023

Conversation

fracasula
Copy link
Collaborator

@fracasula fracasula commented Aug 29, 2023

Description

We can append if:

  • source is not an ETL source
  • source it's not a replay
  • source category is not in a specific set
  • we are in append mode

Additional changes

  • removed duplicated mocked uploader in favour of MockGen

TODOs

  • use Uploader.CanAppend() in the other integrations that support APPEND mode

Linear Ticket

< Linear Link >

Security

  • The code changed/added as part of this pull request won't create any security issues with how the software is being used.

@fracasula
Copy link
Collaborator Author

Blocked by rudderlabs/rudder-go-kit#108

@codecov
Copy link

codecov bot commented Aug 30, 2023

Codecov Report

Patch coverage: 63.35% and project coverage change: -0.04% ⚠️

Comparison is base (60749af) 68.91% compared to head (df6fb3b) 68.88%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3805      +/-   ##
==========================================
- Coverage   68.91%   68.88%   -0.04%     
==========================================
  Files         349      350       +1     
  Lines       52370    52499     +129     
==========================================
+ Hits        36092    36162      +70     
- Misses      13978    14032      +54     
- Partials     2300     2305       +5     
Files Changed Coverage Δ
warehouse/jobs/jobs.go 7.40% <0.00%> (-0.29%) ⬇️
warehouse/utils/utils.go 85.23% <ø> (ø)
warehouse/validations/validate.go 81.26% <15.78%> (+0.39%) ⬆️
warehouse/internal/mocks/utils/mock_uploader.go 65.57% <65.57%> (ø)
warehouse/integrations/deltalake/deltalake.go 74.49% <100.00%> (ø)
warehouse/integrations/snowflake/snowflake.go 60.45% <100.00%> (-0.11%) ⬇️
warehouse/upload.go 71.60% <100.00%> (+0.19%) ⬆️

... and 8 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@fracasula fracasula marked this pull request as ready for review August 30, 2023 15:32
@achettyiitr achettyiitr mentioned this pull request Aug 31, 2023
1 task
@fracasula
Copy link
Collaborator Author

@lvrach this one can potentially be merged if we temporarily remove the -race from the Makefile. I still need another approval though.

@fracasula
Copy link
Collaborator Author

@lvrach kind reminder that this is ready to review and passing.

@fracasula fracasula merged commit 56ab090 into master Sep 14, 2023
32 checks passed
@fracasula fracasula deleted the chore.pipe-104 branch September 14, 2023 16:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants