Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: break out default batch file writer into separate class #1668

Merged
merged 11 commits into from
May 5, 2023

Conversation

kgpayne
Copy link
Contributor

@kgpayne kgpayne commented May 4, 2023

@kgpayne kgpayne self-assigned this May 4, 2023
@kgpayne kgpayne marked this pull request as ready for review May 4, 2023 12:17
@codecov
Copy link

codecov bot commented May 4, 2023

Codecov Report

Merging #1668 (37d1262) into main (2974649) will increase coverage by 0.01%.
The diff coverage is 95.12%.

@@            Coverage Diff             @@
##             main    #1668      +/-   ##
==========================================
+ Coverage   85.05%   85.07%   +0.01%     
==========================================
  Files          58       59       +1     
  Lines        4845     4864      +19     
  Branches      805      807       +2     
==========================================
+ Hits         4121     4138      +17     
- Misses        526      527       +1     
- Partials      198      199       +1     
Impacted Files Coverage Δ
singer_sdk/helpers/_batch.py 88.78% <60.00%> (-1.42%) ⬇️
singer_sdk/batch.py 100.00% <100.00%> (ø)
singer_sdk/streams/core.py 83.01% <100.00%> (-0.66%) ⬇️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@kgpayne
Copy link
Contributor Author

kgpayne commented May 4, 2023

@edgarrmondragon this is to better reuse batch file writers on the target side. I.e. cases where the target supports batch files (in json or csv) but receives records. E.g. target-snowflake

@edgarrmondragon
Copy link
Collaborator

@edgarrmondragon this is to better reuse batch file writers on the target side. I.e. cases where the target supports batch files (in json or csv) but receives records. E.g. target-snowflake

@kgpayne This is looking great. Should we document these new batcher classes in the reference section (which is actually getting crowded imo)?

@kgpayne kgpayne requested review from a team as code owners May 5, 2023 17:48
@kgpayne kgpayne merged commit e029e30 into main May 5, 2023
@kgpayne kgpayne deleted the kgpayne/refactor-default-batch-writer branch May 5, 2023 19:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants