Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(taps): Implement reference paginators #732

Merged
merged 16 commits into from
Sep 1, 2022

Conversation

edgarrmondragon
Copy link
Collaborator

@edgarrmondragon edgarrmondragon commented Jun 17, 2022

@edgarrmondragon edgarrmondragon self-assigned this Jun 17, 2022
@edgarrmondragon edgarrmondragon force-pushed the 316-add-reference-paginator-implementations branch from b010229 to 5a184a4 Compare June 17, 2022 16:15
@edgarrmondragon edgarrmondragon force-pushed the 316-add-reference-paginator-implementations branch from 3180eea to 9952838 Compare June 17, 2022 17:22
@edgarrmondragon edgarrmondragon marked this pull request as ready for review June 17, 2022 17:26
@edgarrmondragon edgarrmondragon force-pushed the 316-add-reference-paginator-implementations branch from 9952838 to 029410d Compare June 17, 2022 17:36
Copy link
Contributor

@aaronsteers aaronsteers left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@edgarrmondragon - This is so cool!

My comments are only cosmetic, more about getting this ready for a prod release.

I think it's worth discussing if we should warn (as you have it) or just fail if they are using the legacy approach. And if we fail, would be nice to do so in a unit test (nested in get_standard_tests or similar) rather than at runtime. 😄

docs/porting.md Outdated Show resolved Hide resolved
singer_sdk/streams/rest.py Show resolved Hide resolved
@codecov-commenter
Copy link

codecov-commenter commented Jun 17, 2022

Codecov Report

Merging #732 (0b6b143) into main (e84036b) will increase coverage by 0.58%.
The diff coverage is 98.01%.

@@            Coverage Diff             @@
##             main     #732      +/-   ##
==========================================
+ Coverage   80.68%   81.27%   +0.58%     
==========================================
  Files          35       36       +1     
  Lines        3480     3520      +40     
  Branches      692      706      +14     
==========================================
+ Hits         2808     2861      +53     
+ Misses        499      486      -13     
  Partials      173      173              
Impacted Files Coverage Δ
singer_sdk/streams/rest.py 82.58% <83.33%> (-0.03%) ⬇️
singer_sdk/pagination.py 100.00% <100.00%> (ø)
singer_sdk/mapper_base.py 55.55% <0.00%> (-2.03%) ⬇️
singer_sdk/mapper.py 84.23% <0.00%> (-0.35%) ⬇️
singer_sdk/sinks/core.py 78.74% <0.00%> (-0.05%) ⬇️
singer_sdk/streams/core.py 81.37% <0.00%> (+0.11%) ⬆️
singer_sdk/sinks/record.py 76.92% <0.00%> (+1.92%) ⬆️
singer_sdk/io_base.py 77.77% <0.00%> (+2.02%) ⬆️
singer_sdk/sinks/batch.py 100.00% <0.00%> (+5.00%) ⬆️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

Copy link
Contributor

@aaronsteers aaronsteers left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work, @edgarrmondragon !

Just one small question (nit).

singer_sdk/streams/rest.py Outdated Show resolved Hide resolved
tests/core/rest/test_pagination.py Show resolved Hide resolved
@edgarrmondragon edgarrmondragon changed the title feat: Implement reference paginators feat(taps): Implement reference paginators Jul 15, 2022
@edgarrmondragon edgarrmondragon requested review from a team as code owners July 22, 2022 21:12
@edgarrmondragon edgarrmondragon force-pushed the 316-add-reference-paginator-implementations branch from 28a98fb to 8044212 Compare July 22, 2022 21:13
@edgarrmondragon
Copy link
Collaborator Author

I think it's worth discussing if we should warn (as you have it) or just fail if they are using the legacy approach. And if we fail, would be nice to do so in a unit test (nested in get_standard_tests or similar) rather than at runtime. 😄

@aaronsteers I think we could warn that it will fail in the future. Wdyt?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add reference paginator implementations
3 participants