Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Sink schema comparison before adding metadata columns #1778

Merged
merged 6 commits into from
Jun 20, 2023

Conversation

pnadolny13
Copy link
Contributor

@pnadolny13 pnadolny13 commented Jun 20, 2023

Closes #1777

  • moves test mock classes from the csv test to conftest so its reusable. Let me know if this should go somewhere else.
  • adds test for target_base get_sink method to assert that when add_metadata_columns is used it doesnt affect the comparison. The tests fail in CI at first, then pass once I added a fix
  • I chose to fix it by adding an original_schema attribute to the sink object. The alternative, and what I did in target-snowflake originally, is to add logic to remove those columns before doing the comparison but that sounded like more work.

@edgarrmondragon let me know what you think!


📚 Documentation preview 📚: https://meltano-sdk--1778.org.readthedocs.build/en/1778/

@pnadolny13 pnadolny13 marked this pull request as draft June 20, 2023 19:45
@pnadolny13 pnadolny13 marked this pull request as ready for review June 20, 2023 19:59
@codecov
Copy link

codecov bot commented Jun 20, 2023

Codecov Report

Merging #1778 (bfeb482) into main (88d7586) will increase coverage by 0.18%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main    #1778      +/-   ##
==========================================
+ Coverage   85.93%   86.12%   +0.18%     
==========================================
  Files          59       59              
  Lines        4942     4944       +2     
  Branches      809      809              
==========================================
+ Hits         4247     4258      +11     
+ Misses        492      485       -7     
+ Partials      203      201       -2     
Impacted Files Coverage Δ
singer_sdk/target_base.py 91.12% <ø> (+0.93%) ⬆️
singer_sdk/sinks/core.py 84.41% <100.00%> (+4.81%) ⬆️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

Copy link
Collaborator

@edgarrmondragon edgarrmondragon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great @pnadolny13 😁

@edgarrmondragon edgarrmondragon changed the title fix: sink schema compare fix: Sink schema comparison before adding metadata columns Jun 20, 2023
@pnadolny13 pnadolny13 merged commit fd807e6 into main Jun 20, 2023
27 checks passed
@pnadolny13 pnadolny13 deleted the fix_sink_schema_compare branch June 20, 2023 21:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

bug: target existing sink check returns incorrect boolean when add_metadata_columns=True
2 participants