fix: Allow Singer schemas to include the required and enum fields
#917
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Jack-Burnett
requested review from
edgarrmondragon,
cjohnhanson and
aaronsteers
as code owners
Codecov Report
@@ Coverage Diff @@
## main #917 +/- ##
==========================================
+ Coverage 80.40% 80.66% +0.26%
==========================================
Files 34 35 +1
Lines 3429 3476 +47
Branches 681 692 +11
==========================================
+ Hits 2757 2804 +47
Misses 499 499
Partials 173 173
📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
|
Hi @Jack-Burnett! Thanks for the PR. I'll take a look in the next few days 😺 |
edgarrmondragon
changed the title
edgarrmondragon
requested changes
Aug 24, 2022
edgarrmondragon
requested changes
Aug 24, 2022
There was a problem hiding this comment.
@Jack-Burnett this is looking great!
I initially missed those pylint comments which we don't need. Once that's addressed this is g2g 🙂
Co-authored-by: Edgar R. M. <edgarrm358@gmail.com>
Co-authored-by: Edgar R. M. <edgarrm358@gmail.com>
Co-authored-by: Edgar R. M. <edgarrm358@gmail.com>
Co-authored-by: Edgar R. M. <edgarrm358@gmail.com>
edgarrmondragon
changed the title
edgarrmondragon
changed the title
required and enum fields
Ohh yeah just lazy copy-pasting on my part 😅 All gone now |
edgarrmondragon
approved these changes
Aug 24, 2022
|
Thank you @Jack-Burnett! |
This was
linked to
issues
Aug 24, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #901
The use of the Schema class from a dependency was limiting schemas to only include certain declared json schema fields, since it would remove properties if they were not whitelisted. Since this is from a dependency, we can't simply expand the whitelist so instead we override the class and expand the list there. An alternative implementation would be to simply replace the class outright (no inheritance) - this could break existing taps that pass a Schema to Stream.init though.
It helps with but does not close #332, since 332 is asking for helpers, which this does not add - it makes it possible to add them though.
I think there are lots more improvements that could be made to SchemaPlus in follow-up PRs (adding many more legitimate properties to the whitelist and having better support for some non-simple properties.
Also, I removed the 'selected' and 'inclusion' fields from the new class - because they are part of the singer spec and not actually valid fields to see on a json schema.