Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[MAINTENANCE] filter RemovedInMarshmallow4 warnings #6602

Merged
merged 10 commits into from Dec 19, 2022
Merged

[MAINTENANCE] filter RemovedInMarshmallow4 warnings #6602

merged 10 commits into from Dec 19, 2022

Conversation

Kilo59
Copy link
Member

@Kilo59 Kilo59 commented Dec 19, 2022 •
edited

Resolves #6440, #6130

The deprecations discussed in the above issues are for a RemovedInMarshmallow4 warning.
We are unlikely to upgrade to marshmallow v4 for multiple reasons.

  1. Our new code is using pydantic for schema definitions and validation.
  2. The dependencies and other applications in our ecosystem also rely on marshmallow v3 so upgrading to v4 would also break compatibility with them.

If a package in our ecosystem does need to upgrade to marshmallow v4 we can opt to re-vendorize the marshmallow which would remove the v3 constraint for any dependency resolver.

Changes proposed in this pull request:

  • Filter out the RemovedInMarshmallow4 warnings from our codebase.
  • Update our pytest configuration to error if the above warning is encountered. This ensures we properly deal with these warnings.

Alternatives considered

  1. Branching logic based on the version of marshmallow.
    a. [MAINTENANCE] Update Marshmallow API Signature #6440 (comment)
    b. Initial deprecation_wrappers.py POC
  2. Always use the non-deprecated arguments and bump the minimum marshmallow version.
    a. [MAINTENANCE] Small clean-up of Marshmallow warnings (missing parameter changed to load_default as of 3.13) #6213
    b. [MAINTENANCE] Reverting marshmallow version bump #6271

@netlify
Copy link

netlify bot commented Dec 19, 2022 •
edited

✅ Deploy Preview for niobium-lead-7998 ready!

Name Link
🔨 Latest commit 6fe0755
🔍 Latest deploy log https://app.netlify.com/sites/niobium-lead-7998/deploys/63a097ff09ddcc0008f66a7e
😎 Deploy Preview https://deploy-preview-6602--niobium-lead-7998.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

@ghost
Copy link

ghost commented Dec 19, 2022 •
edited by ghost

👇 Click on the image for a new way to code review

  • Make big changes easier — review code in small groups of related files

  • Know where to start — see the whole change at a glance

  • Take a code tour — explore the change with an interactive tour

  • Make comments and review — all fully sync’ed with github

    Try it now!

Review these changes using an interactive CodeSee Map

Legend

CodeSee Map Legend

Kilo59
Kilo59 commented Dec 19, 2022
View reviewed changes
great_expectations/deprecation_wrappers.py Outdated
Comment on lines 41 to 54
def mm_field(
field_type: Type[marshmallow.fields.Field], load_default: Any = mm_missing, **kwargs
) -> marshmallow.fields.Field:
"""
Wrapper around `marshmallow.fields.String` that preserves backwards compatibility on older versions.
"""
_raise_for_deprecated_kwargs(DEPRECATED_FIELD_ARGS, kwargs)

if MARSHMALLOW_VERSION < MM_LOAD_DEFAULT_DEPRECATED_VERSION:
kwargs["missing"] = load_default
else:
kwargs["load_default"] = load_default
print(kwargs)
return field_type(**kwargs)
Copy link
Member Author

@Kilo59 Kilo59 Dec 19, 2022 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Decided that it's much simpler (and easier to maintain) to simply filter out the marshmallow4 deprecation warnings. We aren't moving to marshmallow v4 anytime soon (if ever).

Kilo59
Kilo59 commented Dec 19, 2022
View reviewed changes
great_expectations/data_context/types/base.py
@@ -976,7 +982,7 @@ class Meta:
module_name = fields.String(
required=False,
allow_none=True,
load_default="great_expectations.execution_engine",
missing="great_expectations.execution_engine",
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like we forgot to revert this use of missing as part of #6271

@Kilo59 Kilo59 marked this pull request as ready for review December 19, 2022 13:59
@Kilo59 Kilo59 requested review from anthonyburdi and a team December 19, 2022 14:00
Kilo59
Kilo59 commented Dec 19, 2022
View reviewed changes
pyproject.toml
Comment on lines +192 to +193
# We likely won't be updating to `marhsmallow` 4, these errors should be filtered out
"error::marshmallow.warnings.RemovedInMarshmallow4Warning"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line will cause the tests to error if this warning is encountered.

@Kilo59 Kilo59 changed the title [MAINTENANCE] filter RemovedInMarshmallow4Warning warnings [MAINTENANCE] filter RemovedInMarshmallow4 warnings Dec 19, 2022
@Kilo59 Kilo59 self-assigned this Dec 19, 2022
anthonyburdi
anthonyburdi approved these changes Dec 19, 2022
View reviewed changes
Copy link
Member

@anthonyburdi anthonyburdi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@NathanFarmer NathanFarmer mentioned this pull request Dec 19, 2022
Merged
6 tasks
Shinnnyshinshin
Shinnnyshinshin approved these changes Dec 19, 2022
View reviewed changes
Copy link
Contributor

@Shinnnyshinshin Shinnnyshinshin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for this @Kilo59 🙇🏼 I think the change + note is a elegant solution

@Kilo59 Kilo59 enabled auto-merge (squash) December 19, 2022 17:20
@Kilo59 Kilo59 merged commit f6897f6 into develop Dec 19, 2022
@Kilo59 Kilo59 deleted the m/none/deprecation-wrapper branch December 19, 2022 17:39
This was referenced Dec 19, 2022
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@anthonyburdi anthonyburdi anthonyburdi approved these changes

@Shinnnyshinshin Shinnnyshinshin Shinnnyshinshin approved these changes

Assignees

@Kilo59 Kilo59

Labels
core-team platform
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@Kilo59 @anthonyburdi @Shinnnyshinshin