Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stricter validation for JSON-RPC responses #3359

Merged
merged 8 commits into from Apr 24, 2024

Conversation

fselmo
Copy link
Collaborator

@fselmo fselmo commented Apr 23, 2024

What was wrong?

Closes #3095 by more tightly validating the JSON-RPC responses coming from providers (not just the id as the issue states) while providing good messaging on errors.

This PR also includes the following changes:

  • Use Web3RPCError instead of a generic Web3ValueError for JSON-RPC errors returned by the provider.
  • Error log the user_message, debug log the response for RPC errors for better clarity / debugging.
  • In the spirit of wrapping up breaking changes for v7 and snake-casing some final remnants of camelCase args, capitalize "Of": BlockNumberOutofRange -> BlockNumberOutOfRange
  • Distinguish between MethodUnavailable (a JSON-RPC error from the node) and MethodNotSupported (web3.py does not support the method, internally, under some conditions).
  • Re-organize the formatted_response tests to be more readable. Separate test cases into tests where they make sense together: test valid responses, test invalid fields, test valid error responses, test invalid error object, test error formatters and null formatters together.

How was it fixed?

Todo:

  • Clean up commit history
  • Add or update documentation related to these changes (including the v7 migration guide)
  • Add entry to the release notes

Cute Animal Picture

20240422_182152

@fselmo fselmo force-pushed the stricter-validation-for-request-id branch 3 times, most recently from 950736a to 2c8e710 Compare April 23, 2024 21:36
fselmo added a commit to fselmo/web3.py that referenced this pull request Apr 23, 2024
@fselmo fselmo force-pushed the stricter-validation-for-request-id branch from 2c8e710 to d61baee Compare April 23, 2024 22:25
@fselmo fselmo marked this pull request as ready for review April 23, 2024 22:27
@fselmo fselmo changed the title [WIP] Stricter validation for JSON-RPC responses Stricter validation for JSON-RPC responses Apr 23, 2024
@fselmo fselmo force-pushed the stricter-validation-for-request-id branch from d61baee to dc44990 Compare April 24, 2024 14:35
web3/exceptions.py Outdated Show resolved Hide resolved
web3/manager.py Show resolved Hide resolved
@@ -295,7 +295,7 @@ class GethSyncingSubscriptionResponse(SubscriptionResponse):


class RPCResponse(TypedDict, total=False):
error: Union[RPCError, str]
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: This change is also breaking but this is how it should be defined. The only reason we allowed str is bc of how EthereumTesterProvider returned errors but that should not be allowed imo.

Copy link
Contributor

@pacrob pacrob left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

Copy link
Collaborator

@kclowes kclowes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like that BlockNumberOutOfRange change, nice! I left a comment around making sure we've tested against a range of providers to make sure it makes sense to validate the rpc response. (I have scarring from this chunk of code, lol) I also left a nit about importing merge from eth-utils instead of from toolz. But once those are addressed, this LGTM!

web3/manager.py Show resolved Hide resolved
@fselmo fselmo merged commit bd2ad50 into ethereum:main Apr 24, 2024
57 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Validate JSON-RPC response id field more strictly
4 participants