New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add plugin to infer more precise regex match types #7803
Open
Michael0x2a
wants to merge
8
commits into
python:master
Choose a base branch
from
Michael0x2a:add-regex-plugin
base: master
Could not load branches
Branch not found: {{ refName }}
Could not load tags
Nothing to show
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Commits on Oct 27, 2019
-
Add plugin to infer more precise regex match types
This pull request adds a plugin to make mypy infer more precise types when grabbing regex groups: the plugin will when possible analyze original regex to deduce whether a given group is required or not. ``` from typing_extensions import Final, Literal import re pattern: Final = re.compile("(a)(b)*") match: Final = pattern.match("") if match: reveal_type(match.groups()) # Revealed type is Tuple[str, Optional[str]] reveal_type(match.group(0)) # Revealed type is str reveal_type(match.group(1)) # Revealed type is str reveal_type(match.group(2)) # Revealed type is Optional[str] index: int reveal_type(match.group(index)) # Revealed type is Optional[str] # Error: Regex has 3 total groups, given group number 5 is too big match.group(5) ``` To track this information, I added in an optional 'metadata' dict field to the Instance class, similar to the metadata dict for plugins in TypeInfos. We skip serializing this dict if it does not contain any data. A limitation of this plugin is that both the pattern and the match variables must be declared to be final. Otherwise, we just default to using whatever types are defined in typeshed. This is because we set and erase the metadata field in exactly the same way we set and erase the `last_known_value` field in Instances: both kinds of info are "transient" and are unsafe to keep around if the variable reference is mutable. This limitation *does* end up limiting the usefulness of this plugin to some degree: it won't support common patterns like the below, since variables aren't allowed to be declared final inside loops: ``` for line in file: match = pattern.match(line) if match: ... ``` Possibly we can remove this limitation by making mypy less aggressive about removing this transient info by tracking the "lifetime" of this sort of data in some way? This pull request should mostly address python#7363, though it's unclear if it really fully resolves it: we might want to do something about the limitation described above and re-tune typeshed first. The other mostly unrelated change this PR makes is to refactor some of the helper functions in checker.py into typeops.py so I could use them more cleanly in the plugin.
Configuration menu - View commit details
-
Copy full SHA for 54dfad8 - Browse repository at this point
Copy the full SHA 54dfad8View commit details -
Configuration menu - View commit details
-
Copy full SHA for 9a7097f - Browse repository at this point
Copy the full SHA 9a7097fView commit details
Commits on Jan 5, 2022
-
Configuration menu - View commit details
-
Copy full SHA for 03bd647 - Browse repository at this point
Copy the full SHA 03bd647View commit details -
Configuration menu - View commit details
-
Copy full SHA for 45168cd - Browse repository at this point
Copy the full SHA 45168cdView commit details -
Configuration menu - View commit details
-
Copy full SHA for 9d468df - Browse repository at this point
Copy the full SHA 9d468dfView commit details -
Configuration menu - View commit details
-
Copy full SHA for ced8cda - Browse repository at this point
Copy the full SHA ced8cdaView commit details -
Configuration menu - View commit details
-
Copy full SHA for f54eff2 - Browse repository at this point
Copy the full SHA f54eff2View commit details -
Configuration menu - View commit details
-
Copy full SHA for 3b38dc9 - Browse repository at this point
Copy the full SHA 3b38dc9View commit details
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.