Fix bug with mypy plugin and `no_strict_optional = True` #8666

dmontagu · 2024-01-29T18:00:53Z

The issue was originally introduced in #7411 (which, if you look at the body, I noted that I was concerned that there was a deeper issue not being addressed). I still have those concerns, but at the very least I think with the changes in this PR, the _DeferAnalysis exception should always get handled, even if it does something that causes the mypy plugin to otherwise misbehave. (That exception was the source of the issue reported in #8665.)

There were some code paths that could raise a _DeferAnalysis exception but which were not included in the try: except:.

I was able to reproduce the error in #8665 with the code sample provided, and this change fixed it. I also checked and I am pretty confident that there are no code paths that can raise that error now that aren't covered by the updated try: except:.

cloudflare-pages · 2024-01-29T18:01:01Z

Deploying with Cloudflare Pages

Latest commit:	`b8b500b`
Status:	✅ Deploy successful!
Preview URL:	https://7a5ed98e.pydantic-docs2.pages.dev
Branch Preview URL:	https://dmontagu-fix-mypy-defer-anal.pydantic-docs2.pages.dev

View logs

codspeed-hq · 2024-01-29T18:04:02Z

CodSpeed Performance Report

Merging #8666 will not alter performance

_{Comparing dmontagu/fix-mypy-defer-analysis-bug (b8b500b) with main (5fc166c)}

Summary

✅ 10 untouched benchmarks

sydney-runkle

Thanks for the fix, @dmontagu!

Revert this when pydantic/pydantic#8666 is released. Likely in Pydantic 2.6.1

This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [pydantic](https://github.com/pydantic/pydantic) ([changelog](https://docs.pydantic.dev/latest/changelog/)) | dependencies | minor | `2.5.3` -> `2.6.1` | --- ### Release Notes <details> <summary>pydantic/pydantic (pydantic)</summary> ### [`v2.6.1`](https://github.com/pydantic/pydantic/blob/HEAD/HISTORY.md#v261-2024-02-05) [Compare Source](pydantic/pydantic@v2.6.0...v2.6.1) [GitHub release](https://github.com/pydantic/pydantic/releases/tag/v2.6.1) ##### What's Changed ##### Packaging - Upgrade to `pydantic-core` 2.16.2 by [@sydney-runkle](https://github.com/sydney-runkle) in [#8717](pydantic/pydantic#8717) ##### Fixes - Fix bug with `mypy` plugin and `no_strict_optional = True` by [@dmontagu](https://github.com/dmontagu) in [#8666](pydantic/pydantic#8666) - Fix `ByteSize` error `type` change by [@sydney-runkle](https://github.com/sydney-runkle) in [#8681](pydantic/pydantic#8681) - Fix inheriting `Field` annotations in dataclasses by [@sydney-runkle](https://github.com/sydney-runkle) in [#8679](pydantic/pydantic#8679) - Fix regression in core schema generation for indirect definition references by [@dmontagu](https://github.com/dmontagu) in [#8702](pydantic/pydantic#8702) - Fix unsupported types bug with `PlainValidator` by [@sydney-runkle](https://github.com/sydney-runkle) in [#8710](pydantic/pydantic#8710) - Reverting problematic fix from 2.6 release, fixing schema building bug by [@sydney-runkle](https://github.com/sydney-runkle) in [#8718](pydantic/pydantic#8718) - Fix warning for tuple of wrong size in `Union` by [@davidhewitt](https://github.com/davidhewitt) in [pydantic/pydantic-core#1174](pydantic/pydantic-core#1174) - Fix `computed_field` JSON serializer `exclude_none` behavior by [@sydney-runkle](https://github.com/sydney-runkle) in [pydantic/pydantic-core#1187](pydantic/pydantic-core#1187) ### [`v2.6.0`](https://github.com/pydantic/pydantic/blob/HEAD/HISTORY.md#v260-2024-01-23) [Compare Source](pydantic/pydantic@v2.5.3...v2.6.0) [GitHub release](https://github.com/pydantic/pydantic/releases/tag/v2.6.0) The code released in v2.6.0 is practically identical to that of v2.6.0b1. ##### What's Changed ##### Packaging - Check for `email-validator` version >= 2.0 by [@commonism](https://github.com/commonism) in [#6033](pydantic/pydantic#6033) - Upgrade \`ruff\`\` target version to Python 3.8 by [@Elkiwa](https://github.com/Elkiwa) in [#8341](pydantic/pydantic#8341) - Update to `pydantic-extra-types==2.4.1` by [@yezz123](https://github.com/yezz123) in [#8478](pydantic/pydantic#8478) - Update to `pyright==1.1.345` by [@Viicos](https://github.com/Viicos) in [#8453](pydantic/pydantic#8453) - Update pydantic-core from 2.14.6 to 2.16.1, significant changes from these updates are described below, full changelog [here](pydantic/pydantic-core@v2.14.6...v2.16.1) ##### New Features - Add `NatsDsn` by [@ekeew](https://github.com/ekeew) in [#6874](pydantic/pydantic#6874) - Add `ConfigDict.ser_json_inf_nan` by [@davidhewitt](https://github.com/davidhewitt) in [#8159](pydantic/pydantic#8159) - Add `types.OnErrorOmit` by [@adriangb](https://github.com/adriangb) in [#8222](pydantic/pydantic#8222) - Support `AliasGenerator` usage by [@sydney-runkle](https://github.com/sydney-runkle) in [#8282](pydantic/pydantic#8282) - Add Pydantic People Page to docs by [@sydney-runkle](https://github.com/sydney-runkle) in [#8345](pydantic/pydantic#8345) - Support `yyyy-MM-DD` datetime parsing by [@sydney-runkle](https://github.com/sydney-runkle) in [#8404](pydantic/pydantic#8404) - Added bits conversions to the `ByteSize` class [#8415](pydantic/pydantic#8415) by [@luca-matei](https://github.com/luca-matei) in [#8507](pydantic/pydantic#8507) - Enable json schema creation with type `ByteSize` by [@geospackle](https://github.com/geospackle) in [#8537](pydantic/pydantic#8537) - Add `eval_type_backport` to handle union operator and builtin generic subscripting in older Pythons by [@alexmojaki](https://github.com/alexmojaki) in [#8209](pydantic/pydantic#8209) - Add support for `dataclass` fields `init` by [@dmontagu](https://github.com/dmontagu) in [#8552](pydantic/pydantic#8552) - Implement pickling for `ValidationError` by [@davidhewitt](https://github.com/davidhewitt) in [pydantic/pydantic-core#1119](pydantic/pydantic-core#1119) - Add unified tuple validator that can handle "variadic" tuples via PEP-646 by [@dmontagu](https://github.com/dmontagu) in [pydantic/pydantic-core#865](pydantic/pydantic-core#865) ##### Changes - Drop Python3.7 support by [@hramezani](https://github.com/hramezani) in [#7188](pydantic/pydantic#7188) - Drop Python 3.7, and PyPy 3.7 and 3.8 by [@davidhewitt](https://github.com/davidhewitt) in [pydantic/pydantic-core#1129](pydantic/pydantic-core#1129) - Use positional-only `self` in `BaseModel` constructor, so no field name can ever conflict with it by [@ariebovenberg](https://github.com/ariebovenberg) in [#8072](pydantic/pydantic#8072) - Make `@validate_call` return a function instead of a custom descriptor - fixes binding issue with inheritance and adds `self/cls` argument to validation errors by [@alexmojaki](https://github.com/alexmojaki) in [#8268](pydantic/pydantic#8268) - Exclude `BaseModel` docstring from JSON schema description by [@sydney-runkle](https://github.com/sydney-runkle) in [#8352](pydantic/pydantic#8352) - Introducing `classproperty` decorator for `model_computed_fields` by [@Jocelyn-Gas](https://github.com/Jocelyn-Gas) in [#8437](pydantic/pydantic#8437) - Explicitly raise an error if field names clashes with types by [@Viicos](https://github.com/Viicos) in [#8243](pydantic/pydantic#8243) - Use stricter serializer for unions of simple types by [@alexdrydew](https://github.com/alexdrydew) [pydantic/pydantic-core#1132](pydantic/pydantic-core#1132) ##### Performance - Add Codspeed profiling Actions workflow by [@lambertsbennett](https://github.com/lambertsbennett) in [#8054](pydantic/pydantic#8054) - Improve `int` extraction by [@samuelcolvin](https://github.com/samuelcolvin) in [pydantic/pydantic-core#1155](pydantic/pydantic-core#1155) - Improve performance of recursion guard by [@samuelcolvin](https://github.com/samuelcolvin) in [pydantic/pydantic-core#1156](pydantic/pydantic-core#1156) - `dataclass` serialization speedups by [@samuelcolvin](https://github.com/samuelcolvin) in [pydantic/pydantic-core#1162](pydantic/pydantic-core#1162) - Avoid `HashMap` creation when looking up small JSON objects in `LazyIndexMaps` by [@samuelcolvin](https://github.com/samuelcolvin) in [pydantic/jiter#55](pydantic/jiter#55) - use hashbrown to speedup python string caching by [@davidhewitt](https://github.com/davidhewitt) in [pydantic/jiter#51](pydantic/jiter#51) - Replace `Peak` with more efficient `Peek` by [@davidhewitt](https://github.com/davidhewitt) in [pydantic/jiter#48](pydantic/jiter#48) ##### Fixes - Move `getattr` warning in deprecated `BaseConfig` by [@tlambert03](https://github.com/tlambert03) in [#7183](pydantic/pydantic#7183) - Only hash `model_fields`, not whole `__dict__` by [@alexmojaki](https://github.com/alexmojaki) in [#7786](pydantic/pydantic#7786) - Fix mishandling of unions while freezing types in the `mypy` plugin by [@dmontagu](https://github.com/dmontagu) in [#7411](pydantic/pydantic#7411) - Fix `mypy` error on untyped `ClassVar` by [@vincent-hachin-wmx](https://github.com/vincent-hachin-wmx) in [#8138](pydantic/pydantic#8138) - Only compare pydantic fields in `BaseModel.__eq__` instead of whole `__dict__` by [@QuentinSoubeyranAqemia](https://github.com/QuentinSoubeyranAqemia) in [#7825](pydantic/pydantic#7825) - Update `strict` docstring in `model_validate` method. by [@LukeTonin](https://github.com/LukeTonin) in [#8223](pydantic/pydantic#8223) - Fix overload position of `computed_field` by [@Viicos](https://github.com/Viicos) in [#8227](pydantic/pydantic#8227) - Fix custom type type casting used in multiple attributes by [@ianhfc](https://github.com/ianhfc) in [#8066](pydantic/pydantic#8066) - Fix issue not allowing `validate_call` decorator to be dynamically assigned to a class method by [@jusexton](https://github.com/jusexton) in [#8249](pydantic/pydantic#8249) - Fix issue `unittest.mock` deprecation warnings by [@ibleedicare](https://github.com/ibleedicare) in [#8262](pydantic/pydantic#8262) - Added tests for the case `JsonValue` contains subclassed primitive values by [@jusexton](https://github.com/jusexton) in [#8286](pydantic/pydantic#8286) - Fix `mypy` error on free before validator (classmethod) by [@sydney-runkle](https://github.com/sydney-runkle) in [#8285](pydantic/pydantic#8285) - Fix `to_snake` conversion by [@jevins09](https://github.com/jevins09) in [#8316](pydantic/pydantic#8316) - Fix type annotation of `ModelMetaclass.__prepare__` by [@slanzmich](https://github.com/slanzmich) in [#8305](pydantic/pydantic#8305) - Disallow `config` specification when initializing a `TypeAdapter` when the annotated type has config already by [@sydney-runkle](https://github.com/sydney-runkle) in [#8365](pydantic/pydantic#8365) - Fix a naming issue with JSON schema for generics parametrized by recursive type aliases by [@dmontagu](https://github.com/dmontagu) in [#8389](pydantic/pydantic#8389) - Fix type annotation in pydantic people script by [@shenxiangzhuang](https://github.com/shenxiangzhuang) in [#8402](pydantic/pydantic#8402) - Add support for field `alias` in `dataclass` signature by [@NeevCohen](https://github.com/NeevCohen) in [#8387](pydantic/pydantic#8387) - Fix bug with schema generation with `Field(...)` in a forward ref by [@dmontagu](https://github.com/dmontagu) in [#8494](pydantic/pydantic#8494) - Fix ordering of keys in `__dict__` with `model_construct` call by [@sydney-runkle](https://github.com/sydney-runkle) in [#8500](pydantic/pydantic#8500) - Fix module `path_type` creation when globals does not contain `__name__` by [@hramezani](https://github.com/hramezani) in [#8470](pydantic/pydantic#8470) - Fix for namespace issue with dataclasses with `from __future__ import annotations` by [@sydney-runkle](https://github.com/sydney-runkle) in [#8513](pydantic/pydantic#8513) - Fix: make function validator types positional-only by [@pmmmwh](https://github.com/pmmmwh) in [#8479](pydantic/pydantic#8479) - Fix usage of `@deprecated` by [@Viicos](https://github.com/Viicos) in [#8294](pydantic/pydantic#8294) - Add more support for private attributes in `model_construct` call by [@sydney-runkle](https://github.com/sydney-runkle) in [#8525](pydantic/pydantic#8525) - Use a stack for the types namespace by [@dmontagu](https://github.com/dmontagu) in [#8378](pydantic/pydantic#8378) - Fix schema-building bug with `TypeAliasType` for types with refs by [@dmontagu](https://github.com/dmontagu) in [#8526](pydantic/pydantic#8526) - Support `pydantic.Field(repr=False)` in dataclasses by [@tigeryy2](https://github.com/tigeryy2) in [#8511](pydantic/pydantic#8511) - Override `dataclass_transform` behavior for `RootModel` by [@Viicos](https://github.com/Viicos) in [#8163](pydantic/pydantic#8163) - Refactor signature generation for simplicity by [@sydney-runkle](https://github.com/sydney-runkle) in [#8572](pydantic/pydantic#8572) - Fix ordering bug of PlainValidator annotation by [@Anvil](https://github.com/Anvil) in [#8567](pydantic/pydantic#8567) - Fix `exclude_none` for json serialization of `computed_field`s by [@sydney-runkle](https://github.com/sydney-runkle) in [pydantic/pydantic-core#1098](pydantic/pydantic-core#1098) - Support yyyy-MM-DD string for datetimes by [@sydney-runkle](https://github.com/sydney-runkle) in [pydantic/pydantic-core#1124](pydantic/pydantic-core#1124) - Tweak ordering of definitions in generated schemas by [@StrawHatDrag0n](https://github.com/StrawHatDrag0n) in [#8583](pydantic/pydantic#8583) ##### New Contributors ##### `pydantic` - [@ekeew](https://github.com/ekeew) made their first contribution in [#6874](pydantic/pydantic#6874) - [@lambertsbennett](https://github.com/lambertsbennett) made their first contribution in [#8054](pydantic/pydantic#8054) - [@vincent-hachin-wmx](https://github.com/vincent-hachin-wmx) made their first contribution in [#8138](pydantic/pydantic#8138) - [@QuentinSoubeyranAqemia](https://github.com/QuentinSoubeyranAqemia) made their first contribution in [#7825](pydantic/pydantic#7825) - [@ariebovenberg](https://github.com/ariebovenberg) made their first contribution in [#8072](pydantic/pydantic#8072) - [@LukeTonin](https://github.com/LukeTonin) made their first contribution in [#8223](pydantic/pydantic#8223) - [@denisart](https://github.com/denisart) made their first contribution in [#8231](pydantic/pydantic#8231) - [@ianhfc](https://github.com/ianhfc) made their first contribution in [#8066](pydantic/pydantic#8066) - [@eonu](https://github.com/eonu) made their first contribution in [#8255](pydantic/pydantic#8255) - [@amandahla](https://github.com/amandahla) made their first contribution in [#8263](pydantic/pydantic#8263) - [@ibleedicare](https://github.com/ibleedicare) made their first contribution in [#8262](pydantic/pydantic#8262) - [@jevins09](https://github.com/jevins09) made their first contribution in [#8316](pydantic/pydantic#8316) - [@cuu508](https://github.com/cuu508) made their first contribution in [#8322](pydantic/pydantic#8322) - [@slanzmich](https://github.com/slanzmich) made their first contribution in [#8305](pydantic/pydantic#8305) - [@jensenbox](https://github.com/jensenbox) made their first contribution in [#8331](pydantic/pydantic#8331) - [@szepeviktor](https://github.com/szepeviktor) made their first contribution in [#8356](pydantic/pydantic#8356) - [@Elkiwa](https://github.com/Elkiwa) made their first contribution in [#8341](pydantic/pydantic#8341) - [@parhamfh](https://github.com/parhamfh) made their first contribution in [#8395](pydantic/pydantic#8395) - [@shenxiangzhuang](https://github.com/shenxiangzhuang) made their first contribution in [#8402](pydantic/pydantic#8402) - [@NeevCohen](https://github.com/NeevCohen) made their first contribution in [#8387](pydantic/pydantic#8387) - [@zby](https://github.com/zby) made their first contribution in [#8497](pydantic/pydantic#8497) - [@patelnets](https://github.com/patelnets) made their first contribution in [#8491](pydantic/pydantic#8491) - [@edwardwli](https://github.com/edwardwli) made their first contribution in [#8503](pydantic/pydantic#8503) - [@luca-matei](https://github.com/luca-matei) made their first contribution in [#8507](pydantic/pydantic#8507) - [@Jocelyn-Gas](https://github.com/Jocelyn-Gas) made their first contribution in [#8437](pydantic/pydantic#8437) - [@bL34cHig0](https://github.com/bL34cHig0) made their first contribution in [#8501](pydantic/pydantic#8501) - [@tigeryy2](https://github.com/tigeryy2) made their first contribution in [#8511](pydantic/pydantic#8511) - [@geospackle](https://github.com/geospackle) made their first contribution in [#8537](pydantic/pydantic#8537) - [@Anvil](https://github.com/Anvil) made their first contribution in [#8567](pydantic/pydantic#8567) - [@hungtsetse](https://github.com/hungtsetse) made their first contribution in [#8546](pydantic/pydantic#8546) - [@StrawHatDrag0n](https://github.com/StrawHatDrag0n) made their first contribution in [#8583](pydantic/pydantic#8583) ##### `pydantic-core` - [@mariuswinger](https://github.com/mariuswinger) made their first contribution in [pydantic/pydantic-core#1087](pydantic/pydantic-core#1087) - [@adamchainz](https://github.com/adamchainz) made their first contribution in [pydantic/pydantic-core#1090](pydantic/pydantic-core#1090) - [@akx](https://github.com/akx) made their first contribution in [pydantic/pydantic-core#1123](pydantic/pydantic-core#1123) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).  Reviewed-on: https://git.apud.pl/jacek/adventofcode/pulls/57 Co-authored-by: Renovate <renovate@apud.pl> Co-committed-by: Renovate <renovate@apud.pl>

Fix bug with deferred mypy analysis

f44eb74

dmontagu mentioned this pull request Jan 29, 2024

Mypy crashes when trying to parse a pydantic model with optional fields #8665

Closed

1 task

dmontagu added 2 commits January 29, 2024 11:06

Add a test

20f7a3b

Fix CI

b8b500b

dmontagu added the relnotes-fix Used for bugfixes. label Jan 29, 2024

dmontagu changed the title ~~Fix bug with deferred mypy analysis~~ Fix bug with mypy plugin and no_strict_optional = True Jan 29, 2024

RogerHYang mentioned this pull request Jan 29, 2024

chore: pin pydantic version for mypy Arize-ai/phoenix#2147

Merged

sydney-runkle approved these changes Jan 30, 2024

View reviewed changes

sydney-runkle merged commit 0404872 into main Jan 30, 2024
55 checks passed

sydney-runkle deleted the dmontagu/fix-mypy-defer-analysis-bug branch January 30, 2024 13:23

frankie567 mentioned this pull request Feb 1, 2024

AssertionError: Internal error: unresolved placeholder type python/mypy#16809

Closed

frankie567 added a commit to polarsource/polar that referenced this pull request Feb 1, 2024

server: pin Pydantic to prevent an ongoing bug with mypy

82a01dc

Revert this when pydantic/pydantic#8666 is released. Likely in Pydantic 2.6.1

frankie567 added a commit to polarsource/polar that referenced this pull request Feb 1, 2024

server: pin Pydantic to prevent an ongoing bug with mypy

1ed8c17

Revert this when pydantic/pydantic#8666 is released. Likely in Pydantic 2.6.1

sydney-runkle pushed a commit that referenced this pull request Feb 5, 2024

Fix bug with mypy plugin and no_strict_optional = True (#8666)

50259d7

HenriBlacksmith mentioned this pull request Feb 7, 2024

Known 2.6 Issues - WIP Fixes for v2.6.2 #8741

Closed

2 tasks

sydney-runkle mentioned this pull request Feb 8, 2024

mypy INTERNAL ERROR: maximum semantic analysis iteration count reached #8760

Closed

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix bug with mypy plugin and `no_strict_optional = True` #8666

Fix bug with mypy plugin and `no_strict_optional = True` #8666

dmontagu commented Jan 29, 2024 •

edited

cloudflare-pages bot commented Jan 29, 2024 •

edited

codspeed-hq bot commented Jan 29, 2024 •

edited

sydney-runkle left a comment

Fix bug with mypy plugin and no_strict_optional = True #8666

Fix bug with mypy plugin and no_strict_optional = True #8666

Conversation

dmontagu commented Jan 29, 2024 • edited

cloudflare-pages bot commented Jan 29, 2024 • edited

Deploying with Cloudflare Pages

codspeed-hq bot commented Jan 29, 2024 • edited

CodSpeed Performance Report

Merging #8666 will not alter performance

Summary

sydney-runkle left a comment

Choose a reason for hiding this comment

Fix bug with mypy plugin and `no_strict_optional = True` #8666

Fix bug with mypy plugin and `no_strict_optional = True` #8666

dmontagu commented Jan 29, 2024 •

edited

cloudflare-pages bot commented Jan 29, 2024 •

edited

codspeed-hq bot commented Jan 29, 2024 •

edited