Skip to content

[ty] Fix panic on incomplete except handlers#23708

Merged
charliermarsh merged 3 commits intomainfrom
charlie/except-none
Mar 5, 2026
Merged

[ty] Fix panic on incomplete except handlers#23708
charliermarsh merged 3 commits intomainfrom
charlie/except-none

Conversation

@charliermarsh
Copy link
Member

@charliermarsh charliermarsh commented Mar 4, 2026

Summary

We no longer assume that ExceptHandlerExceptHandler always has a definition.

Closes astral-sh/ty#2401.

@charliermarsh charliermarsh added the ty Multi-file analysis & type inference label Mar 4, 2026
@astral-sh-bot
Copy link

astral-sh-bot bot commented Mar 4, 2026

Typing conformance results

No changes detected ✅

@astral-sh-bot
Copy link

astral-sh-bot bot commented Mar 4, 2026

mypy_primer results

Changes were detected when running on open source projects
pydantic (https://github.com/pydantic/pydantic)
- pydantic/_internal/_core_metadata.py:87:54: error[invalid-assignment] Invalid assignment to key "pydantic_js_extra" with declared type `dict[str, Divergent] | ((dict[str, Divergent], /) -> None) | ((dict[str, Divergent], type[Any], /) -> None)` on TypedDict `CoreMetadata`: value of type `dict[object, object]`
+ pydantic/_internal/_core_metadata.py:87:54: error[invalid-assignment] Invalid assignment to key "pydantic_js_extra" with declared type `dict[str, int | float | str | ... omitted 3 union elements] | ((dict[str, int | float | str | ... omitted 3 union elements], /) -> None) | ((dict[str, int | float | str | ... omitted 3 union elements], type[Any], /) -> None)` on TypedDict `CoreMetadata`: value of type `dict[object, object]`
- pydantic/fields.py:949:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, Divergent] | ((dict[str, Divergent], /) -> None) | None`
+ pydantic/fields.py:949:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, int | float | str | ... omitted 3 union elements] | ((dict[str, int | float | str | ... omitted 3 union elements], /) -> None) | None`
- pydantic/fields.py:989:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, Divergent] | ((dict[str, Divergent], /) -> None) | None`
+ pydantic/fields.py:989:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, int | float | str | ... omitted 3 union elements] | ((dict[str, int | float | str | ... omitted 3 union elements], /) -> None) | None`
- pydantic/fields.py:1032:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, Divergent] | ((dict[str, Divergent], /) -> None) | None`
+ pydantic/fields.py:1032:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, int | float | str | ... omitted 3 union elements] | ((dict[str, int | float | str | ... omitted 3 union elements], /) -> None) | None`
- pydantic/fields.py:1072:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, Divergent] | ((dict[str, Divergent], /) -> None) | None`
+ pydantic/fields.py:1072:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, int | float | str | ... omitted 3 union elements] | ((dict[str, int | float | str | ... omitted 3 union elements], /) -> None) | None`
- pydantic/fields.py:1115:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, Divergent] | ((dict[str, Divergent], /) -> None) | None`
+ pydantic/fields.py:1115:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, int | float | str | ... omitted 3 union elements] | ((dict[str, int | float | str | ... omitted 3 union elements], /) -> None) | None`
- pydantic/fields.py:1154:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, Divergent] | ((dict[str, Divergent], /) -> None) | None`
+ pydantic/fields.py:1154:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, int | float | str | ... omitted 3 union elements] | ((dict[str, int | float | str | ... omitted 3 union elements], /) -> None) | None`
- pydantic/fields.py:1194:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, Divergent] | ((dict[str, Divergent], /) -> None) | None`
+ pydantic/fields.py:1194:5: error[invalid-parameter-default] Default value of type `PydanticUndefinedType` is not assignable to annotated parameter type `dict[str, int | float | str | ... omitted 3 union elements] | ((dict[str, int | float | str | ... omitted 3 union elements], /) -> None) | None`
- pydantic/fields.py:1573:13: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, Divergent] | ((dict[str, Divergent], /) -> None) | None`, found `Top[dict[Unknown, Unknown]] | (((dict[str, Divergent], /) -> None) & ~Top[dict[Unknown, Unknown]]) | None`
+ pydantic/fields.py:1573:13: error[invalid-argument-type] Argument is incorrect: Expected `dict[str, int | float | str | ... omitted 3 union elements] | ((dict[str, int | float | str | ... omitted 3 union elements], /) -> None) | None`, found `Top[dict[Unknown, Unknown]] | (((dict[str, int | float | str | ... omitted 3 union elements], /) -> None) & ~Top[dict[Unknown, Unknown]]) | None`

@astral-sh-bot
Copy link

astral-sh-bot bot commented Mar 4, 2026

Memory usage report

Memory usage unchanged ✅

@charliermarsh charliermarsh marked this pull request as ready for review March 4, 2026 15:02
@charliermarsh charliermarsh added the bug Something isn't working label Mar 4, 2026
Copy link
Member

@BurntSushi BurntSushi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

/// ## Panics
/// May panic if `self` is from another file than `model`.
fn definition<'db>(&self, model: &SemanticModel<'db>) -> Definition<'db>;
fn definition<'db>(&self, model: &SemanticModel<'db>) -> Option<Definition<'db>>;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the docs here be updated to say when callers ought to return None? It's somewhat odd that in some cases it will panic (e.g., via expect_single_definition) but in others it will return None.

Comment on lines +311 to +326
if handler.name.is_some() {
Some(
handler
.definition(self)?
.scope(self.db)
.file_scope_id(self.db),
)
} else {
handler
.type_
.as_deref()
.and_then(|handled_exceptions| {
index.try_expression_scope_id(handled_exceptions)
})
.or(Some(FileScopeId::global()))
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The name.is_some and definiton(self)? sort of encode the same check. Can we just use if let Some(definition) = handler.definition(..) instead?

impl HasDefinition for $ty {
#[inline]
fn definition<'db>(&self, model: &SemanticModel<'db>) -> Definition<'db> {
fn definition<'db>(&self, model: &SemanticModel<'db>) -> Option<Definition<'db>> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a bit unfortunate. Have you explored whether we could always create a definition in ExceptHandler even if the name is missing? Are there any undesired side effects if we would do so?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hah, isn't this what you suggested in astral-sh/ty#2401 (comment)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will take a look.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haha, I completely forgot about that. It's certainly the easiest change :)

Copy link
Member

@MichaReiser MichaReiser Mar 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could also consider removing the HasDefinition implementation on ExceptHandler. I don't know what the fallout of that is

Edit: Or reconsider implementing HasDefinition and HasType for ExceptHandlerExceptHandler. Maybe it's better to change the call sites to call the same methods on the name instead

Ahhh... this doesn't work because name is not an AST node. Sooo annoying

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I changed things up to remove HasDefinition from ExceptHandler.

ast::AnyNodeRef::StmtFunctionDef(function) => Some(
function
.definition(self)
.definition(self)?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like all these early returns assume that function.definition is always Some (maybe that's fine; it just feels a bit fragile). If it's none, we'd probably want to prefer using the fallback scope instead of returning None.

@AlexWaygood AlexWaygood removed their request for review March 4, 2026 15:24
@carljm carljm removed their request for review March 4, 2026 19:52
@charliermarsh charliermarsh enabled auto-merge (squash) March 5, 2026 00:15
@charliermarsh charliermarsh merged commit b1f43cb into main Mar 5, 2026
50 checks passed
@charliermarsh charliermarsh deleted the charlie/except-none branch March 5, 2026 00:18
impl_binding_has_ty_def!(ast::StmtClassDef);
impl_binding_has_ty_def!(ast::Parameter);
impl_binding_has_ty_def!(ast::ParameterWithDefault);
impl_binding_has_ty_def!(ast::ExceptHandlerExceptHandler);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it was necessary to remove both the HasDefinition and HasType implementations for ExceptHandlerExceptHandler. We can still implement HasType manually, since inferred_type already returns an Option.

Comment on lines +311 to +313
handler.type_.as_deref().and_then(|handled_exceptions| {
index.try_expression_scope_id(handled_exceptions)
})
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
handler.type_.as_deref().and_then(|handled_exceptions| {
index.try_expression_scope_id(handled_exceptions)
})
index.try_expression_scoipe_id(handler.type_.as_deref()?)

Comment on lines +331 to +341
/// Returns the definition for an exception-handler variable.
///
/// Exception handlers only have a definition when they bind a name (`except E as name:`).
pub fn except_handler_definition(
&self,
handler: &ast::ExceptHandlerExceptHandler,
) -> Option<Definition<'db>> {
handler.name.as_ref()?;
let index = semantic_index(self.db, self.file);
Some(index.expect_single_definition(handler))
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The API doesn't feel well aligned with HasType and HasDefinition. Maybe introduce a MaybeHasDefinition or HasOptionalDefinition trait and implement that instead. I agree, it feels a bit overkill but it's unfortunately the only way to add a method to a foreign type

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

charliermarsh added a commit that referenced this pull request Mar 5, 2026
carljm added a commit that referenced this pull request Mar 5, 2026
* main:
  Update conformance suite commit hash (#23746)
  conformance.py: Collapse the summary paragraph when nothing changed (#23745)
  [ty] Make inferred specializations line up with source types more better (#23715)
  Bump 0.15.5 (#23743)
  [ty] Render all changed diagnostics in conformance.py (#23613)
  [ty] Split deferred checks out of `types/infer/builder.rs` (#23740)
  Discover markdown files by default in preview mode (#23434)
  [ty] Use `HasOptionalDefinition` for `except` handlers (#23739)
  [ty] Fix precedence of `all` selector in TOML configurations (#23723)
  [ty] Make `all` selector case sensitive (#23713)
  [ty] Add a diagnostic if a `TypeVar` is used to specialize a `ParamSpec`, or vice versa (#23738)
  [ty] Override home directory in ty tests (#23724)
  [ty] More type-variable default validation (#23639)
  [ty] Validate bare ParamSpec usage in type annotations, and support stringified ParamSpecs as the first argument to `Callable` (#23625)
  [ty] Add `all` selector to ty.json's `schema` (#23721)
  [ty] Add quotes to related issues links (#23720)
  [ty] Fix panic on incomplete except handlers (#23708)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working ty Multi-file analysis & type inference

Projects

None yet

Development

Successfully merging this pull request may close these issues.

panic for incomplete except block

3 participants