Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python 3.13t crashes when constructing code objects with non-standard constants #130851

Closed
colesbury opened this issue Mar 4, 2025 · 0 comments
Closed
Labels
3.13 bugs and security fixes 3.14 new features, bugs and security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) topic-free-threading type-bug An unexpected behavior, bug, or error

Comments

@colesbury
Copy link
Contributor

colesbury commented Mar 4, 2025

Bug report

PyTorch Dynamo constructs code objects with non-standard constants. This is unusual, but seems to work with the default (non-free threaded build) of CPython.

However, it crashes in the free threaded build due to a few assertions when we try to de-dupe (intern) and immortalize constants:

In compare_constants:

cpython/Objects/codeobject.c

Lines 2671 to 2672 in 3dd3675

_Py_FatalErrorFormat("unexpected type in compare_constants: %s",
Py_TYPE(op1)->tp_name);

In `hash_const:

cpython/Objects/codeobject.c

Lines 2692 to 2694 in 3dd3675

// This should never happen: all the constants we support have
// infallible hash functions.
Py_FatalError("code: hash failed");

We should more gracefully handle unexpected code object constants in the free threading build. I think the interning code should behave like _PyCode_ConstantKey() where unhandled objects types are treated as unequal if they are not the same instance (i.e., identity comparison for unexpected types).

cc @williamwen42

Linked PRs

@colesbury colesbury added 3.13 bugs and security fixes 3.14 new features, bugs and security fixes topic-free-threading type-bug An unexpected behavior, bug, or error labels Mar 4, 2025
@picnixz picnixz added the interpreter-core (Objects, Python, Grammar, and Parser dirs) label Mar 4, 2025
colesbury added a commit to colesbury/cpython that referenced this issue Mar 4, 2025
The bytecode compiler only generates a few different types of constants,
like str, int, tuple, slices, etc. Users can construct code objects with
various unusual constants, including ones that are not hashable or not
even constant.

The free threaded build previously crashed with a fatal error when
confronted with these constants. Instead, treat distinct objects of
otherwise unhandled types as not equal for the purposes of deduplication.
colesbury added a commit to colesbury/cpython that referenced this issue Mar 4, 2025
The bytecode compiler only generates a few different types of constants,
like str, int, tuple, slices, etc. Users can construct code objects with
various unusual constants, including ones that are not hashable or not
even constant.

The free threaded build previously crashed with a fatal error when
confronted with these constants. Instead, treat distinct objects of
otherwise unhandled types as not equal for the purposes of deduplication.
Yhg1s pushed a commit that referenced this issue Mar 5, 2025
The bytecode compiler only generates a few different types of constants,
like str, int, tuple, slices, etc. Users can construct code objects with
various unusual constants, including ones that are not hashable or not
even constant.

The free threaded build previously crashed with a fatal error when
confronted with these constants. Instead, treat distinct objects of
otherwise unhandled types as not equal for the purposes of deduplication.
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Mar 5, 2025
…thonGH-130853)

The bytecode compiler only generates a few different types of constants,
like str, int, tuple, slices, etc. Users can construct code objects with
various unusual constants, including ones that are not hashable or not
even constant.

The free threaded build previously crashed with a fatal error when
confronted with these constants. Instead, treat distinct objects of
otherwise unhandled types as not equal for the purposes of deduplication.
(cherry picked from commit 2905690)

Co-authored-by: Sam Gross <colesbury@gmail.com>
colesbury added a commit that referenced this issue Mar 5, 2025
…H-130853) (#130880)

The bytecode compiler only generates a few different types of constants,
like str, int, tuple, slices, etc. Users can construct code objects with
various unusual constants, including ones that are not hashable or not
even constant.

The free threaded build previously crashed with a fatal error when
confronted with these constants. Instead, treat distinct objects of
otherwise unhandled types as not equal for the purposes of deduplication.
(cherry picked from commit 2905690)

Co-authored-by: Sam Gross <colesbury@gmail.com>
@colesbury colesbury reopened this Mar 5, 2025
colesbury added a commit to colesbury/cpython that referenced this issue Mar 5, 2025
The test immortalizes some regular object instances, which causes the
refleak checks to fail.
colesbury added a commit to colesbury/cpython that referenced this issue Mar 5, 2025
The free threading build interns and immortalizes most constants
generated by the bytecode compiler. However, users can construct their
own code objects with arbitrary constants. Don't intern or immortalize
these objects if they are not a type that we know how to handle.

This fixes a refleak failure in the recently added
`test_code.test_unusual_constants` test.

It also fixes a potential crash that could happen when we try to
destory a immortalized object on interpreter shutdown.
colesbury added a commit that referenced this issue Mar 7, 2025
…130901)

The free-threading build interns and immortalizes most constants
generated by the bytecode compiler. However, users can construct their
own code objects with arbitrary constants. We should not intern or
immortalize these objects if they are not of a type that we know how to
handle.

This change fixes a reference leak failure in the recently added
`test_code.test_unusual_constants` test. It also addresses a potential
crash that could occur when attempting to destroy an immortalized
object during interpreter shutdown.
colesbury added a commit to colesbury/cpython that referenced this issue Mar 7, 2025
…he compiler (pythonGH-130901)

The free-threading build interns and immortalizes most constants
generated by the bytecode compiler. However, users can construct their
own code objects with arbitrary constants. We should not intern or
immortalize these objects if they are not of a type that we know how to
handle.

This change fixes a reference leak failure in the recently added
`test_code.test_unusual_constants` test. It also addresses a potential
crash that could occur when attempting to destroy an immortalized
object during interpreter shutdown.
(cherry picked from commit 12db452)

Co-authored-by: Sam Gross <colesbury@gmail.com>
colesbury added a commit that referenced this issue Mar 8, 2025
…piler (GH-130901) (#130953)

The free-threading build interns and immortalizes most constants
generated by the bytecode compiler. However, users can construct their
own code objects with arbitrary constants. We should not intern or
immortalize these objects if they are not of a type that we know how to
handle.

This change fixes a reference leak failure in the recently added
`test_code.test_unusual_constants` test. It also addresses a potential
crash that could occur when attempting to destroy an immortalized
object during interpreter shutdown.
(cherry picked from commit 12db452)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.13 bugs and security fixes 3.14 new features, bugs and security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) topic-free-threading type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

No branches or pull requests

2 participants