Avoid allocating when exiting frame; it may be unsafe.

[markshannon]

BPO	45786
Nosy	@vstinner, @markshannon, @pablogsal
PRs	bpo-45786: Allocate space for frame in frame object. #29729 bpo-45786: Remove _PyFrame_Fini() and _PyFrame_DebugMallocStats() #31874

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/markshannon'
closed_at = <Date 2022-01-13.18:35:35.826>
created_at = <Date 2021-11-11.15:04:09.759>
labels = ['interpreter-core', 'type-crash', '3.11']
title = 'Avoid allocating when exiting frame; it may be unsafe.'
updated_at = <Date 2022-03-18.10:17:20.770>
user = 'https://github.com/markshannon'

bugs.python.org fields:

activity = <Date 2022-03-18.10:17:20.770>
actor = 'vstinner'
assignee = 'Mark.Shannon'
closed = True
closed_date = <Date 2022-01-13.18:35:35.826>
closer = 'Mark.Shannon'
components = ['Interpreter Core']
creation = <Date 2021-11-11.15:04:09.759>
creator = 'Mark.Shannon'
dependencies = []
files = []
hgrepos = []
issue_num = 45786
keywords = ['patch', '3.11regression']
message_count = 6.0
messages = ['406163', '406165', '406166', '407271', '410506', '415480']
nosy_count = 3.0
nosy_names = ['vstinner', 'Mark.Shannon', 'pablogsal']
pr_nums = ['29729', '31874']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue45786'
versions = ['Python 3.11']

[markshannon]

We exiting a frame (returning from a Python function) we have to release the stack allocated frame. If a heap-allocated frame object exists, we need to copy the contents of the (stack) frame into the frame object.
However, this involves allocating memory for the copy. Allocating memory can invoke GC, causing arbitrary code to be run, or the allocation can fail. Either leaves us in a precarious state, which may be unsafe.

I haven't been able to produce a crash, but I'm not sure that there isn't a potential crash lurking there either.

The fix is fairly simple. Allocate space for the copy of the frame at the end of the frame object. Then we need to copy the data, space will have already been allocated, and nothing can fail.

Since, in theory, heap-allocated frames are relatively rare, the extra memory used won't be an issue.

[pablogsal]

Allocating memory can invoke GC

Technically, only allocating objects can trigger GC (specifically initializing them). All the malloc APIs don't currently call the GC by themselves.

[markshannon]

Ok, so it is not as bad as I thought.

A failed allocation might leave us with an invalid frameobject, though. So it is still worth fixing.

[markshannon]

New changeset 6092957 by Mark Shannon in branch 'main':
bpo-45786: Allocate space for frame in frame object. (GH-29729)
6092957

[markshannon]

With https://bugs.python.org/issue45963 both frame objects and generators contain space for the frame, so no allocation occurs on exit.

[vstinner]

New changeset 2217462 by Victor Stinner in branch 'main':
bpo-45786: Remove _PyFrame_Fini() and _PyFrame_DebugMallocStats() (GH-31874)
2217462