Make the `PY_UNWIND` monitoring event local

[P403n1x87]

Feature or enhancement

Proposal:

I would like to propose making the PY_UNWIND monitoring event available as a local event. The rationale for doing this is that some tools can be interested in trapping per-function exit events. Currently, this is only possible with the PY_RETURN event when a function returns with a value. However, if a function is exited with an exception, there is no way of capturing that event on just that function. Making the PY_UNWIND event would allow monitoring tools to catch all function exit events.

import random, sys

m, e = sys.monitoring, sys.monitoring.events

m.use_tool_id(0, "debugger")

def monitor(event):
    def _(f):
        m.register_callback(0, event, f)
    return _

@monitor(e.PY_START)
def _(*_, **__):
    print("entering", sys._getframe(1))

@monitor(e.PY_UNWIND)
def _(*_, **__):
    print("exiting ", sys._getframe(1), "with exception")

def foo():
    raise RuntimeError()

def bar():
    return foo()

def baz():
    return bar()

m.set_local_events(0, random.choice([foo, bar, baz]).__code__, e.PY_UNWIND | e.PY_START)

try:
    baz()
except Exception:
    pass

# entering <frame at 0x101602f80, file 'test_py_unwind_local.py', line 34, code baz>
# exiting  <frame at 0x101602f80, file 'test_py_unwind_local.py', line 35, code baz> with exception

A debugger is an example of a tool that can benefit from PY_UNWIND being a local event.

The proposed change will break the current assumption that local events always require bytecode instrumentation. Instead, the PY_UNWIND event will simply make a local tool check instead of a global one inside monitor_unwind. No changes to bytecode or any other parts are required.

This is an extension of the current implementation of PEP 669 whereby PY_UNWIND will be available as a local event. The behaviour of the already existing global event will not be touched. As such, the proposed change will be guaranteed to be backwards-compatible.

Has this already been discussed elsewhere?

I have already discussed this feature proposal on Discourse

Links to previous discussion of this feature:

https://discuss.python.org/t/pep-669-low-impact-monitoring-for-cpython/13018/82

Linked PRs

• gh-142186: make PY_UNWIND available as a local event #142179

[markshannon]

It should be possible to disable local events, and disabling is specific to the location passed to the callback.
https://docs.python.org/3/library/sys.monitoring.html#local-events
How would you implement that?

Or do you want to make PY_UNWIND its own category, that can be turned on or off per code objects, but cannot be disabled?

If we are breaking the distinction between "local" meaning specific to a location and can be disabled, and "local" meaning can be turned on or off per code object, then we might as well make all events "local" in the second sense.

[P403n1x87]

It should be possible to disable local events, and disabling is specific to the location passed to the callback. https://docs.python.org/3/library/sys.monitoring.html#local-events How would you implement that?

My understanding is that DISABLE is intended for those local events that can trigger at multiple locations. It can be used to fine-tune the events that the user wants to trap (e.g. just a sub-set of all the lines within a code object, or certain return statements). PY_UNWIND is obviously an event that is not local in the sense of coming from a certain bytecode offset. However, one might argue that the event itself happens when the execution has reached a certain offset, so it (and in fact any event) is local in this other sense. But because these events do not instrument the bytecode, they cannot be DISABLEd like other local events.

Or do you want to make PY_UNWIND its own category, that can be turned on or off per code objects, but cannot be disabled?

Because as of now all local events require instrumentation, I would be more inclined to reflect the "new" meaning of local in the documentation. That is, for events that can trigger at pre-defined locations (think compile-time), returning DISABLE will do the current job, otherwise it is ignored. For example, PY_START is a local event with instrumentation at a guaranteed single location. Returning DISABLE from its handler would undo the instrumentation, which is effectively equivalent to turning the event off on the code object. So we already have a local event in its own category.

If we are breaking the distinction between "local" meaning specific to a location and can be disabled, and "local" meaning can be turned on or off per code object, then we might as well make all events "local" in the second sense.

Ideally I would like the idea of having all the events local on a per-code object basis. I haven't looked at the other events, but the PY_UNWIND one was easy enough to implement at potentially no performance cost. If it is equally convenient to turn the other currently non-local events into local ones in this new sense then I think it would be great.

[P403n1x87]

@markshannon for the sake of making progress here, would it be acceptable to merge the proposed change that makes PY_UNWIND "local" (and ignores DISABLE), and follow that up with potentially a more comprehensive change that makes all other events local in this new sense, and restrict the relevance of DISABLE to only some of the events? As I mentioned in my previous comment, the PY_START event has a similar behaviour. The fact that it can be disabled but not turned off is just a mere consequence of the implementation (which I think is hacky anyway, as shown by the "funny" coding required to handle that case in #142136), since the instrumentation for that event takes place in just a single place, regardless of the bytecode content.