Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-110892: Return NULL for PyTrace_RETURN events caused by an exception #110909

Merged
merged 5 commits into from
Nov 2, 2023

Conversation

gaogaotiantian
Copy link
Member

@gaogaotiantian gaogaotiantian commented Oct 15, 2023

According to https://docs.python.org/3/c-api/init.html#c.Py_tracefunc, the C-level trace function should return NULL if PyTrace_RETURN is caused by an exception.

For both PyEval_SetProfile and PyEval_SetTrace, we should simply send NULL for PY_MONITORING_EVENT_PY_UNWIND event. call_trampoline will convert NULL to Py_None before calling Python profile/trace function.

Another arg related issue was fixed - we should always send None as arg to PyTrace_CALL event (or the correspoding call event for sys.setprofile), but currently in _PyEval_SetProfile, sys_profile_func3 is used as callback for PY_MONITORING_EVENT_PY_THROW (okay I wrote it, sorry about that), and sys_profile_func3 uses arg[2] as the arg to trace function, which is different than sys_trace_func3 which uses None. (Wow that's a bit confusing!)

I made it clear the original callback actually uses arg[2] and made sys_profile_func3 similar to sys_trace_func3 - they all use None now.

The test framework of test_sys_setprofile is also modified a bit to check args.

Copy link
Member

@markshannon markshannon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just a few suggestions for names

@@ -59,6 +59,16 @@ static PyObject *
sys_profile_func3(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is now only used for throw events, so maybe rename it to sys_profile_throw?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I always think the callbacks should have better names. I thought there are some naming conventions used here. I changed names of all the single-used callbacks to involve the event they are used for.

}

static PyObject *
sys_profile_func3_args2(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is only used for PyTrace_RETURN events, maybe rename it to sys_profile_return?

@@ -173,6 +183,16 @@ sys_trace_func3(
return call_trace_func(self, Py_None);
}

static PyObject *
sys_trace_func3_null(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Likewise, rename this to sys_trace_return, and rename sys_trace_func3 to sys_trace_throw?

@markshannon
Copy link
Member

Thanks for this. Much appreciated.

@markshannon markshannon merged commit f4b5588 into python:main Nov 2, 2023
27 checks passed
@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot aarch64 Debian Clang LTO + PGO 3.x has failed when building commit f4b5588.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/1084/builds/2493) and take a look at the build logs.
  4. Check if the failure is related to this commit (f4b5588) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/1084/builds/2493

Summary of the results of the build (if available):

==

Click to see traceback logs
remote: Enumerating objects: 12, done.        
remote: Counting objects:   9% (1/11)        
remote: Counting objects:  18% (2/11)        
remote: Counting objects:  27% (3/11)        
remote: Counting objects:  36% (4/11)        
remote: Counting objects:  45% (5/11)        
remote: Counting objects:  54% (6/11)        
remote: Counting objects:  63% (7/11)        
remote: Counting objects:  72% (8/11)        
remote: Counting objects:  81% (9/11)        
remote: Counting objects:  90% (10/11)        
remote: Counting objects: 100% (11/11)        
remote: Counting objects: 100% (11/11), done.        
remote: Compressing objects:   9% (1/11)        
remote: Compressing objects:  18% (2/11)        
remote: Compressing objects:  27% (3/11)        
remote: Compressing objects:  36% (4/11)        
remote: Compressing objects:  45% (5/11)        
remote: Compressing objects:  54% (6/11)        
remote: Compressing objects:  63% (7/11)        
remote: Compressing objects:  72% (8/11)        
remote: Compressing objects:  81% (9/11)        
remote: Compressing objects:  90% (10/11)        
remote: Compressing objects: 100% (11/11)        
remote: Compressing objects: 100% (11/11), done.        
remote: Total 12 (delta 0), reused 0 (delta 0), pack-reused 1        
From https://github.com/python/cpython
 * branch                  main       -> FETCH_HEAD
Note: switching to 'f4b5588bde656d8ad048b66a0be4cb5131f0d83f'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at f4b5588bde gh-110892: Return NULL for `PyTrace_RETURN` events caused by an exception (GH-110909)
Switched to and reset branch 'main'

find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
make[2]: [Makefile:2846: clean-retain-profile] Error 1 (ignored)
Python/ceval_gil.c:429:1: warning: unused function 'current_thread_holds_gil' [-Wunused-function]
current_thread_holds_gil(struct _gil_runtime_state *gil, PyThreadState *tstate)
^
1 warning generated.
Python/optimizer.c:428:9: warning: variable 'reserved' set but not used [-Wunused-but-set-variable]
    int reserved = 0;
        ^
1 warning generated.
./Modules/expat/xmlparse.c:3116:9: warning: code will never be executed [-Wunreachable-code]
        parser->m_characterDataHandler(parser->m_handlerArg, parser->m_dataBuf,
        ^~~~~~
./Modules/expat/xmlparse.c:3115:16: note: silence by adding parentheses to mark code as explicitly dead
      else if (0 && parser->m_characterDataHandler)
               ^
               /* DISABLES CODE */ ( )
./Modules/expat/xmlparse.c:4059:9: warning: code will never be executed [-Wunreachable-code]
        parser->m_characterDataHandler(parser->m_handlerArg, parser->m_dataBuf,
        ^~~~~~
./Modules/expat/xmlparse.c:4058:16: note: silence by adding parentheses to mark code as explicitly dead
      else if (0 && parser->m_characterDataHandler)
               ^
               /* DISABLES CODE */ ( )
2 warnings generated.
Python/ceval_gil.c:429:1: warning: unused function 'current_thread_holds_gil' [-Wunused-function]
current_thread_holds_gil(struct _gil_runtime_state *gil, PyThreadState *tstate)
^
1 warning generated.
Python/optimizer.c:428:9: warning: variable 'reserved' set but not used [-Wunused-but-set-variable]
    int reserved = 0;
        ^
1 warning generated.
warning: no profile data available for file "getpath_noop.c" [-Wprofile-instr-unprofiled]
1 warning generated.
warning: no profile data available for file "_freeze_module.c" [-Wprofile-instr-unprofiled]
1 warning generated.
warning: no profile data available for file "_csv.c" [-Wprofile-instr-unprofiled]
warning: no profile data available for file "rotatingtree.c" [-Wprofile-instr-unprofiled]
1 warning generated.
warning: no profile data available for file "_lsprof.c" [-Wprofile-instr-unprofiled]
1 warning generated.
1 warning generated.
warning: no profile data available for file "_xxsubinterpretersmodule.c" [-Wprofile-instr-unprofiled]
warning: no profile data available for file "_xxinterpchannelsmodule.c" [-Wprofile-instr-unprofiled]
1 warning generated.
warning: no profile data available for file "_zoneinfo.c" [-Wprofile-instr-unprofiled]
1 warning generated.
1 warning generated.
./Modules/expat/xmlparse.c:3116:9: warning: code will never be executed [-Wunreachable-code]
        parser->m_characterDataHandler(parser->m_handlerArg, parser->m_dataBuf,
        ^~~~~~
./Modules/expat/xmlparse.c:3115:16: note: silence by adding parentheses to mark code as explicitly dead
      else if (0 && parser->m_characterDataHandler)
               ^
               /* DISABLES CODE */ ( )
./Modules/expat/xmlparse.c:4059:9: warning: code will never be executed [-Wunreachable-code]
        parser->m_characterDataHandler(parser->m_handlerArg, parser->m_dataBuf,
        ^~~~~~
./Modules/expat/xmlparse.c:4058:16: note: silence by adding parentheses to mark code as explicitly dead
      else if (0 && parser->m_characterDataHandler)
               ^
               /* DISABLES CODE */ ( )
2 warnings generated.
warning: no profile data available for file "mmapmodule.c" [-Wprofile-instr-unprofiled]
1 warning generated.
warning: no profile data available for file "syslogmodule.c" [-Wprofile-instr-unprofiled]
warning: no profile data available for file "posixshmem.c" [-Wprofile-instr-unprofiled]
1 warning generated.
1 warning generated.
warning: no profile data available for file "_curses_panel.c" [-Wprofile-instr-unprofiled]
1 warning generated.
warning: no profile data available for file "_cursesmodule.c" [-Wprofile-instr-unprofiled]
warning: no profile data available for file "_uuidmodule.c" [-Wprofile-instr-unprofiled]
1 warning generated.
1 warning generated.
warning: no profile data available for file "xxsubtype.c" [-Wprofile-instr-unprofiled]
1 warning generated.
warning: no profile data available for file "_xxtestfuzz.c" [-Wprofile-instr-unprofiled]
1 warning generated.
warning: no profile data available for file "fuzzer.c" [-Wprofile-instr-unprofiled]
1 warning generated.
warning: no profile data available for file "_testimportmultiple.c" [-Wprofile-instr-unprofiled]
1 warning generated.
warning: no profile data available for file "_testclinic_limited.c" [-Wprofile-instr-unprofiled]
1 warning generated.
warning: no profile data available for file "_ctypes_test.c" [-Wprofile-instr-unprofiled]
warning: no profile data available for file "_testsinglephase.c" [-Wprofile-instr-unprofiled]
1 warning generated.
warning: no profile data available for file "xxlimited.c" [-Wprofile-instr-unprofiled]
1 warning generated.
1 warning generated.
warning: no profile data available for file "_testmultiphase.c" [-Wprofile-instr-unprofiled]
warning: no profile data available for file "xxlimited_35.c" [-Wprofile-instr-unprofiled]
1 warning generated.
1 warning generated.
warning: no profile data available for file "_testclinic.c" [-Wprofile-instr-unprofiled]
1 warning generated.
warning: profile data may be out of date: of 3 functions, 1 has mismatched data that will be ignored [-Wprofile-instr-out-of-date]
1 warning generated.

make: *** [Makefile:2067: buildbottest] Error 3

@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot x86-64 macOS 3.x has failed when building commit f4b5588.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/366/builds/5799) and take a look at the build logs.
  4. Check if the failure is related to this commit (f4b5588) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/366/builds/5799

Summary of the results of the build (if available):

==

Click to see traceback logs
remote: Enumerating objects: 12, done.        
remote: Counting objects:   9% (1/11)        
remote: Counting objects:  18% (2/11)        
remote: Counting objects:  27% (3/11)        
remote: Counting objects:  36% (4/11)        
remote: Counting objects:  45% (5/11)        
remote: Counting objects:  54% (6/11)        
remote: Counting objects:  63% (7/11)        
remote: Counting objects:  72% (8/11)        
remote: Counting objects:  81% (9/11)        
remote: Counting objects:  90% (10/11)        
remote: Counting objects: 100% (11/11)        
remote: Counting objects: 100% (11/11), done.        
remote: Compressing objects:   9% (1/11)        
remote: Compressing objects:  18% (2/11)        
remote: Compressing objects:  27% (3/11)        
remote: Compressing objects:  36% (4/11)        
remote: Compressing objects:  45% (5/11)        
remote: Compressing objects:  54% (6/11)        
remote: Compressing objects:  63% (7/11)        
remote: Compressing objects:  72% (8/11)        
remote: Compressing objects:  81% (9/11)        
remote: Compressing objects:  90% (10/11)        
remote: Compressing objects: 100% (11/11)        
remote: Compressing objects: 100% (11/11), done.        
remote: Total 12 (delta 0), reused 0 (delta 0), pack-reused 1        
From https://github.com/python/cpython
 * branch                  main       -> FETCH_HEAD
Note: switching to 'f4b5588bde656d8ad048b66a0be4cb5131f0d83f'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at f4b5588bde gh-110892: Return NULL for `PyTrace_RETURN` events caused by an exception (GH-110909)
Switched to and reset branch 'main'

configure: WARNING: pkg-config is missing. Some dependencies may not be detected correctly.

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: libpython3.13d.a(dynamic_annotations.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: libpython3.13d.a(pymath.o) has no symbols
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups
ld: warning: -undefined dynamic_lookup may not work with chained fixups

make: *** [buildbottest] Error 3

FullteaR pushed a commit to FullteaR/cpython that referenced this pull request Nov 3, 2023
@gaogaotiantian gaogaotiantian deleted the trace-profile-args branch November 3, 2023 17:29
aisk pushed a commit to aisk/cpython that referenced this pull request Feb 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants