gh-142737: Handle lost io.open in _Py_FindSourceFile

[johnslavik]

This fixes the problem in the issue.
WIP: a regrtest is needed.

• Issue: assertion failure at Objects/call.c:618: PyObject *callmethod(PyThreadState *, PyObject *, const char *, struct __va_list_tag *): Assertion 'callable != NULL' failed. #142737

[johnslavik]

Testing this will be harder because we're in a scenario where display_source_line is fully broken, and the C impl of traceback doesn't use linecache. So need to fix up tests to be able to not expect lines displayed.

[johnslavik]

Not sure about whether reusing TracebackFormatMixin is a good idea. These tests are specifically built on an assumption that lines will appear in the tracebacks.

[johnslavik]

For some reason the test doesn't crash for a build without the fix (rc is 1, not 134; stderr is as expected after fix).
Outside the test, the snippet from the test does crash for the old build.

I'll need to tweak the test and we're ready.

[johnslavik]

Oh, I think I know why. It doesn't look for source when we pass the code via -c.

[ZeroIntensity]

And perhaps we should put an assert(callable != NULL) in _PyObject_CallMethodFormat, what do you think? It relies on that assumption.

I was going to suggest that, go for it.

[johnslavik]

@ZeroIntensity ready

[ZeroIntensity]

LGTM!

[miss-islington-app]

Thanks @johnslavik for the PR, and @ZeroIntensity for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13, 3.14.
🐍🍒⛏🤖

[miss-islington-app]

Sorry, @johnslavik and @ZeroIntensity, I could not cleanly backport this to 3.13 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker f277781bba684322dffffe45cd878f4652ccf7e4 3.13

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot wasm32-wasi 3.x (tier-2) has failed when building commit f277781.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1046/builds/10487) and take a look at the build logs.
4. Check if the failure is related to this commit (f277781) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1046/builds/10487

Summary of the results of the build (if available):

Click to see traceback logs

remote: Enumerating objects: 14, done.        
remote: Counting objects:   9% (1/11)        
remote: Counting objects:  18% (2/11)        
remote: Counting objects:  27% (3/11)        
remote: Counting objects:  36% (4/11)        
remote: Counting objects:  45% (5/11)        
remote: Counting objects:  54% (6/11)        
remote: Counting objects:  63% (7/11)        
remote: Counting objects:  72% (8/11)        
remote: Counting objects:  81% (9/11)        
remote: Counting objects:  90% (10/11)        
remote: Counting objects: 100% (11/11)        
remote: Counting objects: 100% (11/11), done.        
remote: Compressing objects:  12% (1/8)        
remote: Compressing objects:  25% (2/8)        
remote: Compressing objects:  37% (3/8)        
remote: Compressing objects:  50% (4/8)        
remote: Compressing objects:  62% (5/8)        
remote: Compressing objects:  75% (6/8)        
remote: Compressing objects:  87% (7/8)        
remote: Compressing objects: 100% (8/8)        
remote: Compressing objects: 100% (8/8), done.        
remote: Total 14 (delta 3), reused 3 (delta 3), pack-reused 3 (from 2)        
From https://github.com/python/cpython
 * branch                    main       -> FETCH_HEAD
Note: switching to 'f277781bba684322dffffe45cd878f4652ccf7e4'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at f277781bba6 gh-142737: Handle lost `io.open` in `_Py_FindSourceFile` (GH-142747)
Switched to and reset branch 'main'

⚠️ WARNING: This script is deprecated and slated for removal in Python 3.20; execute the `wasi/` directory instead (i.e. `python Tools/wasm/wasi`)

configure: WARNING: no system libmpdec found; falling back to pure-Python version for the decimal module

⚠️ WARNING: This script is deprecated and slated for removal in Python 3.20; execute the `wasi/` directory instead (i.e. `python Tools/wasm/wasi`)


⚠️ WARNING: This script is deprecated and slated for removal in Python 3.20; execute the `wasi/` directory instead (i.e. `python Tools/wasm/wasi`)

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.bcannon-wasi.wasi.debug/build/Tools/wasm/wasi.py", line 12, in <module>
    runpy.run_path(pathlib.Path(__file__).parent / "wasi", run_name="__main__")
  File "<frozen runpy>", line 304, in run_path
  File "<frozen runpy>", line 88, in _run_code
  File "/home/buildbot/buildarea/3.x.bcannon-wasi.wasi.debug/build/Tools/wasm/wasi/__main__.py", line 554, in <module>
    main()
  File "/home/buildbot/buildarea/3.x.bcannon-wasi.wasi.debug/build/Tools/wasm/wasi/__main__.py", line 550, in main
    dispatch[context.subcommand](context)
  File "/home/buildbot/buildarea/3.x.bcannon-wasi.wasi.debug/build/Tools/wasm/wasi/__main__.py", line 123, in wrapper
    return func(context, working_dir)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.bcannon-wasi.wasi.debug/build/Tools/wasm/wasi/__main__.py", line 340, in configure_wasi_python
    raise FileNotFoundError(
FileNotFoundError: wasmtime not found; download from https://github.com/bytecodealliance/wasmtime

⚠️ WARNING: This script is deprecated and slated for removal in Python 3.20; execute the `wasi/` directory instead (i.e. `python Tools/wasm/wasi`)

[bedevere-app]

GH-142773 is a backport of this pull request to the 3.14 branch.

[bedevere-app]

GH-142774 is a backport of this pull request to the 3.13 branch.