gh-108511: Add C API functions which do not silently ignore errors #109025

serhiy-storchaka · 2023-09-06T20:13:30Z

Add the following functions:

PyObject_HasAttrWithError()
PyObject_HasAttrStringWithError()
PyMapping_HasKeyWithError()
PyMapping_HasKeyStringWithError()

Issue: C API: Add replacements for PyObject_HasAttr() etc #108511

📚 Documentation preview 📚: https://cpython-previews--109025.org.readthedocs.build/

Add the following functions: * PyObject_HasAttrWithError() * PyObject_HasAttrStringWithError() * PyMapping_HasKeyWithError() * PyMapping_HasKeyStringWithError()

bedevere-bot · 2023-09-17T11:36:25Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x RHEL8 3.x has failed when building commit add16f1.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/509/builds/4924) and take a look at the build logs.
Check if the failure is related to this commit (add16f1) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/509/builds/4924

Failed tests:

test_tools

Failed subtests:

test_freeze_simple_script - test.test_tools.test_freeze.TestFreeze.test_freeze_simple_script

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel8-z/build/Lib/test/test_tools/test_freeze.py", line 28, in test_freeze_simple_script
    outdir, scriptfile, python = helper.prepare(script, outdir)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel8-z/build/Tools/freeze/test/freeze.py", line 146, in prepare
    copy_source_tree(srcdir, SRCDIR)
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel8-z/build/Tools/freeze/test/freeze.py", line 95, in copy_source_tree
    shutil.copytree(oldroot, newroot, ignore=ignore_non_src)
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel8-z/build/Lib/shutil.py", line 588, in copytree
    return _copytree(entries=entries, src=src, dst=dst, symlinks=symlinks,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel8-z/build/Lib/shutil.py", line 542, in _copytree
    raise Error(errors)
shutil.Error: [('/home/dje/cpython-buildarea/3.x.edelsohn-rhel8-z/build/build/test_python_2730921æ/tmpiuadha4u', '/tmp/test_python_yi9eetzr/tmp8zxr_xn3/cpython/build/test_python_2730921æ/tmpiuadha4u', "[Errno 2] No such file or directory: '/home/dje/cpython-buildarea/3.x.edelsohn-rhel8-z/build/build/test_python_2730921æ/tmpiuadha4u'")]

bedevere-bot · 2023-09-17T11:43:18Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot aarch64 Debian Clang LTO + PGO 3.x has failed when building commit add16f1.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/1084/builds/2016) and take a look at the build logs.
Check if the failure is related to this commit (add16f1) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/1084/builds/2016

Failed tests:

test.test_multiprocessing_fork.test_misc

Failed subtests:

test_nested_startmethod - test.test_multiprocessing_fork.test_misc.TestStartMethod.test_nested_startmethod

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/var/lib/buildbot/workers/arm64-clang/3.x.gps-arm64-debian.clang.lto-pgo/build/Lib/test/_test_multiprocessing.py", line 5475, in test_nested_startmethod
    self.assertEqual(results, [2, 1])
AssertionError: Lists differ: [1, 2] != [2, 1]

bedevere-bot · 2023-09-17T11:50:14Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x RHEL7 LTO + PGO 3.x has failed when building commit add16f1.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/244/builds/5456) and take a look at the build logs.
Check if the failure is related to this commit (add16f1) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/244/builds/5456

Failed tests:

test.test_asyncio.test_subprocess

Failed subtests:

test_subprocess_consistent_callbacks - test.test_asyncio.test_subprocess.SubprocessThreadedWatcherTests.test_subprocess_consistent_callbacks

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto-pgo/build/Lib/test/test_asyncio/test_subprocess.py", line 788, in test_subprocess_consistent_callbacks
    self.loop.run_until_complete(main())
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto-pgo/build/Lib/asyncio/base_events.py", line 664, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel-z.lto-pgo/build/Lib/test/test_asyncio/test_subprocess.py", line 780, in main
    self.assertEqual(events, [
AssertionError: Lists differ: [('pi[29 chars]t'), 'pipe_connection_lost', ('pipe_data_recei[57 chars]ted'] != [('pi[29 chars]t'), ('pipe_data_received', 2, b'stderr'), 'pi[57 chars]ted']

bedevere-bot · 2023-09-17T12:03:31Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Debian root 3.x has failed when building commit add16f1.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/345/builds/5854) and take a look at the build logs.
Check if the failure is related to this commit (add16f1) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/345/builds/5854

Failed tests:

test_signal

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_signal.py", line 1287, in test_stress_delivery_dependent
    self.assertEqual(len(sigs), N, "Some signals were lost")
AssertionError: 4543 != 10000 : Some signals were lost

bedevere-bot · 2023-09-17T12:17:36Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot PPC64LE RHEL7 LTO + PGO 3.x has failed when building commit add16f1.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/43/builds/4241) and take a look at the build logs.
Check if the failure is related to this commit (add16f1) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/43/builds/4241

Failed tests:

test.test_asyncio.test_unix_events
test.test_multiprocessing_fork.test_misc

Failed subtests:

test_fork_signal_handling - test.test_asyncio.test_unix_events.TestFork.test_fork_signal_handling
test_nested_startmethod - test.test_multiprocessing_fork.test_misc.TestStartMethod.test_nested_startmethod

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/test/_test_multiprocessing.py", line 5475, in test_nested_startmethod
    self.assertEqual(results, [2, 1])
AssertionError: Lists differ: [1, 2] != [2, 1]


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/unittest/async_case.py", line 90, in _callTestMethod
    if self._callMaybeAsync(method) is not None:
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/unittest/async_case.py", line 117, in _callMaybeAsync
    return self._asyncioTestContext.run(func, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/test/support/hashlib_helper.py", line 49, in wrapper
    return func_or_class(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/test/test_asyncio/test_unix_events.py", line 1937, in test_fork_signal_handling
    self.assertTrue(child_handled.is_set())
AssertionError: False is not true

bedevere-bot · 2023-09-17T12:37:10Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot PPC64LE RHEL8 LTO 3.x has failed when building commit add16f1.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/361/builds/4062) and take a look at the build logs.
Check if the failure is related to this commit (add16f1) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/361/builds/4062

Failed tests:

test.test_asyncio.test_unix_events
test.test_asyncio.test_subprocess

Failed subtests:

test_subprocess_consistent_callbacks - test.test_asyncio.test_subprocess.SubprocessThreadedWatcherTests.test_subprocess_consistent_callbacks
test_fork_signal_handling - test.test_asyncio.test_unix_events.TestFork.test_fork_signal_handling

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le.lto/build/Lib/test/test_asyncio/test_subprocess.py", line 788, in test_subprocess_consistent_callbacks
    self.loop.run_until_complete(main())
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le.lto/build/Lib/asyncio/base_events.py", line 664, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le.lto/build/Lib/test/test_asyncio/test_subprocess.py", line 780, in main
    self.assertEqual(events, [
AssertionError: Lists differ: ['process_exited', ('pipe_data_received', 1, b'stdout')] != [('pipe_data_received', 1, b'stdout'), ('p[95 chars]ted']


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le.lto/build/Lib/unittest/async_case.py", line 90, in _callTestMethod
    if self._callMaybeAsync(method) is not None:
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le.lto/build/Lib/unittest/async_case.py", line 117, in _callMaybeAsync
    return self._asyncioTestContext.run(func, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le.lto/build/Lib/test/support/hashlib_helper.py", line 49, in wrapper
    return func_or_class(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le.lto/build/Lib/test/test_asyncio/test_unix_events.py", line 1937, in test_fork_signal_handling
    self.assertTrue(child_handled.is_set())
AssertionError: False is not true

bedevere-bot · 2023-09-17T12:50:22Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot PPC64LE RHEL7 LTO 3.x has failed when building commit add16f1.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/503/builds/3918) and take a look at the build logs.
Check if the failure is related to this commit (add16f1) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/503/builds/3918

Failed tests:

test.test_asyncio.test_unix_events

Failed subtests:

test_fork_signal_handling - test.test_asyncio.test_unix_events.TestFork.test_fork_signal_handling

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-ppc64le.lto/build/Lib/unittest/async_case.py", line 90, in _callTestMethod
    if self._callMaybeAsync(method) is not None:
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-ppc64le.lto/build/Lib/unittest/async_case.py", line 117, in _callMaybeAsync
    return self._asyncioTestContext.run(func, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-ppc64le.lto/build/Lib/test/support/hashlib_helper.py", line 49, in wrapper
    return func_or_class(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-ppc64le.lto/build/Lib/test/test_asyncio/test_unix_events.py", line 1937, in test_fork_signal_handling
    self.assertTrue(child_handled.is_set())
AssertionError: False is not true

vstinner · 2023-09-18T17:52:08Z

I wrote a first PR to add PyMapping_HasKeyWithError() and PyMapping_HasKeyStringWithError() functions to pythoncapi-compat: python/pythoncapi-compat#73

…ors (pythonGH-109025) Add the following functions: * PyObject_HasAttrWithError() * PyObject_HasAttrStringWithError() * PyMapping_HasKeyWithError() * PyMapping_HasKeyStringWithError()

scoder · 2023-10-15T10:45:44Z

Doc/c-api/mapping.rst

+
+   Return ``1`` if the mapping object has the key *key* and ``0`` otherwise.
+   This is equivalent to the Python expression ``key in o``.
+   On failure, return ``-1``.


Given the relevance of error handling for these new functions, I think the docs should mention explicitly under which conditions users must expect exceptions.

Suggested change

On failure, return ``-1``.

On failure, return ``-1`` with an exception set.

Sometimes I write "On error, raise an exception and return -1." Even if in C, technically, it's more "setting" an exception, I like to use Python "raise" semantics in C, it's easy for me to map C/Python this way :-)

scoder · 2023-10-15T10:46:02Z

Doc/c-api/object.rst

+
+   Returns ``1`` if *o* has the attribute *attr_name*, and ``0`` otherwise.
+   This is equivalent to the Python expression ``hasattr(o, attr_name)``.
+   On failure, return ``-1``.


Given the relevance of error handling for these new functions, I think the docs should mention explicitly under which conditions users must expect exceptions.

Suggested change

On failure, return ``-1``.

On failure, return ``-1`` with an exception set.

pythongh-108511: Add C API functions which do not silently ignore errors

ae6e789

Add the following functions: * PyObject_HasAttrWithError() * PyObject_HasAttrStringWithError() * PyMapping_HasKeyWithError() * PyMapping_HasKeyStringWithError()

serhiy-storchaka added type-feature A feature request or enhancement topic-C-API labels Sep 6, 2023

serhiy-storchaka requested review from kumaraditya303, markshannon and methane as code owners September 6, 2023 20:13

bedevere-bot added the awaiting core review label Sep 6, 2023

bedevere-bot mentioned this pull request Sep 6, 2023

C API: Add replacements for PyObject_HasAttr() etc #108511

Closed

regen-limited-abi

7bcdfad

serhiy-storchaka requested review from a team and encukou as code owners September 6, 2023 20:45

kumaraditya303 removed their request for review September 12, 2023 17:46

methane approved these changes Sep 12, 2023

View reviewed changes

serhiy-storchaka merged commit add16f1 into python:main Sep 17, 2023
23 checks passed

bedevere-app bot removed the awaiting core review label Sep 17, 2023

serhiy-storchaka deleted the PyObject_HasAttrWithError-PyMapping_HasKeyWithError branch September 17, 2023 11:23

scoder reviewed Oct 15, 2023

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

gh-108511: Add C API functions which do not silently ignore errors #109025

gh-108511: Add C API functions which do not silently ignore errors #109025

serhiy-storchaka commented Sep 6, 2023 •

edited by github-actions bot

bedevere-bot commented Sep 17, 2023

bedevere-bot commented Sep 17, 2023

bedevere-bot commented Sep 17, 2023

bedevere-bot commented Sep 17, 2023

bedevere-bot commented Sep 17, 2023

bedevere-bot commented Sep 17, 2023

bedevere-bot commented Sep 17, 2023

vstinner commented Sep 18, 2023

scoder Oct 15, 2023

vstinner Oct 16, 2023

scoder Oct 15, 2023

	On failure, return ``-1``.
	On failure, return ``-1`` with an exception set.

gh-108511: Add C API functions which do not silently ignore errors #109025

gh-108511: Add C API functions which do not silently ignore errors #109025

Conversation

serhiy-storchaka commented Sep 6, 2023 • edited by github-actions bot

bedevere-bot commented Sep 17, 2023

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

bedevere-bot commented Sep 17, 2023

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

bedevere-bot commented Sep 17, 2023

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

bedevere-bot commented Sep 17, 2023

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

bedevere-bot commented Sep 17, 2023

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

bedevere-bot commented Sep 17, 2023

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

bedevere-bot commented Sep 17, 2023

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

vstinner commented Sep 18, 2023

scoder Oct 15, 2023

Choose a reason for hiding this comment

vstinner Oct 16, 2023

Choose a reason for hiding this comment

scoder Oct 15, 2023

Choose a reason for hiding this comment

serhiy-storchaka commented Sep 6, 2023 •

edited by github-actions bot