Use PyDict_Contains() and PyDict_SetDefault() instead of PyDict_GetItemWithError()

[serhiy-storchaka]

BPO	42152
Nosy	@vstinner, @methane, @serhiy-storchaka
PRs	bpo-42152: Optimize out uses of PyDict_GetItemWithError if its result is not used. #22986

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-04-14.07:12:00.852>
created_at = <Date 2020-10-26.07:16:24.189>
labels = ['interpreter-core', 'type-feature', '3.10']
title = 'Use PyDict_Contains() and PyDict_SetDefault() instead of PyDict_GetItemWithError()'
updated_at = <Date 2021-04-14.07:12:00.851>
user = 'https://github.com/serhiy-storchaka'

bugs.python.org fields:

activity = <Date 2021-04-14.07:12:00.851>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = True
closed_date = <Date 2021-04-14.07:12:00.852>
closer = 'serhiy.storchaka'
components = ['Interpreter Core']
creation = <Date 2020-10-26.07:16:24.189>
creator = 'serhiy.storchaka'
dependencies = []
files = []
hgrepos = []
issue_num = 42152
keywords = ['patch']
message_count = 5.0
messages = ['379647', '379648', '379659', '391033', '391037']
nosy_count = 3.0
nosy_names = ['vstinner', 'methane', 'serhiy.storchaka']
pr_nums = ['22986']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue42152'
versions = ['Python 3.10']

[serhiy-storchaka]

It was common to use the code PyDict_GetItem(dict, key) == NULL to check whether the key is in the dict. PyDict_GetItem() returns borrowed reference, so no clean up is needed. And if we need to check only existence of the key, we do not need to store a value.

But PyDict_GetItem() which suppresses all internal exceptions is considered not safe, so PyDict_GetItemWithError() should be used. The code looks like:

    if (PyDict_GetItemWithError(dict, key) == NULL) {
        if (PyErr_Occurred()) {
            goto error;
        }
        // does not have the key
    }
    else {
        // has the key
    }

It can be written with using PyDict_Contains():

    int r = PyDict_Contains(dict, key);
    if (r < 0) {
        goto error;
    }
    if (r == 0) {
        // does not have the key
    }
    else {
        // has the key
    }

Advantages:

• It is more clear expression of what we do.
• It is more simple and efficient in PyPy, because it has to keep borrowed references and track their lifetime.
• It may be more efficient in CPython, because calling PyErr_Occurred() has small but non-zero cost.
• PyErr_Occurred() would not be fooled by exception raised before. So you can use this code even if an exception is set.

Disadvantages:

• You need to use an int variable.

In some cases, when this check is used in combinations with PyDict_SetItem(), the code can be replaced with PyDict_SetDefault(), which is bot more terse and efficient. And when it is used in combinations with PyDict_DelItem(), the code can be replaced with _PyDict_Pop().

The proposed patch makes the code using PyDict_Contains() and PyDict_SetDefault() if appropriate. All use cases for _PyDict_Pop() were already covered by previous changes.

[serhiy-storchaka]

The code uses also _PyDict_ContainsId() added in bpo-42006 instead of _PyDict_GetItemIdWithError().

Few uses of _PyDict_GetItemStringWithError() are left as is. I do not think it is worth to introduce _PyDict_ContainsString(). They could be rewritten using _PyDict_ContainsId() if we find it worth.

[serhiy-storchaka]

New changeset b510e10 by Serhiy Storchaka in branch 'master':
bpo-42152: Use PyDict_Contains and PyDict_SetDefault if appropriate. (GH-22986)
b510e10

[methane]

May I close this issue?

[serhiy-storchaka]

Sure. Thanks for reminder.