Skip to content

Commit

Permalink
Browse files Browse the repository at this point in the history
Reimplement __Pyx_PyDict_GetItemStrWithError() for Py2 to fix a memor…
…y leak (GH-3593)

Reimplement __Pyx_PyDict_GetItemStrWithError() as a hacky version in Py2 to get the semantics right of returning a borrowed reference with non-KeyError exceptions left in place.

Closes #3578
  • Loading branch information
scoder committed May 9, 2020
1 parent ac694af commit 34eafb7
Show file tree
Hide file tree
Showing 2 changed files with 26 additions and 4 deletions.
4 changes: 4 additions & 0 deletions CHANGES.rst
Expand Up @@ -25,6 +25,10 @@ Bugs fixed
could fail to acquire the GIL in some cases on function exit.
(Github issue #3590 etc.)

* A reference leak when processing keyword arguments in Py2 was resolved,
that appeared in 3.0a1.
(Github issue #3578)

* The outdated getbuffer/releasebuffer implementations in the NumPy
declarations were removed so that buffers declared as ``ndarray``
now use the normal implementation in NumPy.
Expand Down
26 changes: 22 additions & 4 deletions Cython/Utility/ModuleSetupCode.c
Expand Up @@ -676,10 +676,28 @@ static CYTHON_INLINE PyObject * __Pyx_PyDict_GetItemStr(PyObject *dict, PyObject
#define __Pyx_PyDict_GetItemStr PyDict_GetItem
#else
static CYTHON_INLINE PyObject * __Pyx_PyDict_GetItemStrWithError(PyObject *dict, PyObject *name) {
PyObject *res = PyObject_GetItem(dict, name);
if (res == NULL && PyErr_ExceptionMatches(PyExc_KeyError))
PyErr_Clear();
return res;
// This is tricky - we should return a borrowed reference but not swallow non-KeyError exceptions. 8-|
// But: this function is only used in Py2 and older PyPys,
// and currently only for argument parsing and other non-correctness-critical lookups
// and we know that 'name' is an interned 'str' with pre-calculated hash value (only comparisons can fail),
// thus, performance matters more than correctness here, especially in the "not found" case.
#if CYTHON_COMPILING_IN_PYPY
// So we ignore any exceptions in old PyPys ...
return PyDict_GetItem(dict, name);
#else
// and hack together a stripped-down and modified PyDict_GetItem() in CPython 2.
PyDictEntry *ep;
PyDictObject *mp = (PyDictObject*) dict;
long hash = ((PyStringObject *) name)->ob_shash;
assert(hash != -1); /* hash values of interned strings are always initialised */
ep = (mp->ma_lookup)(mp, name, hash);
if (ep == NULL) {
// error occurred
return NULL;
}
// found or not found
return ep->me_value;
#endif
}
#define __Pyx_PyDict_GetItemStr PyDict_GetItem
#endif
Expand Down

0 comments on commit 34eafb7

Please sign in to comment.