Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bpo-31505: Fix an assertion failure in json, in case _json.make_encoder() received a bad encoder() argument #3643

Merged
merged 5 commits into from Sep 24, 2017
Merged

bpo-31505: Fix an assertion failure in json, in case _json.make_encoder() received a bad encoder() argument #3643

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
a043ba6
init commit
orenmn Sep 18, 2017
fb91a87
fix test name
orenmn Sep 18, 2017
99244b7
improve patch C code
orenmn Sep 18, 2017
da90d1e
various fixes according to Serhiy's advice
orenmn Sep 18, 2017
014db74
remove the redundant import
orenmn Sep 19, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
21 changes: 21 additions & 0 deletions Lib/test/test_json/test_speedups.py
View file
Open in desktop
@@ -1,4 +1,5 @@
from test.test_json import CTest
import test.support


class BadBool:
Expand Down Expand Up @@ -36,6 +37,26 @@ def test_make_encoder(self):
b"\xCD\x7D\x3D\x4E\x12\x4C\xF9\x79\xD7\x52\xBA\x82\xF2\x27\x4A\x7D\xA0\xCA\x75",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't related to this issue, but it seems to me that TypeError is raised for different cause than was originally in this test. This test doesn't work as intended.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIUC, Victor wrote this test to verify that the interpreter doesn't crash (as part of https://bugs.python.org/issue6986, as originally PyArg_ParseTupleAndKeywords() was given actual fields of the new PyEncoderObject, instead of temp vars), so raising a TypeError is the wanted outcome here.
maybe we can just change the name of the test to 'test_issue6986' (as it tests only this specific issue)?
and maybe do the same for test_make_scanner()?
anyway, I guess any such change should be in a PR referring to bpo-6986..

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems your are right. Thank you for a reference.

None)

@test.support.cpython_only
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this test should be passed on other implementations that have compatible c_make_encoder.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i thought about using test.support.get_attribute() to check whether c_make_encoder() is available, but test_make_encoder() just uses it, and would have failed if it hadn't existed.

maybe we can assume that every other C implementations implements test_make_encoder()?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that if other C implementation doesn't implement test_make_encoder() we should get a request for making this test optional.

def test_issue31505(self):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this can be named test_bad_str_encoder.

# There shouldn't be an assertion failure in case c_make_encoder()
# receives a bad encoder() argument.
def _bad_encoder1(*args):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to use starting underscore. This is a local name.

return None
enc = self.json.encoder.c_make_encoder(None, None, _bad_encoder1, None,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use more meaningful arguments. Otherwise TypeError could be raised for different reason (for example if default is not a callable).

c_make_encoder(None, default, _bad_encoder1, None, ': ', ', ', False, False, False)

'foo', 'bar', None, None, None)
with self.assertRaises(TypeError):
enc(obj='spam', _current_indent_level=4)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why keyword arguments are used? In the code an encoder is called with positional arguments.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no reason, my bad.

with self.assertRaises(TypeError):
enc(obj={'spam': 42}, _current_indent_level=4)

def _bad_encoder2(*args):
1/0
enc = self.json.encoder.c_make_encoder(None, None, _bad_encoder2, None,
'foo', 'bar', None, None, None)
with self.assertRaises(ZeroDivisionError):
enc(obj='spam', _current_indent_level=4)

def test_bad_bool_args(self):
def test(name):
self.json.encoder.JSONEncoder(**{name: BadBool()}).encode({'a': 1})
Expand Down
2 changes: 2 additions & 0 deletions Misc/NEWS.d/next/Core and Builtins/2017-09-18-12-07-39.bpo-31505.VomaFa.rst
View file
Open in desktop
@@ -0,0 +1,2 @@
Fix an assertion failure in `json`, in case `_json.make_encoder()` received
a bad `encoder()` argument. Patch by Oren Milman.
13 changes: 11 additions & 2 deletions Modules/_json.c
View file
Open in desktop
Expand Up @@ -1429,10 +1429,19 @@ static PyObject *
encoder_encode_string(PyEncoderObject *s, PyObject *obj)
{
/* Return the JSON representation of a string */
PyObject *encoded;

if (s->fast_encode)
return s->fast_encode(NULL, obj);
else
return PyObject_CallFunctionObjArgs(s->encoder, obj, NULL);
encoded = PyObject_CallFunctionObjArgs(s->encoder, obj, NULL);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add braces around return above.

if (encoded != NULL && !PyUnicode_Check(encoded)) {
PyErr_Format(PyExc_TypeError,
"encoder() must return a string, not %.80s",
Py_TYPE(encoded)->tp_name);
Py_DECREF(encoded);
return NULL;
}
return encoded;
}

static int
Expand Down