Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

An additional option to allow the the non-encoding of python tuples #6

Closed
wants to merge 10 commits into from

3 participants

@Singletoned

I added an option to the JSONEncoder to allow tuples to not be automatically encoded to arrays, for the use case when you need to preserve the difference between tuples and lists (in this case you can now encode tuples to a custom object using the default() method).

I added a basic test (tests/test_tuple.py) and checked that all the other tests ran.

Unfortunately I couldn't add it into the _speedups.c, so I addd a check to iterencode that if this option has been changed, it shouldn't use the speedups.

Yours

Ed

@etrepum
Owner

This looks like a good idea, but I think I will do the implementation differently. Also, unittest.skipUnless wasn't added until Python 2.7 so you can't skip the test that way.

@etrepum
Owner

Actually, I don't think that being able to remove tuple from the array encoding is very useful... but encoding namedtuple as an object seems like the right thing to do. To support people who depended on the old behavior, there should also be a flag to turn this off.

@etrepum
Owner
@etrepum etrepum closed this
@jperkin jperkin referenced this pull request from a commit in joyent/pkgsrc
wiz Update to 2.2.1:
Version 2.2.1 released 2011-09-06

* Fix MANIFEST.in issue when building a sdist from a sdist.
  simplejson/simplejson#16

Version 2.2.0 released 2011-09-04

* Remove setuptools requirement, reverted to pure distutils
* use_decimal default for encoding (dump, dumps, JSONEncoder) is now True
* tuple encoding as JSON objects can be turned off with new
  tuple_as_array=False option.
  simplejson/simplejson#6
* namedtuple (or other tuple subclasses with _asdict methods) are now
  encoded as JSON objects rather than arrays by default. Can be disabled
  and treated as a tuple with the new namedtuple_as_object=False option.
  simplejson/simplejson#6
* JSONDecodeError is now raised instead of ValueError when a document
  ends with an opening quote and the C speedups are in use.
  simplejson/simplejson#15
* Updated documentation with information about JSONDecodeError
* Force unicode linebreak characters to be escaped (U+2028 and U+2029)
  http://timelessrepo.com/json-isnt-a-javascript-subset
* Moved documentation from a git submodule to
  http://simplejson.readthedocs.org/

Version 2.1.6 released 2011-05-08

* Prevent segfaults with deeply nested JSON documents
  simplejson/simplejson#11
* Fix compatibility with Python 2.5
  simplejson/simplejson#5

Version 2.1.5 released 2011-04-17

* Built sdist tarball with setuptools_git installed. Argh.

Version 2.1.4 released 2011-04-17

* Does not try to build the extension when using PyPy
* Trailing whitespace after commas no longer emitted when indent is used
* Migrated to github http://github.com/simplejson/simplejson
15a9f54
@jsonn jsonn referenced this pull request from a commit in jsonn/pkgsrc
wiz Update to 2.2.1:
Version 2.2.1 released 2011-09-06

* Fix MANIFEST.in issue when building a sdist from a sdist.
  simplejson/simplejson#16

Version 2.2.0 released 2011-09-04

* Remove setuptools requirement, reverted to pure distutils
* use_decimal default for encoding (dump, dumps, JSONEncoder) is now True
* tuple encoding as JSON objects can be turned off with new
  tuple_as_array=False option.
  simplejson/simplejson#6
* namedtuple (or other tuple subclasses with _asdict methods) are now
  encoded as JSON objects rather than arrays by default. Can be disabled
  and treated as a tuple with the new namedtuple_as_object=False option.
  simplejson/simplejson#6
* JSONDecodeError is now raised instead of ValueError when a document
  ends with an opening quote and the C speedups are in use.
  simplejson/simplejson#15
* Updated documentation with information about JSONDecodeError
* Force unicode linebreak characters to be escaped (U+2028 and U+2029)
  http://timelessrepo.com/json-isnt-a-javascript-subset
* Moved documentation from a git submodule to
  http://simplejson.readthedocs.org/

Version 2.1.6 released 2011-05-08

* Prevent segfaults with deeply nested JSON documents
  simplejson/simplejson#11
* Fix compatibility with Python 2.5
  simplejson/simplejson#5

Version 2.1.5 released 2011-04-17

* Built sdist tarball with setuptools_git installed. Argh.

Version 2.1.4 released 2011-04-17

* Does not try to build the extension when using PyPy
* Trailing whitespace after commas no longer emitted when indent is used
* Migrated to github http://github.com/simplejson/simplejson
ad9e762
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
This page is out of date. Refresh to see the latest.
View
1  CHANGES.txt
@@ -1,6 +1,7 @@
Version 2.1.4 released XXXX-XX-XX
* Migrated to github http://github.com/simplejson/simplejson
+* namedtuples now serialize to objects, rather than arrays
Version 2.1.3 released 2011-01-17
View
22 simplejson/encoder.py
@@ -78,7 +78,7 @@ class JSONEncoder(object):
+-------------------+---------------+
| Python | JSON |
+===================+===============+
- | dict | object |
+ | dict, namedtuple | object |
+-------------------+---------------+
| list, tuple | array |
+-------------------+---------------+
@@ -101,6 +101,7 @@ class JSONEncoder(object):
"""
item_separator = ', '
key_separator = ': '
+ array_types = (list, tuple)
def __init__(self, skipkeys=False, ensure_ascii=True,
check_circular=True, allow_nan=True, sort_keys=False,
indent=None, separators=None, encoding='utf-8', default=None,
@@ -268,7 +269,7 @@ def floatstr(o, allow_nan=self.allow_nan,
key_memo = {}
if (_one_shot and c_make_encoder is not None
- and self.indent is None):
+ and self.indent is None and self.array_types==(list, tuple)):
_iterencode = c_make_encoder(
markers, self.default, _encoder, self.indent,
self.key_separator, self.item_separator, self.sort_keys,
@@ -277,7 +278,7 @@ def floatstr(o, allow_nan=self.allow_nan,
_iterencode = _make_iterencode(
markers, self.default, _encoder, self.indent, floatstr,
self.key_separator, self.item_separator, self.sort_keys,
- self.skipkeys, _one_shot, self.use_decimal)
+ self.skipkeys, _one_shot, self.use_decimal, self.array_types)
try:
return _iterencode(o, 0)
finally:
@@ -313,7 +314,7 @@ def iterencode(self, o, _one_shot=False):
def _make_iterencode(markers, _default, _encoder, _indent, _floatstr,
_key_separator, _item_separator, _sort_keys, _skipkeys, _one_shot,
- _use_decimal,
+ _use_decimal, _array_types,
## HACK: hand-optimized bytecode; turn globals into locals
False=False,
True=True,
@@ -371,7 +372,9 @@ def _iterencode_list(lst, _current_indent_level):
yield buf + str(value)
else:
yield buf
- if isinstance(value, (list, tuple)):
+ if isinstance(value, tuple) and hasattr(value, '_asdict'):
+ chunks = _iterencode_dict(value._asdict(), _current_indent_level)
+ elif isinstance(value, _array_types):
chunks = _iterencode_list(value, _current_indent_level)
elif isinstance(value, dict):
chunks = _iterencode_dict(value, _current_indent_level)
@@ -450,7 +453,9 @@ def _iterencode_dict(dct, _current_indent_level):
elif _use_decimal and isinstance(value, Decimal):
yield str(value)
else:
- if isinstance(value, (list, tuple)):
+ if isinstance(value, tuple) and hasattr(value, '_asdict'):
+ chunks = _iterencode_dict(value._asdict(), _current_indent_level)
+ elif isinstance(value, _array_types):
chunks = _iterencode_list(value, _current_indent_level)
elif isinstance(value, dict):
chunks = _iterencode_dict(value, _current_indent_level)
@@ -478,7 +483,10 @@ def _iterencode(o, _current_indent_level):
yield str(o)
elif isinstance(o, float):
yield _floatstr(o)
- elif isinstance(o, (list, tuple)):
+ elif isinstance(o, tuple) and hasattr(o, '_asdict'):
+ for chunk in _iterencode_dict(o._asdict(), _current_indent_level):
+ yield chunk
+ elif isinstance(o, _array_types):
for chunk in _iterencode_list(o, _current_indent_level):
yield chunk
elif isinstance(o, dict):
View
2  simplejson/tests/__init__.py
@@ -46,6 +46,8 @@ def all_tests_suite():
'simplejson.tests.test_speedups',
'simplejson.tests.test_unicode',
'simplejson.tests.test_decimal',
+ 'simplejson.tests.test_tuples',
+ 'simplejson.tests.test_namedtuples',
])
suite = additional_tests(suite)
return OptionalExtensionTestSuite([suite])
View
43 simplejson/tests/test_namedtuples.py
@@ -0,0 +1,43 @@
+import unittest
+import simplejson
+try:
+ from collections import namedtuple
+except ImportError:
+ namedtuple = None
+
+@unittest.skipUnless(namedtuple, "namedtuple tests can only be run if the namedtuple function is available")
+class TestNamedTuples(unittest.TestCase):
+ def test_namedtuple(self):
+ Point = namedtuple("Point", ['x', 'y'])
+ point_dict = {'x': 5, 'y': 7}
+ point_tuple = Point(**point_dict)
+
+ # It would be nice if we could just compare the results of simplejson.dumps
+ # for these two objects, but since dicts do not have a defined ordering,
+ # they could come out different but equally valid. Instead, we'll manually
+ # define all possibile reorderings and test to be sure that the result is
+ # in that list.
+ possibilities = ['{"y": 7, "x": 5}', '{"x": 5, "y": 7}']
+
+ # quick sanity check to be sure that simplejson.dumps is behaving as we expect
+ dict_json = simplejson.dumps(point_dict)
+ self.assertIn(dict_json, possibilities, "sanity check failed")
+
+ # and now, the actual test
+ tuple_json = simplejson.dumps(point_tuple)
+ self.assertIn(tuple_json, possibilities,
+ "namedtuple JSON does not match dict JSON")
+
+ def test_nested_namedtuple(self):
+ Point = namedtuple("Point", ['x', 'y'])
+ point = Point(1, 2)
+
+ dictnested = {'outer': point}
+ possibilities = ['{"outer": {"x": 1, "y": 2}}', '{"outer": {"y": 2, "x": 1}}']
+ self.assertIn(simplejson.dumps(dictnested), possibilities,
+ "namedtuple nested in dict serialized incorrectly")
+
+ listnested = [1, 2, 3, point]
+ possibilities = ['[1, 2, 3, {"x": 1, "y": 2}]', '[1, 2, 3, {"y": 2, "x": 1}}']
+ self.assertIn(simplejson.dumps(listnested), possibilities,
+ "namedtuple nested in list serialized incorrectly")
View
35 simplejson/tests/test_tuples.py
@@ -0,0 +1,35 @@
+import unittest
+
+import simplejson
+
+class TestCustomTuples(unittest.TestCase):
+ def test_custom_tuple_failure(self):
+ "Test that removing tuple from array_types causes encoding a tuple to fail"
+ class CustomEncoder(simplejson.JSONEncoder):
+ array_types = (list,)
+
+ assert CustomEncoder().array_types == (list,)
+
+ try:
+ result = simplejson.dumps((1,2,3), cls=CustomEncoder)
+ print repr(result)
+ except TypeError:
+ pass
+ else:
+ self.fail("Unexpected success when parsing a tuple")
+
+ def test_custom_tuple_success(self):
+ "Test that you can custom encode a tuple"
+
+ class CustomEncoder(simplejson.JSONEncoder):
+ array_types = (list,)
+
+ def default(self, o):
+ if isinstance(o, tuple):
+ return dict(__class__='tuple', __data__=list(o))
+ else:
+ return simplejson.JSONEncoder.default(self, o)
+
+ result = simplejson.dumps((1,2,3), cls=CustomEncoder)
+ expected = simplejson.dumps(dict(__class__='tuple', __data__=[1,2,3]))
+ self.assertEqual(result, expected)
Something went wrong with that request. Please try again.