Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

An additional option to allow the the non-encoding of python tuples #6

Closed
wants to merge 10 commits into from

3 participants

Ed Singleton Bob Ippolito David Baumgold
Ed Singleton

I added an option to the JSONEncoder to allow tuples to not be automatically encoded to arrays, for the use case when you need to preserve the difference between tuples and lists (in this case you can now encode tuples to a custom object using the default() method).

I added a basic test (tests/test_tuple.py) and checked that all the other tests ran.

Unfortunately I couldn't add it into the _speedups.c, so I addd a check to iterencode that if this option has been changed, it shouldn't use the speedups.

Yours

Ed

Bob Ippolito
Owner

This looks like a good idea, but I think I will do the implementation differently. Also, unittest.skipUnless wasn't added until Python 2.7 so you can't skip the test that way.

Bob Ippolito
Owner

Actually, I don't think that being able to remove tuple from the array encoding is very useful... but encoding namedtuple as an object seems like the right thing to do. To support people who depended on the old behavior, there should also be a flag to turn this off.

Bob Ippolito etrepum closed this September 04, 2011
Jonathan Perkin jperkin referenced this pull request from a commit in joyent/pkgsrc May 29, 2012
Update to 2.2.1:
Version 2.2.1 released 2011-09-06

* Fix MANIFEST.in issue when building a sdist from a sdist.
  simplejson/simplejson#16

Version 2.2.0 released 2011-09-04

* Remove setuptools requirement, reverted to pure distutils
* use_decimal default for encoding (dump, dumps, JSONEncoder) is now True
* tuple encoding as JSON objects can be turned off with new
  tuple_as_array=False option.
  simplejson/simplejson#6
* namedtuple (or other tuple subclasses with _asdict methods) are now
  encoded as JSON objects rather than arrays by default. Can be disabled
  and treated as a tuple with the new namedtuple_as_object=False option.
  simplejson/simplejson#6
* JSONDecodeError is now raised instead of ValueError when a document
  ends with an opening quote and the C speedups are in use.
  simplejson/simplejson#15
* Updated documentation with information about JSONDecodeError
* Force unicode linebreak characters to be escaped (U+2028 and U+2029)
  http://timelessrepo.com/json-isnt-a-javascript-subset
* Moved documentation from a git submodule to
  http://simplejson.readthedocs.org/

Version 2.1.6 released 2011-05-08

* Prevent segfaults with deeply nested JSON documents
  simplejson/simplejson#11
* Fix compatibility with Python 2.5
  simplejson/simplejson#5

Version 2.1.5 released 2011-04-17

* Built sdist tarball with setuptools_git installed. Argh.

Version 2.1.4 released 2011-04-17

* Does not try to build the extension when using PyPy
* Trailing whitespace after commas no longer emitted when indent is used
* Migrated to github http://github.com/simplejson/simplejson
15a9f54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
This page is out of date. Refresh to see the latest.
1  CHANGES.txt
... ...
@@ -1,6 +1,7 @@
1 1
 Version 2.1.4 released XXXX-XX-XX
2 2
 
3 3
 * Migrated to github http://github.com/simplejson/simplejson
  4
+* namedtuples now serialize to objects, rather than arrays
4 5
 
5 6
 Version 2.1.3 released 2011-01-17
6 7
 
22  simplejson/encoder.py
@@ -78,7 +78,7 @@ class JSONEncoder(object):
78 78
     +-------------------+---------------+
79 79
     | Python            | JSON          |
80 80
     +===================+===============+
81  
-    | dict              | object        |
  81
+    | dict, namedtuple  | object        |
82 82
     +-------------------+---------------+
83 83
     | list, tuple       | array         |
84 84
     +-------------------+---------------+
@@ -101,6 +101,7 @@ class JSONEncoder(object):
101 101
     """
102 102
     item_separator = ', '
103 103
     key_separator = ': '
  104
+    array_types = (list, tuple)
104 105
     def __init__(self, skipkeys=False, ensure_ascii=True,
105 106
             check_circular=True, allow_nan=True, sort_keys=False,
106 107
             indent=None, separators=None, encoding='utf-8', default=None,
@@ -268,7 +269,7 @@ def floatstr(o, allow_nan=self.allow_nan,
268 269
 
269 270
         key_memo = {}
270 271
         if (_one_shot and c_make_encoder is not None
271  
-                and self.indent is None):
  272
+                and self.indent is None and self.array_types==(list, tuple)):
272 273
             _iterencode = c_make_encoder(
273 274
                 markers, self.default, _encoder, self.indent,
274 275
                 self.key_separator, self.item_separator, self.sort_keys,
@@ -277,7 +278,7 @@ def floatstr(o, allow_nan=self.allow_nan,
277 278
             _iterencode = _make_iterencode(
278 279
                 markers, self.default, _encoder, self.indent, floatstr,
279 280
                 self.key_separator, self.item_separator, self.sort_keys,
280  
-                self.skipkeys, _one_shot, self.use_decimal)
  281
+                self.skipkeys, _one_shot, self.use_decimal, self.array_types)
281 282
         try:
282 283
             return _iterencode(o, 0)
283 284
         finally:
@@ -313,7 +314,7 @@ def iterencode(self, o, _one_shot=False):
313 314
 
314 315
 def _make_iterencode(markers, _default, _encoder, _indent, _floatstr,
315 316
         _key_separator, _item_separator, _sort_keys, _skipkeys, _one_shot,
316  
-        _use_decimal,
  317
+        _use_decimal, _array_types,
317 318
         ## HACK: hand-optimized bytecode; turn globals into locals
318 319
         False=False,
319 320
         True=True,
@@ -371,7 +372,9 @@ def _iterencode_list(lst, _current_indent_level):
371 372
                 yield buf + str(value)
372 373
             else:
373 374
                 yield buf
374  
-                if isinstance(value, (list, tuple)):
  375
+                if isinstance(value, tuple) and hasattr(value, '_asdict'):
  376
+                    chunks = _iterencode_dict(value._asdict(), _current_indent_level)
  377
+                elif isinstance(value, _array_types):
375 378
                     chunks = _iterencode_list(value, _current_indent_level)
376 379
                 elif isinstance(value, dict):
377 380
                     chunks = _iterencode_dict(value, _current_indent_level)
@@ -450,7 +453,9 @@ def _iterencode_dict(dct, _current_indent_level):
450 453
             elif _use_decimal and isinstance(value, Decimal):
451 454
                 yield str(value)
452 455
             else:
453  
-                if isinstance(value, (list, tuple)):
  456
+                if isinstance(value, tuple) and hasattr(value, '_asdict'):
  457
+                    chunks = _iterencode_dict(value._asdict(), _current_indent_level)
  458
+                elif isinstance(value, _array_types):
454 459
                     chunks = _iterencode_list(value, _current_indent_level)
455 460
                 elif isinstance(value, dict):
456 461
                     chunks = _iterencode_dict(value, _current_indent_level)
@@ -478,7 +483,10 @@ def _iterencode(o, _current_indent_level):
478 483
             yield str(o)
479 484
         elif isinstance(o, float):
480 485
             yield _floatstr(o)
481  
-        elif isinstance(o, (list, tuple)):
  486
+        elif isinstance(o, tuple) and hasattr(o, '_asdict'):
  487
+            for chunk in _iterencode_dict(o._asdict(), _current_indent_level):
  488
+                yield chunk
  489
+        elif isinstance(o, _array_types):
482 490
             for chunk in _iterencode_list(o, _current_indent_level):
483 491
                 yield chunk
484 492
         elif isinstance(o, dict):
2  simplejson/tests/__init__.py
@@ -46,6 +46,8 @@ def all_tests_suite():
46 46
         'simplejson.tests.test_speedups',
47 47
         'simplejson.tests.test_unicode',
48 48
         'simplejson.tests.test_decimal',
  49
+        'simplejson.tests.test_tuples',
  50
+        'simplejson.tests.test_namedtuples',
49 51
     ])
50 52
     suite = additional_tests(suite)
51 53
     return OptionalExtensionTestSuite([suite])
43  simplejson/tests/test_namedtuples.py
... ...
@@ -0,0 +1,43 @@
  1
+import unittest
  2
+import simplejson
  3
+try:
  4
+    from collections import namedtuple
  5
+except ImportError:
  6
+    namedtuple = None
  7
+
  8
+@unittest.skipUnless(namedtuple, "namedtuple tests can only be run if the namedtuple function is available")
  9
+class TestNamedTuples(unittest.TestCase):
  10
+    def test_namedtuple(self):
  11
+        Point = namedtuple("Point", ['x', 'y'])
  12
+        point_dict = {'x': 5, 'y': 7}
  13
+        point_tuple = Point(**point_dict)
  14
+
  15
+        # It would be nice if we could just compare the results of simplejson.dumps
  16
+        # for these two objects, but since dicts do not have a defined ordering,
  17
+        # they could come out different but equally valid. Instead, we'll manually
  18
+        # define all possibile reorderings and test to be sure that the result is
  19
+        # in that list.
  20
+        possibilities = ['{"y": 7, "x": 5}', '{"x": 5, "y": 7}']
  21
+
  22
+        # quick sanity check to be sure that simplejson.dumps is behaving as we expect
  23
+        dict_json = simplejson.dumps(point_dict)
  24
+        self.assertIn(dict_json, possibilities, "sanity check failed")
  25
+
  26
+        # and now, the actual test
  27
+        tuple_json = simplejson.dumps(point_tuple)
  28
+        self.assertIn(tuple_json, possibilities, 
  29
+                "namedtuple JSON does not match dict JSON")
  30
+
  31
+    def test_nested_namedtuple(self):
  32
+        Point = namedtuple("Point", ['x', 'y'])
  33
+        point = Point(1, 2)
  34
+
  35
+        dictnested = {'outer': point}
  36
+        possibilities = ['{"outer": {"x": 1, "y": 2}}', '{"outer": {"y": 2, "x": 1}}']
  37
+        self.assertIn(simplejson.dumps(dictnested), possibilities,
  38
+                "namedtuple nested in dict serialized incorrectly")
  39
+
  40
+        listnested = [1, 2, 3, point]
  41
+        possibilities = ['[1, 2, 3, {"x": 1, "y": 2}]', '[1, 2, 3, {"y": 2, "x": 1}}']
  42
+        self.assertIn(simplejson.dumps(listnested), possibilities,
  43
+                "namedtuple nested in list serialized incorrectly")
35  simplejson/tests/test_tuples.py
... ...
@@ -0,0 +1,35 @@
  1
+import unittest
  2
+
  3
+import simplejson
  4
+
  5
+class TestCustomTuples(unittest.TestCase):
  6
+    def test_custom_tuple_failure(self):
  7
+        "Test that removing tuple from array_types causes encoding a tuple to fail"
  8
+        class CustomEncoder(simplejson.JSONEncoder):
  9
+            array_types = (list,)
  10
+
  11
+        assert CustomEncoder().array_types == (list,)
  12
+
  13
+        try:
  14
+            result = simplejson.dumps((1,2,3), cls=CustomEncoder)
  15
+            print repr(result)
  16
+        except TypeError:
  17
+            pass
  18
+        else:
  19
+            self.fail("Unexpected success when parsing a tuple")
  20
+
  21
+    def test_custom_tuple_success(self):
  22
+        "Test that you can custom encode a tuple"
  23
+
  24
+        class CustomEncoder(simplejson.JSONEncoder):
  25
+            array_types = (list,)
  26
+
  27
+            def default(self, o):
  28
+                if isinstance(o, tuple):
  29
+                    return dict(__class__='tuple', __data__=list(o))
  30
+                else:
  31
+                    return simplejson.JSONEncoder.default(self, o)
  32
+
  33
+        result = simplejson.dumps((1,2,3), cls=CustomEncoder)
  34
+        expected = simplejson.dumps(dict(__class__='tuple', __data__=[1,2,3]))
  35
+        self.assertEqual(result, expected)
Commit_comment_tip

Tip: You can add notes to lines in a file. Hover to the left of a line to make a note

Something went wrong with that request. Please try again.