Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for preprocessed JSON strings (with optimizations) in encoder #143

Merged
merged 1 commit into from Oct 28, 2016

Conversation

@lamflam
Copy link
Contributor

@lamflam lamflam commented Sep 1, 2016

Fixes #125

In some situations, you may have a large python dictionary you need to JSONify but one of the values inside the dict is already a JSON string. This is common when pulling an object from a database, for example, where one of the fields is a JSON blob/string. Previously you would have to deserialize and then reserialize that string just to serialize the high level object, but obviously this is unnecessarily slow. This changes adds a method/type that can be used to wrap a str and tell the serializer to just pass it through instead.

Original attempts were #141 and #142 but this is even cleaner.

…oder

In some situations, you may have a large python dictionary you need to JSONify but one of the values inside the dict is already a JSON string. This is common when pulling an object from a database, for example, where one of the fields is a JSON blob/string. Previously you would have to deserialize and then reserialize that string just to serialize the high level object, but obviously this is unnecessarily slow. This changes adds a method/type that can be used to wrap a str and tell the serializer to just pass it through instead.
@lamflam
Copy link
Contributor Author

@lamflam lamflam commented Sep 1, 2016

@etrepum I believe this is a little closer to the implementation you suggested, but let me know if there are any other changes you'd like to see.

Copy link
Member

@etrepum etrepum left a comment

Should also merge or rebase with master to fix the travis build

else if (is_raw_json(obj))
{
PyObject *encoded = PyObject_GetAttrString(obj, "encoded_json");
if (encoded != NULL)

This comment has been minimized.

@etrepum

etrepum Oct 24, 2016
Member

In the other cases when there is a failure break is used, harder to track what's going to happen to this failure if it just falls through.

This comment has been minimized.

@etrepum

etrepum Oct 28, 2016
Member

Under closer scrutiny this looks fine as-is.

@@ -624,6 +632,8 @@ def _iterencode(o, _current_indent_level):
if (isinstance(o, string_types) or
(_PY3 and isinstance(o, binary_type))):
yield _encoder(o)
elif isinstance(o, RawJSON):
buf + o.encoded_json

This comment has been minimized.

@etrepum

etrepum Oct 24, 2016
Member

yield o.encoded_json
'key2': 'value2',
'd1': json.RawJSON(json.dumps(dct1))
}

This comment has been minimized.

@etrepum

etrepum Oct 24, 2016
Member

The other important cases to test are to serialize it by itself, and to in a list.

@etrepum etrepum merged commit da17e35 into simplejson:master Oct 28, 2016
1 of 2 checks passed
1 of 2 checks passed
continuous-integration/travis-ci/pr The Travis CI build could not complete due to an error
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Nov 1, 2016
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143
clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Nov 2, 2016
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143
clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Nov 3, 2016
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143
clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jan 11, 2017
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143
clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jan 11, 2017
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143
clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jan 12, 2017
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143
clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jan 12, 2017
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143
clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jan 13, 2017
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143
clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jan 13, 2017
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143
clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jan 13, 2017
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143
clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jan 14, 2017
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143
clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jan 17, 2017
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143
clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jan 17, 2017
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143
clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jan 18, 2017
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143
clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jan 19, 2017
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143
clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jan 19, 2017
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143
clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jan 19, 2017
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143
clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jun 21, 2017
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143
clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jun 22, 2017
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143
clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jun 22, 2017
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143
clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jun 23, 2017
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143
clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jun 24, 2017
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143
clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jun 26, 2017
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143
clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jun 27, 2017
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143
clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jun 27, 2017
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143
clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jun 28, 2017
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143
clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jun 28, 2017
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143
clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jun 29, 2017
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143
clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jun 30, 2017
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143
clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jul 4, 2017
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143
clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jul 4, 2017
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143
clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jul 5, 2017
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143
clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jul 5, 2017
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143
clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jul 6, 2017
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143
clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jul 6, 2017
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143
clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jul 15, 2017
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants
You can’t perform that action at this time.