New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for preprocessed JSON strings (with optimizations) in encoder #143

Merged
merged 1 commit into from Oct 28, 2016

Conversation

Projects
None yet
3 participants
@lamflam
Contributor

lamflam commented Sep 1, 2016

Fixes #125

In some situations, you may have a large python dictionary you need to JSONify but one of the values inside the dict is already a JSON string. This is common when pulling an object from a database, for example, where one of the fields is a JSON blob/string. Previously you would have to deserialize and then reserialize that string just to serialize the high level object, but obviously this is unnecessarily slow. This changes adds a method/type that can be used to wrap a str and tell the serializer to just pass it through instead.

Original attempts were #141 and #142 but this is even cleaner.

Add support for preprocessed JSON strings (with optimizations) in enc…
…oder

In some situations, you may have a large python dictionary you need to JSONify but one of the values inside the dict is already a JSON string. This is common when pulling an object from a database, for example, where one of the fields is a JSON blob/string. Previously you would have to deserialize and then reserialize that string just to serialize the high level object, but obviously this is unnecessarily slow. This changes adds a method/type that can be used to wrap a str and tell the serializer to just pass it through instead.
@lamflam

This comment has been minimized.

Show comment
Hide comment
@lamflam

lamflam Sep 1, 2016

Contributor

@etrepum I believe this is a little closer to the implementation you suggested, but let me know if there are any other changes you'd like to see.

Contributor

lamflam commented Sep 1, 2016

@etrepum I believe this is a little closer to the implementation you suggested, but let me know if there are any other changes you'd like to see.

@etrepum

Should also merge or rebase with master to fix the travis build

else if (is_raw_json(obj))
{
PyObject *encoded = PyObject_GetAttrString(obj, "encoded_json");
if (encoded != NULL)

This comment has been minimized.

@etrepum

etrepum Oct 24, 2016

Member

In the other cases when there is a failure break is used, harder to track what's going to happen to this failure if it just falls through.

@etrepum

etrepum Oct 24, 2016

Member

In the other cases when there is a failure break is used, harder to track what's going to happen to this failure if it just falls through.

This comment has been minimized.

@etrepum

etrepum Oct 28, 2016

Member

Under closer scrutiny this looks fine as-is.

@etrepum

etrepum Oct 28, 2016

Member

Under closer scrutiny this looks fine as-is.

@@ -624,6 +632,8 @@ def _iterencode(o, _current_indent_level):
if (isinstance(o, string_types) or
(_PY3 and isinstance(o, binary_type))):
yield _encoder(o)
elif isinstance(o, RawJSON):
buf + o.encoded_json

This comment has been minimized.

@etrepum

etrepum Oct 24, 2016

Member
yield o.encoded_json
@etrepum

etrepum Oct 24, 2016

Member
yield o.encoded_json
'key2': 'value2',
'd1': json.RawJSON(json.dumps(dct1))
}

This comment has been minimized.

@etrepum

etrepum Oct 24, 2016

Member

The other important cases to test are to serialize it by itself, and to in a list.

@etrepum

etrepum Oct 24, 2016

Member

The other important cases to test are to serialize it by itself, and to in a list.

@etrepum etrepum merged commit da17e35 into simplejson:master Oct 28, 2016

1 of 2 checks passed

continuous-integration/travis-ci/pr The Travis CI build could not complete due to an error
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Nov 1, 2016

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

@pyup-bot pyup-bot referenced this pull request Nov 1, 2016

Merged

Update simplejson to 3.10.0 #26

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Nov 2, 2016

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Nov 3, 2016

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

@pyup-bot pyup-bot referenced this pull request Nov 30, 2016

Merged

Initial Update #82

@pyup-bot pyup-bot referenced this pull request Jan 6, 2017

Open

Initial Update #6

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jan 11, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jan 11, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jan 12, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jan 12, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jan 13, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jan 13, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jan 13, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jan 14, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jan 17, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jan 17, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jan 18, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jan 19, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jan 19, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jan 19, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jan 19, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jan 20, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jan 20, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jan 21, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jan 24, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jan 24, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jan 25, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jan 25, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jan 26, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jun 13, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jun 14, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jun 14, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jun 14, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jun 15, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jun 15, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jun 16, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jun 16, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jun 19, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jun 20, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jun 20, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jun 21, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jun 22, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jun 22, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jun 23, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jun 24, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jun 26, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jun 27, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jun 27, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jun 28, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jun 28, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jun 29, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jun 30, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jul 4, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jul 4, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jul 5, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jul 5, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jul 6, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jul 6, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

clrpackages pushed a commit to clearlinux-pkgs/simplejson that referenced this pull request Jul 15, 2017

simplejson: Autospec creation for update from version 3.9.0 to versio…
…n 3.10.0

Version 3.10.0 released 2016-10-28

* Add RawJSON class to allow a faster path for already encoded JSON.
  simplejson/simplejson#143

@pyup-bot pyup-bot referenced this pull request Dec 23, 2017

Open

Initial Update #1

@pyup-bot pyup-bot referenced this pull request Jan 21, 2018

Open

Initial Update #1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment