Skip to content


Subversion checkout URL

You can clone with
Download ZIP


Make ElasticSearch.json_encoder private or something #42

erikrose opened this Issue · 4 comments

1 participant


I'm a little flummoxed at what the extensibility story is supposed to be for JSON encoding. It looks like there are 2 equivalent hook points:

  • Subclassing ElasticSearch and overriding from_python
  • Sticking an alternative encoder class into ElasticSearch.json_encoder

Have 1.


We need to get this figured out by 1.0 so we don't start exposing a public API we'd later have to retract.


@jezdez I'm curious: why does django-haystack call from_python on everything as it builds the query, rather than letting (for instance) search() do the encoding implicitly? Is it just to fail sooner and provide an easier debugging experience?


@jezdez If you get a chance, I could use your help on erikrose@307271b.

Specifically, it looks like _encode_json and from_python are similar in spirit—they both purport to encode Python to JSON—and yet subtly different in that from_python doesn't quote its strings or change True to "true". (I've left failing tests in my patch unchanged to illustrate this.) Is there a strangeness in haystack we should fix, as per my earlier comment?

More broadly, I'm not sure what to_python and from_python are for, and I'd like to see them go away. It seems to me that any JSON coming back from from ES should be converted to Python types to the fullest extent possible: if to_python knows tricks _decode_response doesn't, we should improve _decode_response. Haystack shouldn't have to call the conversion stuff explicitly at all.

Of course, it's entirely possible I misunderstand something about haystack, not having read the whole thing. I await your response. :-)


Hey @jezdez, are you at PyCon? We've got a pretty good sprint going here. I'd really like to remove those two methods and would value your weighing in.

@erikrose erikrose was assigned
@erikrose erikrose closed this issue from a commit
@erikrose erikrose Make JSON encoder extension point clear, and call super() properly fr…
…om decoder. Remove from_python(). Fix #42, #44.

We move JsonEncoder out to the module level and transplant the guts of from_python into JsonEncoder.default(). This makes it possible to call super() from the decoder, which would be very awkward from from_python().

The way to make the decoder aware of new types is now to subclass our JsonEncoder and stick your new class into an ElasticSearch object's json_encoder attr.

We take advantage of the new design to call super() from the decoder, which leads us raise a TypeError on unexpected input, as is usual with the json module, rather than a ValueError or other error.
@erikrose erikrose closed this in c89ad05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.