Allow encoding of already encoded JSON strings.. #96

Open
pkieltyka opened this Issue Feb 14, 2012 · 6 comments

Projects

None yet

3 participants

@pkieltyka

Imagine you have a string, that already has JSON.. but you'd like to wrap that JSON object in an array with others. You may have gotten that JSON object from a cache, or maybe its a Javascript function in a string (like a Handlebars pre-compiled template). It would be nice to effectively "join" JSON objects during encoding. This is a nice optimization, and also convenient. Below is an example and idea on how to implement this.

# One idea to implement this is....

# Get an array of json-encoded strings with ids 1, 2, 3, and 4
x = JsonCache.get([1,2,3,4]).map {|o| Yajl::JsonString.new(o) }

# x[0] = "{ id: 1, name: John}"
# x[1] = "{ id: 2, name: Peter}"
# x[2] = "{ id: 3, name: Mike}"
# x[3] = "{ id: 4, name: Allan}"

json = Yajl::Encoder.encode(x) 

# json should look like:
# "[{ id: 1, name: John}, { id: 2, name: Peter}, { id: 3, name: Mike}, { id: 4, name: Allan}]"

# currently Yajl will encode each string and instead will return:
# "[\"{ id: 1, name: John}\", \"{ id: 2, name: Peter}\", \"{ id: 3, name: Mike}\", \"{ id: 4, name: Allan}\"]"

Of course you could just decode each object in x, put it in an array, and then re-encode. Fine. But thats unnecessary, and here's a better example that can't be achieved unless the encoder understands strings as native JSON types to just concatenate them to the result.

# Pseudo class..
compiled_template = HandlebarsCompiler.compile("hi {{var}}")

# x
# => "function (Handlebars,depth0,helpers,partials,data) {\n  helpers = helpers || Handlebars.helpers;\n  var buffer = \"\", stack1, foundHelper, self=this, functionType=\"function\", helperMissing=helpers.helperMissing, undef=void 0, escapeExpression=this.escapeExpression;\n\n\n  buffer += \"hi \";\n  foundHelper = helpers['var'];\n  stack1 = foundHelper || depth0['var'];\n  if(typeof stack1 === functionType) { stack1 = stack1.call(depth0, { hash: {} }); }\n  else if(stack1=== undef) { stack1 = helperMissing.call(depth0, \"var\", { hash: {} }); }\n  buffer += escapeExpression(stack1);\n  return buffer;}"

template = {
  :name => Yajl::JsonString.new(compiled_template)
}

puts Yajl::Encoder.encode(template)

# The idea is the encoder would return: "{ name: function (Handlebars,depth0,helpers,partials,data) { ..etc. etc. } }"

Now, when parsed by the browser, it doesn't have the eval() the function.

I just came up with the Yajl::JsonString class, which would inherit from a String, and add a method called "is_json" set to true. This way when the encoder is traversing strings, it can test for is_json, and just concatenate instead of encode.

What do you think?

@brianmario
Owner

Interesting idea, I'll see if I can explore this a little more in my yajl-ruby 2.0 stuff (there's a branch going already). For now, something like this should work in place of Yajl::JsonString in your example:

class JsonWrapper
  def initialize(json_string)
    @json_string = json_string
  end

  # yajl-ruby will check if this method exists and call it if so
  # then append the return value directly onto the output buffer as-is
  # this means that this method is assumed to be returning valid JSON
  def to_json
    @json_string
  end
end

But, let me know if otherwise ;)

@pkieltyka

Thanks. The JsonWrapper worked. I tried to write a JsonString class using your example as so:

class JsonString < String
  def initialize(json_string)
    @json_string = json_string
    super(json_string)
  end

  def to_json
    @json_string
  end
end

However, this didn't seem to work, the encoder still encoded the string. How come? I see in yajl_ext.c that a to_json method is being defined for Strings.. and interestingly, the to_json method in JsonString isn't being called.. it's calling the to_json in String.

Either way, the JsonWrapper will get me by, thanks!

@brianmario
Owner

Ah - this is because yajl-ruby will (for the sake of efficiency) handle encoding anything that directly translates to a native JSON type (string, number, float, true, false, nil, array and hash) directly down in C.

@brianmario
Owner

Damn, didn't mean submit that yet...

Anyway, in those cases I don't check for to_json being defined. The to_json check is more of a fallback in case the object being encoded isn't one of those natively translatable types (like an ActiveRecord::Base instance for example). If the object doesn't response to to_json either, then I finally fall back to just calling to_s.

@teamon

Any progress on that? My use case is encoding quite big arrays of complex objects and right now I ended up with cached objects as json string and creating the final json "by hand" just joining strings etc. I'd love to see yajl encoder that could handle that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment