Issues - brianmario/yajl-ruby

brianmario / yajl-ruby

257

Description:

An extremely efficient streaming JSON parsing and encoding library. Ruby C bindings to Yajl

Sort by: Priority | Votes | Last Updated

Loading…

7.

0 votes

encoder
x

feature request
x

Wish: Sort Keys
5 comments Created 3 months ago by TobiX

Title

Body
I'm using Yajl to dump data structures into a readable format for version control. It would be nice if the keys in the JSON output would be sorted to get a stable representation of the data. A option like :pretty would be nice...
or cancel

I'm using Yajl to dump data structures into a readable format for version control. It would be nice if the keys in the JSON output would be sorted to get a stable representation of the data. A option like :pretty would be nice...

Comments

TobiX Mon Oct 05 23:48:55 -0700 2009 | link

Please see my fork for an implementation...

Comment
Please see my fork for an implementation...
or cancel

brianmario Tue Oct 06 00:38:23 -0700 2009 | link

Was just taking a look at that, awesome!

Comment
Was just taking a look at that, awesome!
or cancel

brianmario Tue Oct 06 00:53:04 -0700 2009 | link

Ok so I pulled down the patch and played with it a bit.
Everything looks good except the sort option used in conjunction with a Hash that has Symbols for keys will fail to sort because the Symbol class doesn't have a <=> method out of the box.
I have a couple of ideas for how to handle that, but I wanted to see what you thought in case this is a use-case you'd need to cover for your own needs as well?

Comment
Ok so I pulled down the patch and played with it a bit. Everything looks good except the sort option used in conjunction with a Hash that has Symbols for keys will fail to sort because the Symbol class doesn't have a <=> method out of the box. I have a couple of ideas for how to handle that, but I wanted to see what you thought in case this is a use-case you'd need to cover for your own needs as well?
or cancel

TobiX Tue Oct 06 12:38:43 -0700 2009 | link

Since all of my keys are strings, I didn't test that case... A graceful fallback when the keys can't be sorted would be nice.

Comment
Since all of my keys are strings, I didn't test that case... A graceful fallback when the keys can't be sorted would be nice.
or cancel

TobiX Tue Oct 13 00:09:53 -0700 2009 | link

Another case that fails are mixed keys (Strings and Fixnum and Symbols)...

Since I don't have any strong opinion about the "correct" sort order, how about something in the spirit of this:

a.sort_by {|x| [x.class.to_s, (x.respond_to? :to_s)? x.to_s : x.inspect]}

This would group keys by their class and sort them by some kind of string representation, falling back to ".inspect" if there is no to_s defined...

Comment
Another case that fails are mixed keys (Strings and Fixnum and Symbols)... Since I don't have any strong opinion about the "correct" sort order, how about something in the spirit of this: a.sort_by {|x| [x.class.to_s, (x.respond_to? :to_s)? x.to_s : x.inspect]} This would group keys by their class and sort them by some kind of string representation, falling back to ".inspect" if there is no to_s defined...
or cancel

Parsed with GitHub Flavored Markdown
10.

-1 votes

JSON gem compat
x

yajl/json_gem & json conflict in ohai+chef
4 comments Created 2 months ago by fujin

Title

Body
With this commit to ohai: http://github.com/opscode/ohai/commit/f89baccc3b9ab587d23e0b6257f6fedffe223c02 However, if you remove 'json, 'json_pure', then run 'rake spec' from another project [chef] that requires ohai (HAS NOT BEEN SWITCHED TO YAJL YET): <pre> aj@AJ-Christensens-MacBook (branch: master) ~/Development/chef/chef$ rake spec (in /Users/aj/Development/chef/chef) /Users/aj/.gem/ruby/1.8/gems/yajl-ruby-0.6.3/lib/yajl/json_gem/parsing.rb:5: superclass mismatch for class ParserError (TypeError) from /opt/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require' from /opt/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `require' from /Users/aj/.gem/ruby/1.8/gems/yajl-ruby-0.6.3/lib/yajl/json_gem.rb:3 from /opt/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:36:in `gem_original_require' from /opt/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:36:in `require' from /opt/local/lib/ruby/gems/1.8/gems/ohai-0.3.4/lib/ohai/system.rb:27 from /opt/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require' from /opt/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `require' from /opt/local/lib/ruby/gems/1.8/gems/ohai-0.3.4/lib/ohai.rb:23 from /opt/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:36:in `gem_original_require' from /opt/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:36:in `require' from /Users/aj/Development/chef/chef/spec/../lib/chef/client.rb:31 from /opt/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require' from /opt/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `require' from /Users/aj/Development/chef/chef/spec/../lib/chef/application/client.rb:19 from /opt/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require' from /opt/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `require' from /Users/aj/Development/chef/chef/spec/spec_helper.rb:34 from /Users/aj/Development/chef/chef/spec/spec_helper.rb:26:in `each' from /Users/aj/Development/chef/chef/spec/spec_helper.rb:26 from ./spec/unit/client_spec.rb:19:in `require' from ./spec/unit/client_spec.rb:19 </pre> Any ideas? chef project ticket ref# http://tickets.opscode.com/browse/CHEF-592 Changing the require to straight up 'yajl' supposedly works, although I haven't verified this.
or cancel
With this commit to ohai: http://github.com/opscode/ohai/commit/f89baccc3b9ab587d23e0b6257f6fedffe223c02

However, if you remove 'json, 'json_pure', then run 'rake spec' from another project [chef] that requires ohai (HAS NOT BEEN SWITCHED TO YAJL YET):
```
aj@AJ-Christensens-MacBook (branch: master) ~/Development/chef/chef$ rake spec
(in /Users/aj/Development/chef/chef) /Users/aj/.gem/ruby/1.8/gems/yajl-ruby-0.6.3/lib/yajl/json_gem/parsing.rb:5: superclass mismatch for class ParserError (TypeError)
from /opt/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require'
from /opt/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `require'
from /Users/aj/.gem/ruby/1.8/gems/yajl-ruby-0.6.3/lib/yajl/json_gem.rb:3
from /opt/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:36:in `gem_original_require'
from /opt/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:36:in `require'
from /opt/local/lib/ruby/gems/1.8/gems/ohai-0.3.4/lib/ohai/system.rb:27
from /opt/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require'
from /opt/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `require'
from /opt/local/lib/ruby/gems/1.8/gems/ohai-0.3.4/lib/ohai.rb:23
from /opt/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:36:in `gem_original_require'
from /opt/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:36:in `require'
from /Users/aj/Development/chef/chef/spec/../lib/chef/client.rb:31
from /opt/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require'
from /opt/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `require'
from /Users/aj/Development/chef/chef/spec/../lib/chef/application/client.rb:19
from /opt/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require'
from /opt/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `require'
from /Users/aj/Development/chef/chef/spec/spec_helper.rb:34
from /Users/aj/Development/chef/chef/spec/spec_helper.rb:26:in `each'
from /Users/aj/Development/chef/chef/spec/spec_helper.rb:26
from ./spec/unit/client_spec.rb:19:in `require'
from ./spec/unit/client_spec.rb:19
```
Any ideas? chef project ticket ref# http://tickets.opscode.com/browse/CHEF-592

Changing the require to straight up 'yajl' supposedly works, although I haven't verified this.
Comments

brianmario Thu Oct 08 11:42:40 -0700 2009 | link

I think I've spotted the problem and have a fix, but I'm unable to test at the moment cause running "rake spec" with the master branch of chef is giving me a load error (I installed the chef gem so it would grab all the right dependencies as well):

`gem_original_require': no such file to load -- mixlib/log (LoadError)

ideas?

Comment
I think I've spotted the problem and have a fix, but I'm unable to test at the moment cause running "rake spec" with the master branch of chef is giving me a load error (I installed the chef gem so it would grab all the right dependencies as well): `gem_original_require': no such file to load -- mixlib/log (LoadError) ideas?
or cancel

brianmario Thu Oct 08 12:47:27 -0700 2009 | link

Figured that load error out, for whatever reason the mixlib-* gems installed themselves with 700 permissions...

Comment
Figured that load error out, for whatever reason the mixlib-* gems installed themselves with 700 permissions...
or cancel

fujin Thu Oct 08 20:16:24 -0700 2009 | link

weird. I'll look into that.

Suggestion for fix from one of our users:
"Actually the fix is to patch yajl-ruby and not chef.

a bit of inheritance, but if you patch (pardon the path)

/usr/lib64/ruby/gems/1.8/gems/yajl-ruby-0.6.3/lib/yajl/json_gem/parsing.rb

hit up line 5 and it needs to read,

class ::ParserError < ::Yajl::ParseError; end

not class ParseError ..."

Not sure if this helps, or is relevant.

Comment
weird. I'll look into that. Suggestion for fix from one of our users: "Actually the fix is to patch yajl-ruby and not chef. a bit of inheritance, but if you patch (pardon the path) /usr/lib64/ruby/gems/1.8/gems/yajl-ruby-0.6.3/lib/yajl/json_gem/parsing.rb hit up line 5 and it needs to read, class ::ParserError < ::Yajl::ParseError; end not class ParseError ..." Not sure if this helps, or is relevant.
or cancel

brianmario Mon Oct 19 18:22:57 -0700 2009 | link

Would you mind testing with the latest from yajl-ruby master?
I made some modifications regarding the exception classes that I think will fix that issue. But I'm seeing something else now regarding the custom serialization stuff in the JSON gem that Chef uses.
For example:

1)
'Chef::Node json should deserialize itself from json' FAILED expected {"name"=>"test.example.com", "json_class"=>"Chef::Node", "attributes"=>{"sunshine"=>"in", "something"=>"else"}, "run_list"=>["recipe[operations-master]", "recipe[operations-monitoring]"], "chef_type"=>"node"} to be a kind of Chef::Node
./spec/unit/node_spec.rb:257

Which basically means yajl-ruby doesn't support this custom [de]serialization pattern the JSON gem has implemented. I'd like to at least know you can get past the Exception class problems you're having before I investigate further.

Comment
Would you mind testing with the latest from yajl-ruby master? I made some modifications regarding the exception classes that I think will fix that issue. But I'm seeing something else now regarding the custom serialization stuff in the JSON gem that Chef uses. For example: 1) 'Chef::Node json should deserialize itself from json' FAILED expected {"name"=>"test.example.com", "json_class"=>"Chef::Node", "attributes"=>{"sunshine"=>"in", "something"=>"else"}, "run_list"=>["recipe[operations-master]", "recipe[operations-monitoring]"], "chef_type"=>"node"} to be a kind of Chef::Node ./spec/unit/node_spec.rb:257 Which basically means yajl-ruby doesn't support this custom [de]serialization pattern the JSON gem has implemented. I'd like to at least know you can get past the Exception class problems you're having before I investigate further.
or cancel

Parsed with GitHub Flavored Markdown

Labels

encoderx feature requestx Wish: Sort Keys

Comments

JSON gem compatx yajl/json_gem & json conflict in ohai+chef

Comments

encoder
x

feature request
x

Wish: Sort Keys

JSON gem compat
x

yajl/json_gem & json conflict in ohai+chef