Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Newer
Older
100644 292 lines (221 sloc) 13.034 kb
6248665 ready for use with limitations
Peter Ohler authored
1 # Oj gem
2 A fast JSON parser and Object marshaller as a Ruby gem.
3
4 ## <a name="installation">Installation</a>
5 gem install oj
6
7 ## <a name="source">Source</a>
8
9 *GitHub* *repo*: https://github.com/ohler55/oj
10
11 *RubyGems* *repo*: https://rubygems.org/gems/oj
12
13 ## <a name="build_status">Build Status</a>
14
da0330c getting the travis image to show status
Peter Ohler authored
15 [![Build Status](https://secure.travis-ci.org/ohler55/oj.png?branch=master)](http://travis-ci.org/ohler55/oj)
6248665 ready for use with limitations
Peter Ohler authored
16
17 ## <a name="links">Links of Interest</a>
18
afe4e99 changed json object format
Peter Ohler authored
19 *Fast XML parser and marshaller on RubyGems*: https://rubygems.org/gems/ox
6248665 ready for use with limitations
Peter Ohler authored
20
afe4e99 changed json object format
Peter Ohler authored
21 *Fast XML parser and marshaller on GitHub*: https://rubygems.org/gems/ox
57afd74 fixed a few bugs and added options
Peter Ohler authored
22
afe4e99 changed json object format
Peter Ohler authored
23 ## <a name="release">Release Notes</a>
6248665 ready for use with limitations
Peter Ohler authored
24
5b248cc added Bag class and auto_define flag
Peter Ohler authored
25 ### Release 0.8.0
8711c85 serializes any Object now. Tests added as well.
Peter Ohler authored
26
5b248cc added Bag class and auto_define flag
Peter Ohler authored
27 - Auto creation of data classes when unmarshalling Objects if the Class is not defined
6248665 ready for use with limitations
Peter Ohler authored
28
29 ## <a name="description">Description</a>
30
31 Optimized JSON (Oj), as the name implies was written to provide speed
32 optimized JSON handling. It was designed as a faster alternative to Yajl and
33 other the common Ruby JSON parsers. So far is has achieved that at about 2
34 time faster than Yajl for parsing and 3 or more times faster writing JSON.
35
8711c85 serializes any Object now. Tests added as well.
Peter Ohler authored
36 Oj has several dump or serialization modes which control how Objects are
37 converted to JSON. These modes are set with the :effort option in either the
e0fb1d3 updatings docs and notes
Peter Ohler authored
38 default options or as one of the options to the dump() method.
39
40 - :strict mode will only allow the 7 basic JSON types to be serialized. Any other Object
41 will raise and Exception.
42
43 - :null mode replaces any Object that is not one of the JSON types is replaced by a JSON null.
44
45 - :object mode will dump any Object as a JSON Object with keys that match the
46 Ruby Object's variable names without the '@' character. This is the highest
47 performance mode.
48
49 - :compat mode is is the compatible with other systems. It will serialize any
50 Object but will check to see if the Object implements a to_hash() or to_json()
51 method. If either exists that method is used for serializing the Object. The
52 to_hash() is more flexible and produces more consistent output so it has a
53 preference over the to_json() method. If neither the to_json() or to_hash()
54 methods exist then the Oj internal Object variable encoding is used.
8711c85 serializes any Object now. Tests added as well.
Peter Ohler authored
55
5b248cc added Bag class and auto_define flag
Peter Ohler authored
56 Oj is compatible with Ruby 1.8.7, 1.9.2, 1.9.3, JRuby, and RBX.
57
58 ## <a name="plans">Planned Releases</a>
6248665 ready for use with limitations
Peter Ohler authored
59
5b248cc added Bag class and auto_define flag
Peter Ohler authored
60 - Release 0.9: Support for circular references.
6248665 ready for use with limitations
Peter Ohler authored
61
5b248cc added Bag class and auto_define flag
Peter Ohler authored
62 - Release 1.0: A JSON stream parser.
6248665 ready for use with limitations
Peter Ohler authored
63
6669206 ready for 0.6.0 release
Peter Ohler authored
64 ## <a name="compare">Comparisons</a>
65
66 The following table shows the difference is speeds between several
67 serialization packages. The tests had to be scaled back due to limitation of
e0fb1d3 updatings docs and notes
Peter Ohler authored
68 some of the gems. I finally gave up trying to get JSON Pure to serialize
69 without errors with Ruby 1.9.3. It had internal errors on anything other than
70 a simple JSON structure. The errors encountered were:
6669206 ready for 0.6.0 release
Peter Ohler authored
71
72 - MessagePack fails to convert Bignum to JSON
73
e0fb1d3 updatings docs and notes
Peter Ohler authored
74 - JSON Pure fails to serialize any numbers or Objects with the to_json() method
6669206 ready for 0.6.0 release
Peter Ohler authored
75
e0fb1d3 updatings docs and notes
Peter Ohler authored
76 Options were added to the test/perf_strict.rb test to run the test without
77 Object encoding and without Bignums.
6669206 ready for 0.6.0 release
Peter Ohler authored
78
79 None of the packages except Oj were able to serialize Ruby Objects that did
80 not have a to_json() method or were of the 7 native JSON types.
81
5b248cc added Bag class and auto_define flag
Peter Ohler authored
82 A perf_obj.rb file was added for comparing different Object marshalling
83 packages.
84
6669206 ready for 0.6.0 release
Peter Ohler authored
85 It is also worth noting that although Oj is slightly behind MessagePack for
86 parsing, Oj serialization is much faster than MessagePack even though Oj uses
87 human readable JSON vs the binary MessagePack format.
88
89 The results:
90
91 with Object and Bignum encoding:
92
6234873 ready for a release
Peter Ohler authored
93 MessagePack failed to pack! RangeError: bignum too big to convert into `unsigned long long'.
94 Skipping.
95 --------------------------------------------------------------------------------
96 Load/Parse Performance
97 Oj.load 100000 times in 1.384 seconds or 72230.276 load/sec.
98 Yajl.parse 100000 times in 2.475 seconds or 40401.331 parse/sec.
99 JSON::Ext.parse 100000 times in 2.562 seconds or 39037.263 parse/sec.
100 JSON::Pure.parse 100000 times in 20.914 seconds or 4781.518 parse/sec.
101 Ox.load 100000 times in 1.517 seconds or 65923.576 load/sec.
102
103 Summary:
104 System time (secs) rate (ops/sec)
105 ---------- ----------- --------------
106 Oj 1.384 72230.276
107 Ox 1.517 65923.576
108 Yajl 2.475 40401.331
109 JSON::Ext 2.562 39037.263
110 JSON::Pure 20.914 4781.518
111
112 Comparison Matrix
113 (performance factor, 2.0 row is means twice as fast as column)
114 Oj Ox Yajl JSON::Ext JSON::Pure
115 ---------- ---------- ---------- ---------- ---------- ----------
116 Oj 1.00 1.10 1.79 1.85 15.11
117 Ox 0.91 1.00 1.63 1.69 13.79
118 Yajl 0.56 0.61 1.00 1.03 8.45
119 JSON::Ext 0.54 0.59 0.97 1.00 8.16
120 JSON::Pure 0.07 0.07 0.12 0.12 1.00
121
122
123 --------------------------------------------------------------------------------
124 Dump/Encode/Generate Performance
125 Oj.dump 100000 times in 0.819 seconds or 122096.842 dump/sec.
126 Yajl.encode 100000 times in 2.221 seconds or 45014.913 encode/sec.
127 JSON::Ext.generate 100000 times in 5.082 seconds or 19678.462 generate/sec.
128 ***** JSON::Pure.generate failed! TypeError: wrong argument type JSON::Pure::Generator::State (expected Data)
129 Ox.dump 100000 times in 0.532 seconds or 188014.455 dump/sec.
130
131 Summary:
132 System time (secs) rate (ops/sec)
133 --------- ----------- --------------
134 Ox 0.532 188014.455
135 Oj 0.819 122096.842
136 Yajl 2.221 45014.913
137 JSON::Ext 5.082 19678.462
138
139 Comparison Matrix
140 (performance factor, 2.0 row is means twice as fast as column)
141 Ox Oj Yajl JSON::Ext
142 --------- --------- --------- --------- ---------
143 Ox 1.00 1.54 4.18 9.55
144 Oj 0.65 1.00 2.71 6.20
145 Yajl 0.24 0.37 1.00 2.29
146 JSON::Ext 0.10 0.16 0.44 1.00
6669206 ready for 0.6.0 release
Peter Ohler authored
147
148 without Objects or numbers (for JSON Pure) JSON:
149
6234873 ready for a release
Peter Ohler authored
150 --------------------------------------------------------------------------------
151 Load/Parse Performance
152 Oj.load 100000 times in 0.737 seconds or 135683.185 load/sec.
153 Yajl.parse 100000 times in 1.352 seconds or 73978.778 parse/sec.
154 JSON::Ext.parse 100000 times in 1.433 seconds or 69780.554 parse/sec.
155 JSON::Pure.parse 100000 times in 12.974 seconds or 7707.624 parse/sec.
156 Ox.load 100000 times in 0.904 seconds or 110596.591 load/sec.
157 MessagePack.unpack 100000 times in 0.644 seconds or 155281.191 unpack/sec.
158
159 Summary:
160 System time (secs) rate (ops/sec)
161 ----------- ----------- --------------
162 MessagePack 0.644 155281.191
163 Oj 0.737 135683.185
164 Ox 0.904 110596.591
165 Yajl 1.352 73978.778
166 JSON::Ext 1.433 69780.554
167 JSON::Pure 12.974 7707.624
168
169 Comparison Matrix
170 (performance factor, 2.0 row is means twice as fast as column)
171 MessagePack Oj Ox Yajl JSON::Ext JSON::Pure
172 ----------- ----------- ----------- ----------- ----------- ----------- -----------
173 MessagePack 1.00 1.14 1.40 2.10 2.23 20.15
174 Oj 0.87 1.00 1.23 1.83 1.94 17.60
175 Ox 0.71 0.82 1.00 1.49 1.58 14.35
176 Yajl 0.48 0.55 0.67 1.00 1.06 9.60
177 JSON::Ext 0.45 0.51 0.63 0.94 1.00 9.05
178 JSON::Pure 0.05 0.06 0.07 0.10 0.11 1.00
179
180
181 --------------------------------------------------------------------------------
182 Dump/Encode/Generate Performance
183 Oj.dump 100000 times in 0.161 seconds or 620058.906 dump/sec.
184 Yajl.encode 100000 times in 0.765 seconds or 130637.498 encode/sec.
185 JSON::Ext.generate 100000 times in 3.306 seconds or 30250.212 generate/sec.
186 JSON::Pure.generate 100000 times in 7.067 seconds or 14150.026 generate/sec.
187 Ox.dump 100000 times in 0.178 seconds or 561312.123 dump/sec.
188 MessagePack.pack 100000 times in 0.306 seconds or 326301.535 pack/sec.
189
190 Summary:
191 System time (secs) rate (ops/sec)
192 ----------- ----------- --------------
193 Oj 0.161 620058.906
194 Ox 0.178 561312.123
195 MessagePack 0.306 326301.535
196 Yajl 0.765 130637.498
197 JSON::Ext 3.306 30250.212
198 JSON::Pure 7.067 14150.026
199
200 Comparison Matrix
201 (performance factor, 2.0 row is means twice as fast as column)
202 Oj Ox MessagePack Yajl JSON::Ext JSON::Pure
203 ----------- ----------- ----------- ----------- ----------- ----------- -----------
204 Oj 1.00 1.10 1.90 4.75 20.50 43.82
205 Ox 0.91 1.00 1.72 4.30 18.56 39.67
206 MessagePack 0.53 0.58 1.00 2.50 10.79 23.06
207 Yajl 0.21 0.23 0.40 1.00 4.32 9.23
208 JSON::Ext 0.05 0.05 0.09 0.23 1.00 2.14
209 JSON::Pure 0.02 0.03 0.04 0.11 0.47 1.00
210
211
6248665 ready for use with limitations
Peter Ohler authored
212 ### Simple JSON Writing and Parsing:
213
214 require 'oj'
215
216 h = { 'one' => 1, 'array' => [ true, false ] }
217 json = Oj.dump(h)
218
219 # json =
220 # {
221 # "one":1,
222 # "array":[
223 # true,
224 # false
225 # ]
226 # }
227
228 h2 = Oj.parse(json)
229 puts "Same? #{h == h2}"
230 # true
4d28d2c updated readme with message format for object encoding
Peter Ohler authored
231
232 ### Object JSON format:
233
234 In :object mode Oj generates JSON that follows conventions which allow Class
235 and other information such as Object IDs for circular reference detection. The
236 formating follows the following rules.
237
238 1. JSON native types, true, false, nil, String, Hash, Array, and Number are
239 encoded normally.
240
241 2. If a Hash uses Symbols as keys those keys appear as Strings with a leading
242 ':' character.
243
244 3. The '^' character denotes a special key value when in a JSON Object sequence.
245
246 4. If a String begins with a ':' character such as ':abc' it is encoded as {"^s":":abc"}.
247
248 5. If a Symbol begins with a ':' character such as :":abc" is is encoded as {"^m":":abc"}.
249
250 6. A "^c" JSON Object key indicates the value should be converted to a Ruby
251 class. The sequence {"^c":"Oj::Bag"} is read as the Oj::Bag class.
252
253 7. A "^t" JSON Object key indicates the value should be converted to a Ruby
254 Time. The sequence {"^t":1325775487.000000} is read as Jan 5, 2012 at
255 23:58:07.
256
257 8. A "^o" JSON Object key indicates the value should be converted to a Ruby
258 Object. The first entry in the JSON Object must be a class with the "^o"
259 key. After that each entry is treated as a variable of the Object where the
260 key is the variable name without the preceeding '@'. An example is
261 {"^o":"Oj::Bag","x":58,"y":"marbles"}.
262
0dfa515 now dumping struct in the correct format. Loading still unimplemented
Peter Ohler authored
263 9. A "^u" JSON Object key indicates the value should be converted to a Ruby
264 Struct. The first entry in the JSON Object must be a class with the "^u"
265 key. After that each entry is is given a numeric position in the struct and
266 that is used as the key in the JSON Object. An example is
267 {"^u":["Range",1,7,false]}.
268
269 10. When encoding an Object, if the variable name does not begin with an '@'
4d28d2c updated readme with message format for object encoding
Peter Ohler authored
270 character then the name preceeded by a '~' character. This occurs in the
271 Exception class. An example is {"^o":"StandardError","~mesg":"A
272 Message","~bt":[".\/tests.rb:345:in `test_exception'"]}
273
0dfa515 now dumping struct in the correct format. Loading still unimplemented
Peter Ohler authored
274 11. If a Hash entry has a key that is not a String or Symbol then the entry is
4d28d2c updated readme with message format for object encoding
Peter Ohler authored
275 encoded with a key of the form "^#n" where n is a hex number. The value that
276 is an Array where the first element is the key in the Hash and the second is
277 the value. An example is {"^#3":[2,5]}.
278
0dfa515 now dumping struct in the correct format. Loading still unimplemented
Peter Ohler authored
279 12. A "^i" JSON entry in either an Object or Array is the ID of the Ruby
4d28d2c updated readme with message format for object encoding
Peter Ohler authored
280 Object being encoded. It is used when the :circular flag is set. It can appear
fdacd8d added hash circular support
Peter Ohler authored
281 in either a JSON Object or in a JSON Array. In an Object the "^i" key has a
282 corresponding reference Fixnum. In an array the sequence will include an
283 embedded reference number. An example is
284 {"^o":"Oj::Bag","^i":1,"x":["^i2":2,true],"me":{"^r":1}}.
285
286 13. A "^r" JSON entry in an Object is a references to a Object or Array that
287 already appears in the JSON String. It must match up with a previous "^i"
288 ID. An example is {"^o":"Oj::Bag","^i":1,"x":3,"me":{"^r":1}}.
289
290 14. If an Array element is a String and starts with "^i" then it is encoded as
291 an String Object. An example is [{"^s":"^i37"},3].
Something went wrong with that request. Please try again.