Skip to content
Browse files

updated benchmarks for Marshal.load/dump to fix multiple iteration pr…

…oblems

added C documentation for the JSON compatibility API functions
bumped the internal version, updated changelog, readme and gemspec for 0.5.5 release
  • Loading branch information...
1 parent c9864c9 commit ceec0b517171aeab917bd9f66db66fbbf10d5969 @brianmario committed Jun 18, 2009
Showing with 188 additions and 10 deletions.
  1. +9 −0 CHANGELOG.md
  2. +49 −3 README.rdoc
  3. +2 −2 benchmark/encode_json_and_marshal.rb
  4. +2 −2 benchmark/parse_json_and_marshal.rb
  5. +119 −0 ext/yajl_ext.c
  6. +1 −1 lib/yajl.rb
  7. +6 −2 yajl-ruby.gemspec
View
9 CHANGELOG.md
@@ -1,5 +1,14 @@
# Changelog
+## 0.5.5 (June 17th, 2009)
+* Introduction of the JSON gem API compatibility layer
+** NOTE: this isn't a 1:1 compatibility API, the goal was to be compatible with as many of the projects using the JSON gem as possible - not the JSON gem API itself
+** the compatibility API must be explicitly enabled by requiring 'yajl/json_gem' in your project
+** JSON.parse, JSON.generate, and the #to_json instance method extension to ruby's primitive classes are all included
+* Fix Yajl::Encoder to ensure map keys are strings
+* Encoding multiple JSON objects to a single stream doesn't separate by a newline character anymore
+* Yajl::Encoder now checks for the existence of, and will call #to_json on any non-primitive object
+
## 0.5.4 (June 16th, 2009)
* Yajl::Parser's :symbolize_keys option now defaults to false
* remove use of sprintf for a little speed improvement while parsing
View
52 README.rdoc
@@ -4,6 +4,19 @@ This gem (although not in gem form just yet) is a C binding to the excellent YAJ
You can read more info at the projects website http://lloydforge.org/projects/yajl or check out it's codes at http://github.com/lloyd/yajl.
+== Features
+
+* JSON parsing and encoding directly to and from an IO stream (file, socket, etc) or String. Compressed stream parsing and encoding supported for Bzip2, Gzip and Deflate.
+* Parse and encode *multiple* JSON objects to and from streams or strings continuously.
+* JSON gem compatibility API - allows yajl-ruby to be used as a drop-in replacement for the JSON gem
+* Basic HTTP client (only GET requests supported for now) which parses JSON directly off the response body *as it's being received*
+* ~3.5x faster than JSON.generate
+* ~1.9x faster than JSON.parse
+* ~4.5x faster than YAML.load
+* ~377.5x faster than YAML.dump
+* ~1.5x faster than Marshal.load
+* ~2x faster than Marshal.dump
+
== How to install
Install it like any other gem hosted at the Githubs like so:
@@ -114,7 +127,7 @@ This allows you to encode JSON as a stream, writing directly to a socket
socket = TCPSocket.new(192.168.1.101, 9000)
hash = {:foo => 12425125, :bar => "some string", ... }
encoder = Yajl::Encoder.new
- Yajl::encoder.encode(hash, socket)
+ Yajl::Encoder.encode(hash, socket)
Or what if you wanted to compress the stream over the wire?
@@ -135,7 +148,24 @@ This example will encode and send 50 JSON objects over the same stream, continuo
You can also use Yajl::Bzip2::StreamWriter and Yajl::Deflate::StreamWriter. So you can pick whichever fits your CPU/bandwidth sweet-spot.
-There are a lot more possibilities, some of which I'm going to write other gems/plugins for.
+== JSON gem Compatibility API
+
+The JSON gem compatibility API isn't enabled by default. You have to explicitly require it like so:
+
+ require 'yajl/json_gem'
+
+That's right, you can just replace "require 'json'" with the line above and you're done!
+
+This will require yajl-ruby itself, as well as enable it's JSON gem compatibility API.
+
+This includes the following API:
+
+ JSON.parse, JSON.generate, JSON.pretty_generate, JSON.load, JSON.dump
+ and all of the #to_json instance method overrides for Ruby's primitive objects
+
+Once the compatibility API is enabled, you're existing or new project should work as if the JSON gem itself were being used. Only you'll be using Yajl ;)
+
+There are a lot more possibilities that I'd love to see other gems/plugins for someday.
Some ideas are:
* parsing logs in JSON format
@@ -192,14 +222,30 @@ NOTE: I converted the 2.4MB JSON file to YAML for this test.
* Yajl::Parser#parse: 4.33s
* JSON.parse: 5.37s
-* YAML.load_stream: 19.47s
+* YAML.load: 19.47s
==== Encode Time (to their respective formats)
* Yajl::Encoder#encode: 3.47s
* JSON#to_json: 6.6s
* YAML.dump(obj, io): 1309.93s
+=== Compared to Marshal.load/Marshal.dump
+
+NOTE: I converted the 2.4MB JSON file to a Hash and a dump file from Marshal.dump for this test.
+
+==== Parse Time (from their respective formats)
+
+* Yajl::Parser#parse: 4.54s
+* JSON.parse: 7.40s
+* Marshal.load: 7s
+
+==== Encode Time (to their respective formats)
+
+* Yajl::Encoder#encode: 2.39s
+* JSON#to_json: 8.37s
+* Marshal.dump: 4.66s
+
== Third Party Sources Bundled
This project includes code from the BSD licensed yajl project, copyright 2007-2009 Lloyd Hilaiel
View
4 benchmark/encode_json_and_marshal.rb
@@ -5,12 +5,12 @@
require 'stringio'
require 'json'
-filename = ARGV[0] || 'benchmark/subjects/contacts.json'
+times = ARGV[0] ? ARGV[0].to_i : 1
+filename = 'benchmark/subjects/contacts.json'
json = File.new(filename, 'r')
hash = Yajl::Parser.new.parse(json)
json.close
-times = ARGV[1] ? ARGV[1].to_i : 1
puts "Starting benchmark encoding #{filename} #{times} times\n\n"
Benchmark.bm { |x|
encoder = Yajl::Encoder.new
View
4 benchmark/parse_json_and_marshal.rb
@@ -21,12 +21,11 @@
times = ARGV[0] ? ARGV[0].to_i : 1
puts "Starting benchmark parsing #{File.size(filename)} bytes of JSON data #{times} times\n\n"
Benchmark.bm { |x|
- parser = Yajl::Parser.new
x.report {
puts "Yajl::Parser#parse"
times.times {
json.rewind
- hash = parser.parse(json)
+ hash = Yajl::Parser.new.parse(json)
}
}
x.report {
@@ -39,6 +38,7 @@
x.report {
puts "Marshal.load"
times.times {
+ marshal_file.rewind
Marshal.load(marshal_file)
}
}
View
119 ext/yajl_ext.c
@@ -523,6 +523,19 @@ static VALUE rb_yajl_encoder_encode(int argc, VALUE * argv, VALUE self) {
// JSON Gem compatibility
+
+/*
+ * Document-class: Object
+ */
+/*
+ * Document-method: to_json
+ *
+ * call-seq: to_json(encoder=Yajl::Encoder.new)
+ *
+ * +encoder+ is an existing Yajl::Encoder used to encode JSON
+ *
+ * Encodes an instance of Object to JSON
+ */
static VALUE rb_yajl_json_ext_object_to_json(int argc, VALUE * argv, VALUE self) {
VALUE rb_encoder;
rb_scan_args(argc, argv, "01", &rb_encoder);
@@ -532,6 +545,18 @@ static VALUE rb_yajl_json_ext_object_to_json(int argc, VALUE * argv, VALUE self)
return rb_yajl_encoder_encode(1, &self, rb_encoder);
}
+/*
+ * Document-class: Hash
+ */
+/*
+ * Document-method: to_json
+ *
+ * call-seq: to_json(encoder=Yajl::Encoder.new)
+ *
+ * +encoder+ is an existing Yajl::Encoder used to encode JSON
+ *
+ * Encodes an instance of Hash to JSON
+ */
static VALUE rb_yajl_json_ext_hash_to_json(int argc, VALUE * argv, VALUE self) {
VALUE rb_encoder;
rb_scan_args(argc, argv, "01", &rb_encoder);
@@ -541,6 +566,18 @@ static VALUE rb_yajl_json_ext_hash_to_json(int argc, VALUE * argv, VALUE self) {
return rb_yajl_encoder_encode(1, &self, rb_encoder);
}
+/*
+ * Document-class: Array
+ */
+/*
+ * Document-method: to_json
+ *
+ * call-seq: to_json(encoder=Yajl::Encoder.new)
+ *
+ * +encoder+ is an existing Yajl::Encoder used to encode JSON
+ *
+ * Encodes an instance of Array to JSON
+ */
static VALUE rb_yajl_json_ext_array_to_json(int argc, VALUE * argv, VALUE self) {
VALUE rb_encoder;
rb_scan_args(argc, argv, "01", &rb_encoder);
@@ -550,6 +587,18 @@ static VALUE rb_yajl_json_ext_array_to_json(int argc, VALUE * argv, VALUE self)
return rb_yajl_encoder_encode(1, &self, rb_encoder);
}
+/*
+ * Document-class: Fixnum
+ */
+/*
+ * Document-method: to_json
+ *
+ * call-seq: to_json(encoder=Yajl::Encoder.new)
+ *
+ * +encoder+ is an existing Yajl::Encoder used to encode JSON
+ *
+ * Encodes an instance of Fixnum to JSON
+ */
static VALUE rb_yajl_json_ext_fixnum_to_json(int argc, VALUE * argv, VALUE self) {
VALUE rb_encoder;
rb_scan_args(argc, argv, "01", &rb_encoder);
@@ -559,6 +608,18 @@ static VALUE rb_yajl_json_ext_fixnum_to_json(int argc, VALUE * argv, VALUE self)
return rb_yajl_encoder_encode(1, &self, rb_encoder);
}
+/*
+ * Document-class: Float
+ */
+/*
+ * Document-method: to_json
+ *
+ * call-seq: to_json(encoder=Yajl::Encoder.new)
+ *
+ * +encoder+ is an existing Yajl::Encoder used to encode JSON
+ *
+ * Encodes an instance of Float to JSON
+ */
static VALUE rb_yajl_json_ext_float_to_json(int argc, VALUE * argv, VALUE self) {
VALUE rb_encoder;
rb_scan_args(argc, argv, "01", &rb_encoder);
@@ -568,6 +629,18 @@ static VALUE rb_yajl_json_ext_float_to_json(int argc, VALUE * argv, VALUE self)
return rb_yajl_encoder_encode(1, &self, rb_encoder);
}
+/*
+ * Document-class: String
+ */
+/*
+ * Document-method: to_json
+ *
+ * call-seq: to_json(encoder=Yajl::Encoder.new)
+ *
+ * +encoder+ is an existing Yajl::Encoder used to encode JSON
+ *
+ * Encodes an instance of TrueClass to JSON
+ */
static VALUE rb_yajl_json_ext_string_to_json(int argc, VALUE * argv, VALUE self) {
VALUE rb_encoder;
rb_scan_args(argc, argv, "01", &rb_encoder);
@@ -577,6 +650,18 @@ static VALUE rb_yajl_json_ext_string_to_json(int argc, VALUE * argv, VALUE self)
return rb_yajl_encoder_encode(1, &self, rb_encoder);
}
+/*
+ * Document-class: TrueClass
+ */
+/*
+ * Document-method: to_json
+ *
+ * call-seq: to_json(encoder=Yajl::Encoder.new)
+ *
+ * +encoder+ is an existing Yajl::Encoder used to encode JSON
+ *
+ * Encodes an instance of TrueClass to JSON
+ */
static VALUE rb_yajl_json_ext_true_to_json(int argc, VALUE * argv, VALUE self) {
VALUE rb_encoder;
rb_scan_args(argc, argv, "01", &rb_encoder);
@@ -586,6 +671,18 @@ static VALUE rb_yajl_json_ext_true_to_json(int argc, VALUE * argv, VALUE self) {
return rb_yajl_encoder_encode(1, &self, rb_encoder);
}
+/*
+ * Document-class: FalseClass
+ */
+/*
+ * Document-method: to_json
+ *
+ * call-seq: to_json(encoder=Yajl::Encoder.new)
+ *
+ * +encoder+ is an existing Yajl::Encoder used to encode JSON
+ *
+ * Encodes an instance of FalseClass to JSON
+ */
static VALUE rb_yajl_json_ext_false_to_json(int argc, VALUE * argv, VALUE self) {
VALUE rb_encoder;
rb_scan_args(argc, argv, "01", &rb_encoder);
@@ -595,6 +692,18 @@ static VALUE rb_yajl_json_ext_false_to_json(int argc, VALUE * argv, VALUE self)
return rb_yajl_encoder_encode(1, &self, rb_encoder);
}
+/*
+ * Document-class: NilClass
+ */
+/*
+ * Document-method: to_json
+ *
+ * call-seq: to_json(encoder=Yajl::Encoder.new)
+ *
+ * +encoder+ is an existing Yajl::Encoder used to encode JSON
+ *
+ * Encodes an instance of NilClass to JSON
+ */
static VALUE rb_yajl_json_ext_nil_to_json(int argc, VALUE * argv, VALUE self) {
VALUE rb_encoder;
rb_scan_args(argc, argv, "01", &rb_encoder);
@@ -604,6 +713,16 @@ static VALUE rb_yajl_json_ext_nil_to_json(int argc, VALUE * argv, VALUE self) {
return rb_yajl_encoder_encode(1, &self, rb_encoder);
}
+/*
+ * Document-class: Yajl::Encoder
+ */
+/*
+ * Document-method: enable_json_gem_compatability
+ *
+ * call-seq: enable_json_gem_compatability
+ *
+ * Enables the JSON gem compatibility API
+ */
static VALUE rb_yajl_encoder_enable_json_gem_ext(VALUE klass) {
rb_define_method(rb_cObject, "to_json", rb_yajl_json_ext_object_to_json, -1);
rb_define_method(rb_cHash, "to_json", rb_yajl_json_ext_hash_to_json, -1);
View
2 lib/yajl.rb
@@ -13,7 +13,7 @@
#
# Ruby bindings to the excellent Yajl (Yet Another JSON Parser) ANSI C library.
module Yajl
- VERSION = "0.5.4"
+ VERSION = "0.5.5"
class Parser
# A helper method for parse-and-forget use-cases
View
8 yajl-ruby.gemspec
@@ -2,11 +2,11 @@
Gem::Specification.new do |s|
s.name = %q{yajl-ruby}
- s.version = "0.5.4"
+ s.version = "0.5.5"
s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
s.authors = ["Brian Lopez", "Lloyd Hilaiel"]
- s.date = %q{2009-06-16}
+ s.date = %q{2009-06-18}
s.email = %q{seniorlopez@gmail.com}
s.extensions = ["ext/extconf.rb"]
s.extra_rdoc_files = [
@@ -28,6 +28,7 @@ Gem::Specification.new do |s|
"benchmark/parse_json_and_yaml.rb",
"benchmark/parse_stream.rb",
"benchmark/subjects/contacts.json",
+ "benchmark/subjects/contacts.marshal_dump",
"benchmark/subjects/contacts.yml",
"benchmark/subjects/item.json",
"benchmark/subjects/ohai.json",
@@ -69,12 +70,14 @@ Gem::Specification.new do |s|
"lib/yajl/gzip/stream_reader.rb",
"lib/yajl/gzip/stream_writer.rb",
"lib/yajl/http_stream.rb",
+ "lib/yajl/json_gem.rb",
"spec/encoding/encoding_spec.rb",
"spec/http/fixtures/http.bzip2.dump",
"spec/http/fixtures/http.deflate.dump",
"spec/http/fixtures/http.gzip.dump",
"spec/http/fixtures/http.raw.dump",
"spec/http/http_spec.rb",
+ "spec/json_gem_compatibility/compatibility_spec.rb",
"spec/parsing/active_support_spec.rb",
"spec/parsing/chunked_spec.rb",
"spec/parsing/fixtures/fail.15.json",
@@ -147,6 +150,7 @@ Gem::Specification.new do |s|
s.test_files = [
"spec/encoding/encoding_spec.rb",
"spec/http/http_spec.rb",
+ "spec/json_gem_compatibility/compatibility_spec.rb",
"spec/parsing/active_support_spec.rb",
"spec/parsing/chunked_spec.rb",
"spec/parsing/fixtures_spec.rb",

0 comments on commit ceec0b5

Please sign in to comment.
Something went wrong with that request. Please try again.