Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Port from json to multi_json #142

Closed
wants to merge 6 commits into from

5 participants

@nbudin

Hello! This patch ports rdoc from the json gem to multi_json in order to allow the use of alternative JSON parser/generators such as YAJL or Oj.

The reason for writing this patch is that recently I've been experimenting with running Rails under MacRuby. This is currently impossible because railties depends on rdoc, and rdoc depends on the json gem. Loading the json gem makes MacRuby blow up because its implementation is incompatible, and the authors of MacRuby have written an alternative implementation based on Apple's NSJSONSerialization to work around it. While their implementation works fine, MacRuby users can't load the json gem.

If rdoc accepts this patch, then hopefully this will help packages that depend on rdoc, such as railties, become compatible with MacRuby.

@travisbot

This pull request passes (merged ce2e3f5 into c8f0298).

@drbrain
Owner

Since RDoc ships with ruby it can only depend on gems that exist in the latest releases of ruby. json is part of ruby so I can't replace the dependency with multi_json. From your description, it sounds like installing the json gem on MacRuby is harmless until you try to use it, so this should be acceptable.

I would be willing to accept a modified version of this patch to enable MacRuby compatibility until they have a working JSON implementation built-in.

I think something like this in lib/rdoc/json.rb:

class RDoc

  case RUBY_ENGINE
  when 'macruby' then
    begin
      require 'multi_json'
      JSON = MultiJson
    rescue LoadError
      abort 'Unable to load multi_json. MacRuby cannot use json at this time, so please install multi_json by hand.'
    end
  else
    gem 'json'
    require 'json'
    JSON = ::JSON
  end
end

Along with an autoload in lib/rdoc.rb, and a change of MultiJson to RDoc::JSON.

@travisbot

This pull request fails (merged 5f10900 into c8f0298).

@travisbot

This pull request fails (merged 969158a into c8f0298).

@travisbot

This pull request passes (merged 74fdec5 into c8f0298).

@travisbot

This pull request fails (merged 9fb2319 into c8f0298).

@travisbot

This pull request passes (merged 98ac1de into c8f0298).

@zzak
Collaborator

Is the problem here, that macruby's json is incompatible? How so?

I saw they upgraded the json lib to 1.6.5 in MacRuby/MacRuby#63

@zenspider
Owner

I think this change is massive overkill, but I would like to see the json gem dependency dropped in some way so that I might meet the requirement any way I want (or something else... I don't care HOW it gets done). My current problem is having GEM_HOME set in my shell w/ the json gem compiled for 1.8 and then switching to 1.9 totally horks my system when I interact with gems. I'd rather install json_pure myself and not worry about it.

@drbrain
Owner

I installed MacRuby 0.12:

$ macruby -v
MacRuby 0.12 (ruby 1.9.2) [universal-darwin10.0, x86_64]

I am having extreme difficulty getting it to run the tests. The travis:before task will install all the necessary dependencies for RDoc development:

$ macrake -t travis:before
rake aborted!
no such file to load -- hoe
/Users/drbrain/Work/git/rdoc/Rakefile:3:in `<main>'
/usr/local/bin/macrake:31:in `<main>'

MacRuby claims to be a ruby 1.9.2 implementation, but has not loaded RubyGems. Loading RubyGems manually results in many warnings and a segmentation fault:

$ macruby -rubygems -S macrake -t travis:before
unknown: warning: already initialized constant MAJOR
unknown: warning: already initialized constant MINOR
unknown: warning: already initialized constant BUILD
unknown: warning: already initialized constant NUMBERS
unknown: warning: already initialized constant VERSION
unknown: warning: already initialized constant RAKEVERSION
unknown: warning: already initialized constant EARLY
unknown: warning: already initialized constant RUBY
unknown: warning: already initialized constant LN_SUPPORTED
unknown: warning: already initialized constant DEFAULT
WARNING: Possible conflict with Rake extension: String#ext already exists
WARNING: Possible conflict with Rake extension: String#pathmap already exists
unknown: warning: already initialized constant ARRAY_METHODS
unknown: warning: already initialized constant MUST_DEFINE
unknown: warning: already initialized constant MUST_NOT_DEFINE
unknown: warning: already initialized constant SPECIAL_RETURN
unknown: warning: already initialized constant DELEGATING_METHODS
unknown: warning: already initialized constant DEFAULT_IGNORE_PATTERNS
unknown: warning: already initialized constant DEFAULT_IGNORE_PROCS
unknown: warning: already initialized constant WINDOWS
unknown: warning: already initialized constant DEFAULT_RAKEFILES
unknown: warning: already initialized constant EMPTY_TASK_ARGS
unknown: warning: already initialized constant EMPTY
unknown: warning: already initialized constant DeprecatedCommands
unknown: warning: already initialized constant FileList
unknown: warning: already initialized constant RakeFileUtils
Segmentation fault: 11

Attempting to run a test that uses JSON results in the following failures:

$ macruby -Ilib test/test_rdoc_generator_json_index.rb 
Loaded suite test/test_rdoc_generator_json_index
Started
/Users/drbrain/Work/git/rdoc/lib/rdoc/text.rb:71: regexp `^(.{8}*?)([^\t\r\n]{0,7})\t' compilation error: U_REGEX_RULE_SYNTAX
EEEEEEEEEE
Finished in 0.688472 seconds.

  1) Error:
test_generate(TestRDocGeneratorJsonIndex):
SyntaxError: compile error
    /Users/drbrain/Work/git/rdoc/lib/rdoc/code_object.rb:28:in `block'
    /Users/drbrain/Work/git/rdoc/lib/rdoc/code_object.rb:26:in `<main>'
    /Users/drbrain/Work/git/rdoc/lib/rdoc/generator/markup.rb:59:in `<main>'
    /Users/drbrain/Work/git/rdoc/lib/rdoc/generator/darkfish.rb:6:in `<main>'
    /Users/drbrain/Work/git/rdoc/lib/rdoc/rdoc.rb:540:in `<main>'
    /Users/drbrain/Work/git/rdoc/lib/rdoc/test_case.rb:44:in `setup'
    /Users/drbrain/Work/git/rdoc/test/test_rdoc_generator_json_index.rb:8:in `setup'

[further errors are a result of the SyntaxError]

10 tests, 0 assertions, 0 failures, 11 errors, 0 skips

Test run options: --seed 14461

I worked around the compile error (by changing the first grouping to (?:.{8}*?)), and filed macruby/macruby#169. I then ran into macruby/macruby#170 where /\p{^Word}/ is not supported (used for markdown support, I don't know of a workaround).

Since I'm unable to verify this patch, I can't apply it. Since I'm unable to run the tests due to MacRuby bugs, I'm unable to investigate a different approach.

Removing the dependency would break RDoc for Ruby 1.8.7. This can happen after I drop 1.8.7 support since all supported versions of Ruby will have it built-in, but I wasn't planning on doing that until after Ruby 2.0.0 was released.

Changing the dependency is also not possible. RDoc can't have dependencies that exist outside of the latest release of Ruby.

I think the best way to fix this is in the railties gem. Since you are running a rails app on MacRuby and aren't seeing the syntax errors I see when running the tests, RDoc must not be loaded. I'll bring this up with @tenderlove, maybe it will be sufficient for rdoc to be a dependency in the default gem set for a generated rails application.

@drbrain
Owner

Closing due to Ruby and MacRuby incompatibilities ☹

@drbrain drbrain closed this
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Aug 23, 2012
  1. @nbudin
Commits on Aug 24, 2012
  1. @nbudin
  2. @nbudin
  3. @nbudin

    Add json.rb to Manifest.txt

    nbudin authored
  4. @nbudin

    Drop specific json libraries from the requirements now that rdoc/json…

    nbudin authored
    ….rb will automatically choose one
  5. @nbudin

    Remove multi_json requires

    nbudin authored
This page is out of date. Refresh to see the latest.
View
1  Manifest.txt
@@ -85,6 +85,7 @@ lib/rdoc/generator/template/json_index/js/navigation.js
lib/rdoc/generator/template/json_index/js/searcher.js
lib/rdoc/ghost_method.rb
lib/rdoc/include.rb
+lib/rdoc/json.rb
lib/rdoc/known_classes.rb
lib/rdoc/markdown.kpeg
lib/rdoc/markdown.rb
View
1  Rakefile
@@ -59,7 +59,6 @@ Depending on your version of ruby, you may need to install ruby rdoc/ri data:
self.kpeg_flags = '-fsv' if self.respond_to? :kpeg_flags= # no plugin
require_ruby_version '>= 1.8.7'
- extra_deps << ['json', '~> 1.4']
extra_dev_deps << ['racc', '~> 1.4']
extra_dev_deps << ['minitest', '~> 2']
extra_dev_deps << ['ZenTest', '~> 4']
View
2  lib/rdoc.rb
@@ -185,5 +185,7 @@ def self.load_yaml
autoload :Extend, 'rdoc/extend'
autoload :Require, 'rdoc/require'
+ autoload :JSON, 'rdoc/json'
+
end
View
5 lib/rdoc/generator/json_index.rb
@@ -1,5 +1,3 @@
-require 'json'
-
##
# The JsonIndex generator is designed to complement an HTML generator and
# produces a JSON search index. This generator is derived from sdoc by
@@ -132,8 +130,7 @@ def generate
index_file.open 'w', 0644 do |io|
io.set_encoding Encoding::UTF_8 if Object.const_defined? :Encoding
io.write 'var search_data = '
-
- JSON.dump data, io, 0
+ io.write RDoc::JSON.dump(data)
end
Dir.chdir @template_dir do
View
18 lib/rdoc/json.rb
@@ -0,0 +1,18 @@
+# A workaround to allow RDoc to work on MacRuby, which has a known incompatibility with the json gem.
+
+module RDoc
+ ruby_engine = defined?(RUBY_ENGINE) ? RUBY_ENGINE : "ruby"
+ case ruby_engine
+ when 'macruby' then
+ begin
+ require 'multi_json'
+ JSON = MultiJson
+ rescue LoadError
+ abort 'Unable to load multi_json. MacRuby cannot use json at this time, so please install multi_json by hand.'
+ end
+ else
+ gem 'json'
+ require 'json'
+ JSON = ::JSON
+ end
+end
View
10 lib/rdoc/text.rb
@@ -6,16 +6,6 @@
require 'strscan'
##
-# For RDoc::Text#snippet
-
-begin
- gem 'json'
-rescue Gem::LoadError
-end
-
-require 'json'
-
-##
# Methods for manipulating comment text
module RDoc::Text
View
4 test/test_rdoc_generator_json_index.rb
@@ -92,7 +92,7 @@ def test_generate
refute_empty assignment
- index = JSON.parse index
+ index = RDoc::JSON.load index
info = [
@klass.search_record[2..-1],
@@ -141,7 +141,7 @@ def test_generate_utf_8
index = $'
- index = JSON.parse index
+ index = RDoc::JSON.load index
klass_record = @klass.search_record[2..-1]
klass_record[-1] = "<p>5\xc2\xb0\n"
Something went wrong with that request. Please try again.