JRuby 1.9 problem with Syck #649

Closed
mongrelion opened this Issue Aug 31, 2011 · 27 comments

Comments

Projects
None yet
7 participants

I've been trying to start up a project using jruby-1.6.3 but Padrino seems to be the kind of hard arse hitman that doesn't like Java

mongrelion@squives ➜  code   rvm jruby@padrino
mongrelion@squives ➜  code   padrino --version
Padrino v. 0.10.1
mongrelion@squives ➜  code   padrino g project jrack-test
Please specify generator to use (project, app, mailer, controller, model, migration, plugin)
mongrelion@squives ➜  code   jruby -S padrino g jrack-test    
Please specify generator to use (project, app, mailer, controller, model, migration, plugin)
mongrelion@squives ➜  code   padrino generate project jrack-test
Please specify generator to use (project, app, mailer, controller, model, migration, plugin)
mongrelion@squives ➜  code   jruby -S padrino generate jrack-test    
Please specify generator to use (project, app, mailer, controller, model, migration, plugin)


mongrelion@squives ➜  code   rvm jruby@padrino09
mongrelion@squives ➜  code   padrino --version
Padrino v. 0.9.29
mongrelion@squives ➜  code   padrino g project jrack-test
Please specify generator to use (project, app, mailer, controller, model, migration, plugin)
mongrelion@squives ➜  code   jruby -S padrino g jrack-test    
Please specify generator to use (project, app, mailer, controller, model, migration, plugin)
mongrelion@squives ➜  code   padrino generate project jrack-test
Please specify generator to use (project, app, mailer, controller, model, migration, plugin)
mongrelion@squives ➜  code   jruby -S padrino generate jrack-test    
Please specify generator to use (project, app, mailer, controller, model, migration, plugin)
Owner

nesquena commented Aug 31, 2011

@mongrelion I am curious does padrino-gen project jrack-test work? or the same message. Sorry to hear the generator isn't working with JRuby, it actually does pass the general tests on travis-ci though http://travis-ci.org/#!/padrino/padrino-framework which do include tests for the generators. Odd...

I just found the problem.
I have a line in my ~/.zshrc file with this:

# Run JRuby with support for 1.9
export JRUBY_OPTS=--1.9

If I unset that variable, then padrino generates the project immediately.

Owner

nesquena commented Aug 31, 2011

You should be doing padrino g project rack-test and I am trying in 1.9.2 as well on Mac OS X and it appears to be generating fine...Does it still fail even when you do padrino g project rack-test on 1.9.2. For the ones you printed there that is what you should see (i.e warning telling you to pick a generator)

Owner

nesquena commented Aug 31, 2011

Yeah, so the problem stands though on JRuby with JRUBY_OPTS=--1.9 enabled, this problem persists even when using padrino g project rack-test or padrino-gen project rack-test?

mongrelion@squives ➜  code   echo $JRUBY_OPTS
--1.9
mongrelion@squives ➜  code   padrino g project jrack-test
Please specify generator to use (project, app, mailer, controller, model, migration, plugin)
mongrelion@squives ➜  code   padrino-gen project jrack-test
      create  
      create  .gitignore
      create  config.ru
      create  config/apps.rb
      create  config/boot.rb
      create  public/favicon.ico
      create  public/images
      create  public/javascripts
      create  public/stylesheets
      create  tmp
      create  .components
NameError: uninitialized constant Syck::Yecht
               const_missing at org/jruby/RubyModule.java:2569
                    allocate at org/jruby/RubyClass.java:218
                     emitter at /Users/mongrelion/.rvm/rubies/jruby-1.6.3/lib/ruby/1.9/syck.rb:102
                  quick_emit at /Users/mongrelion/.rvm/rubies/jruby-1.6.3/lib/ruby/1.9/syck.rb:399
                     to_yaml at /Users/mongrelion/.rvm/rubies/jruby-1.6.3/lib/ruby/1.9/syck/rubytypes.rb:40
      store_component_config at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/padrino-gen-0.10.1/lib/padrino-gen/generators/actions.rb:86
                        call at org/jruby/RubyProc.java:256
                      render at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/thor-0.14.6/lib/thor/actions/create_file.rb:54
                     invoke! at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/thor-0.14.6/lib/thor/actions/create_file.rb:63
                        open at org/jruby/RubyIO.java:1111
                     invoke! at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/thor-0.14.6/lib/thor/actions/create_file.rb:63
                        call at org/jruby/RubyProc.java:256
  invoke_with_conflict_check at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/thor-0.14.6/lib/thor/actions/empty_directory.rb:114
                     invoke! at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/thor-0.14.6/lib/thor/actions/create_file.rb:61
                      action at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/thor-0.14.6/lib/thor/actions.rb:95
                 create_file at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/thor-0.14.6/lib/thor/actions/create_file.rb:26
      store_component_config at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/padrino-gen-0.10.1/lib/padrino-gen/generators/actions.rb:85
               setup_project at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/padrino-gen-0.10.1/lib/padrino-gen/generators/project.rb:57
                    __send__ at org/jruby/RubyBasicObject.java:1684
                        send at org/jruby/RubyKernel.java:2113
                         run at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/thor-0.14.6/lib/thor/task.rb:21
                 invoke_task at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/thor-0.14.6/lib/thor/invocation.rb:118
                  invoke_all at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/thor-0.14.6/lib/thor/invocation.rb:124
                        each at org/jruby/RubyHash.java:1175
                         map at org/jruby/RubyEnumerable.java:729
                  invoke_all at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/thor-0.14.6/lib/thor/invocation.rb:124
                    dispatch at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/thor-0.14.6/lib/thor/group.rb:226
                       start at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/thor-0.14.6/lib/thor/base.rb:389
                       setup at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/padrino-gen-0.10.1/lib/padrino-gen/generators/cli.rb:46
                    __send__ at org/jruby/RubyBasicObject.java:1684
                        send at org/jruby/RubyKernel.java:2113
                         run at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/thor-0.14.6/lib/thor/task.rb:21
                 invoke_task at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/thor-0.14.6/lib/thor/invocation.rb:118
                  invoke_all at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/thor-0.14.6/lib/thor/invocation.rb:124
                        each at org/jruby/RubyHash.java:1175
                         map at org/jruby/RubyEnumerable.java:729
                  invoke_all at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/thor-0.14.6/lib/thor/invocation.rb:124
                    dispatch at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/thor-0.14.6/lib/thor/group.rb:226
                       start at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/thor-0.14.6/lib/thor/base.rb:389
                      (root) at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/padrino-gen-0.10.1/bin/padrino-gen:16
                        load at org/jruby/RubyKernel.java:1073
                      (root) at /Users/mongrelion/.rvm/gems/jruby-1.6.3/bin/padrino-gen:19

mongrelion@squives ➜  code   jruby -S padrino g project jrack-test
Please specify generator to use (project, app, mailer, controller, model, migration, plugin)
mongrelion@squives ➜  code   jruby -S padrino-gen project jrack-test
  create  
  create  .gitignore
  create  config.ru
  create  config/apps.rb
  create  config/boot.rb
  create  public/favicon.ico
  create  public/images
  create  public/javascripts
  create  public/stylesheets
  create  tmp
  create  .components
NameError: uninitialized constant Syck::Yecht
           const_missing at org/jruby/RubyModule.java:2569
                allocate at org/jruby/RubyClass.java:218
                 emitter at /Users/mongrelion/.rvm/rubies/jruby-1.6.3/lib/ruby/1.9/syck.rb:102
              quick_emit at /Users/mongrelion/.rvm/rubies/jruby-1.6.3/lib/ruby/1.9/syck.rb:399
                 to_yaml at /Users/mongrelion/.rvm/rubies/jruby-1.6.3/lib/ruby/1.9/syck/rubytypes.rb:40
  store_component_config at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/padrino-gen-0.10.1/lib/padrino-gen/generators/actions.rb:86
                    call at org/jruby/RubyProc.java:256
                  render at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/thor-0.14.6/lib/thor/actions/create_file.rb:54
                 invoke! at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/thor-0.14.6/lib/thor/actions/create_file.rb:63
                    open at org/jruby/RubyIO.java:1111
                 invoke! at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/thor-0.14.6/lib/thor/actions/create_file.rb:63
                    call at org/jruby/RubyProc.java:256
invoke_with_conflict_check at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/thor-0.14.6/lib/thor/actions/empty_directory.rb:114
                 invoke! at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/thor-0.14.6/lib/thor/actions/create_file.rb:61
                  action at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/thor-0.14.6/lib/thor/actions.rb:95
             create_file at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/thor-0.14.6/lib/thor/actions/create_file.rb:26
  store_component_config at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/padrino-gen-0.10.1/lib/padrino-gen/generators/actions.rb:85
           setup_project at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/padrino-gen-0.10.1/lib/padrino-gen/generators/project.rb:57
                __send__ at org/jruby/RubyBasicObject.java:1684
                    send at org/jruby/RubyKernel.java:2113
                     run at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/thor-0.14.6/lib/thor/task.rb:21
             invoke_task at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/thor-0.14.6/lib/thor/invocation.rb:118
              invoke_all at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/thor-0.14.6/lib/thor/invocation.rb:124
                    each at org/jruby/RubyHash.java:1175
                     map at org/jruby/RubyEnumerable.java:729
              invoke_all at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/thor-0.14.6/lib/thor/invocation.rb:124
                dispatch at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/thor-0.14.6/lib/thor/group.rb:226
                   start at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/thor-0.14.6/lib/thor/base.rb:389
                   setup at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/padrino-gen-0.10.1/lib/padrino-gen/generators/cli.rb:46
                __send__ at org/jruby/RubyBasicObject.java:1684
                    send at org/jruby/RubyKernel.java:2113
                     run at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/thor-0.14.6/lib/thor/task.rb:21
             invoke_task at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/thor-0.14.6/lib/thor/invocation.rb:118
              invoke_all at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/thor-0.14.6/lib/thor/invocation.rb:124
                    each at org/jruby/RubyHash.java:1175
                     map at org/jruby/RubyEnumerable.java:729
              invoke_all at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/thor-0.14.6/lib/thor/invocation.rb:124
                dispatch at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/thor-0.14.6/lib/thor/group.rb:226
                   start at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/thor-0.14.6/lib/thor/base.rb:389
                  (root) at /Users/mongrelion/.rvm/gems/jruby-1.6.3/gems/padrino-gen-0.10.1/bin/padrino-gen:16
                    load at org/jruby/RubyKernel.java:1073
                  (root) at /Users/mongrelion/.rvm/gems/jruby-1.6.3/bin/padrino-gen:19
Owner

nesquena commented Aug 31, 2011

Hmm OK, looks like something to do with uninitialized constant Syck::Yecht is the source of the issue now even when generation does work. So weird, as I said we test project generation as part of our unit tests and they all pass on JRuby. What happens if you remove the $JRUBY_OPTS 1.9 flag? Does it work then?

Yes, it works pretty good.
mongrelion@squives ➜ code echo $JRUBY_OPTS
--1.9
mongrelion@squives ➜ code unset JRUBY_OPTS
mongrelion@squives ➜ code padrino g project jrack-test
create
create .gitignore
create config.ru
create config/apps.rb
create config/boot.rb
create public/favicon.ico
create public/images
create public/javascripts
create public/stylesheets
create tmp
create .components
create app
create app/app.rb
create app/controllers
create app/helpers
create app/views
create app/views/layouts
create Gemfile
Skipping generator for orm component...
Skipping generator for test component...
Skipping generator for mock component...
Skipping generator for script component...
Applying 'haml' (renderer)...
apply renderers/haml
insert Gemfile
Skipping generator for stylesheet component...
identical .components

=================================================================
jrack-test is ready for development!
=================================================================
$ cd ./jrack-test
$ bundle install
=================================================================
Owner

nesquena commented Aug 31, 2011

Interesting, something about 1.9 flags on JRuby and the Syck parser. OK Good to know thanks for the investigation.

@ghost ghost assigned DAddYE and nesquena Sep 1, 2011

Owner

nesquena commented Sep 2, 2011

@mongrelion BTW can you retry with 1.6.4? I think they fixed some YAML issues

@nesquena I retried with JRuby 1.6.4, creating a new gemset just to keep it clean and the problem persists.

Owner

nesquena commented Sep 5, 2011

Thanks for testing mongrelion. That's too bad.

Owner

nesquena commented Sep 12, 2011

@headius @enebo Any thoughts on this error: uninitialized constant Syck::Yecht and the easiest way for us to resolve this. Only seems to be an issue for 1.9 flags in JRUBY_OPTS

enebo commented Sep 12, 2011

I would hazard a guess that the code in 1.9/yaml/syck.rb on line 18 or so is not getting called because you hit the code in your stacktrace before this is getting required. I don't have a soln off-hand (getting on plane soon), but you can play with requires to see if it goes away (regardless of whether you get this to work it appears to be a bug in JRuby). Anything you can find on this should help us narrow down a solution for it.

enebo commented Sep 13, 2011

I would hazard a guess that the code in 1.9/yaml/syck.rb on line 18 or so is not getting called because you hit the code in your stacktrace before this is getting required. I don't have a soln off-hand (getting on plane soon), but you can play with requires to see if it goes away (regardless of whether you get this to work it appears to be a bug in JRuby). Anything you can find on this should help us narrow down a solution for it.

headius commented Sep 13, 2011

Is there something in padrino that tries to use "Syck" specifically? In 1.9 mode we do not use the "Yecht" port of "Syck", preferring the YAML 1.1-compliant "Psych" API we ported to wrap the SnakeYAML library. If Syck/Yecht is getting loaded, it could certainly explain the issues here.

headius commented Sep 13, 2011

Is there a reason you have to use Syck? We have gotten away with not shipping Syck in 1.9 mode up to now, and we've been very happy with that...Syck is a broken API (not compliant with any YAML version) and our implementation was always only about 95% correct. Psych is the way everything should be going, if possible.

What are you doing that requires Syck?

Member

skade commented Sep 13, 2011

As far as I know, Psych doesn't handle things like [1] correctly and that was a quick fix for Bundler suddenly activating Psych without any user interaction and leading to problems:

[1] https://github.com/padrino/padrino-framework/blob/master/padrino-core/lib/padrino-core/locale/de.yml#L16

I don't see a reason for keeping this behavior.

@skade skade closed this Sep 13, 2011

@skade skade reopened this Sep 13, 2011

Code should be fixed to comply with the yaml spec and work correctly with pysch.
Breaking things because you forced ruby to use syck in 1.9 is just wrong.

headius commented Sep 13, 2011

@skade Are you saying you don't see a reason to keep the Bundler behavior of switching to Psych, or the padrino-framework behavior of turning Syck back on?

In JRuby, there's basically only one YAML engine in 1.9 mode: Psych. Syck will not load properly and needs a good bit of work to support 1.9 mode the same as 1.8 mode. If there's a YAML incompatibility, the YAML itself should be fixed...we need to get away from using non-compliant YAML all over the place.

I sympathize with the issue, but continuing to use Syck is not the answer.

Member

skade commented Sep 13, 2011

@headius I see no reason for keeping the Syck hack. Me closing the issue was just my mouse hitting the wrong button.

Member

skade commented Sep 13, 2011

@headius: I have a branch here:

 https://github.com/padrino/padrino-framework/tree/psych-fix

But it seems JRuby --1.9 chokes on our czech locale, psych loads it just fine:

> jruby --1.9 -S irb
jruby-1.6.3 :001 > require 'yaml'
 => true 
jruby-1.6.3 :002 > YAML.load File.read 'padrino-core/lib/padrino-core/locale/cs.yml'
Java::OrgYamlSnakeyamlReader::ReaderException: special characters are not allowed
    from org.yaml.snakeyaml.reader.StreamReader.checkPrintable(StreamReader.java:98)
    from org.yaml.snakeyaml.reader.StreamReader.update(StreamReader.java:191)
    from org.yaml.snakeyaml.reader.StreamReader.<init>(StreamReader.java:63)
    from org.jruby.ext.psych.PsychParser.parse(PsychParser.java:101)
    from org.jruby.ext.psych.PsychParser$i$1$0$parse.call(PsychParser$i$1$0$parse.gen:65535)
    from org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:

I'm not an expert in YAML. Is this an issue with our YAML or a JRuby issue?

Member

skade commented Sep 13, 2011

BTW, in theory, we have test coverage for this :

$ [padrino-core] export JRUBY_OPTS="--1.9"
$ [padrino-core] rake test TEST=test/test_locale.rb

Interestingly, the test suite uses YAML.load_file and passes:

$ jruby --1.9 -S irb
> YAML.load_file 'lib/padrino-core/locale/cs.yml'
# perfect!
Member

skade commented Sep 13, 2011

Okay, fix is up for review here:

#669

Thanks to everyone who pushed this and brought it to our attention again.

Owner

nesquena commented Sep 13, 2011

Yes thanks for everyones input, I completely agree that the right approach is to make our YAML compliant. Thanks to skade for putting together the pull request.

Owner

nesquena commented Sep 13, 2011

Fix committed here: e85b5ce

@mongrelion Can you confirm with edge Padrino that JRuby (1.9) works as expected?

@nesquena nesquena closed this Sep 13, 2011

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment