YAML: Can't load files with characters in the supplementary plane

[y8]

Loading YAML fails if they have a character in the supplementary plane:

jruby-9.1.5.0 :002 > require 'yaml'; YAML.load('೬')
 => "೬" 
jruby-9.1.5.0 :003 > require 'yaml'; YAML.load('1🎁')
Psych::SyntaxError: (): 'reader' unacceptable character ' at line 0 column 0
    from org/jruby/ext/psych/PsychParser.java:231:in `parse'
    from /home/glopes/.rvm/rubies/jruby-9.1.5.0/lib/ruby/stdlib/psych.rb:376:in `parse_stream'
    from /home/glopes/.rvm/rubies/jruby-9.1.5.0/lib/ruby/stdlib/psych.rb:324:in `parse'
    from /home/glopes/.rvm/rubies/jruby-9.1.5.0/lib/ruby/stdlib/psych.rb:251:in `load'
    from (irb):3:in `'
    from org/jruby/RubyKernel.java:995:in `eval'
    from org/jruby/RubyKernel.java:1296:in `loop'
    from org/jruby/RubyKernel.java:1115:in `catch'
    from org/jruby/RubyKernel.java:1115:in `catch'
    from /home/glopes/.rvm/rubies/jruby-9.1.5.0/bin/irb:13:in `'

Confusingly, it says the error is at line 0 column 0.

See original issue: ruby/psych#293

I can confirm this behaviour on 1.7.25, 9.1.5.0, 9.1.6.0 and 9.1.7.0

OS:

Darwin <> 16.4.0 Darwin Kernel Version 16.4.0: Thu Dec 22 22:53:21 PST 2016; root:xnu-3789.41.3~3/RELEASE_X86_64 x86_64

Java:

java version "1.8.0_73"
Java(TM) SE Runtime Environment (build 1.8.0_73-b02)
Java HotSpot(TM) 64-Bit Server VM (build 25.73-b02, mixed mode)

ENV:

LANG=en_US.UTF-8
LC_CTYPE=en_US.UTF-8
TERM=xterm-256color

[focusshifter]

Can confirm, got bitten by it while trying to parse emoji-powered YML.

Linux db25a270ce82 4.4.0-31-generic #50-Ubuntu SMP Wed Jul 13 00:07:12 UTC 2016
x86_64 Linux

java version "1.8.0_92"
Java(TM) SE Runtime Environment (build 1.8.0_92-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.92-b14, mixed mode)

LANG=C.UTF-8

[focusshifter]

Can confirm, got bitten by it while trying to parse emoji-powered YML.

Linux db25a270ce82 4.4.0-31-generic #50-Ubuntu SMP Wed Jul 13 00:07:12 UTC 2016
x86_64 Linux

java version "1.8.0_92"
Java(TM) SE Runtime Environment (build 1.8.0_92-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.92-b14, mixed mode)

LANG=C.UTF-8

[headius]

This is likely a bug in the SnakeYAML library we use. It's probably not handling surrogate pairs in its parser.

[headius]

This is likely a bug in the SnakeYAML library we use. It's probably not handling surrogate pairs in its parser.

[headius]

This appears to have been fixed in snakeyaml but there has been no release with that fix. I just requested one here: https://bitbucket.org/asomov/snakeyaml/issues/323/is-there-any-reason-to-not-support

[headius]

This appears to have been fixed in snakeyaml but there has been no release with that fix. I just requested one here: https://bitbucket.org/asomov/snakeyaml/issues/323/is-there-any-reason-to-not-support

[y8]

@asomov could you please save us by rolling out the release with a fix? Спасибо! 🤗

[y8]

@asomov could you please save us by rolling out the release with a fix? Спасибо! 🤗

[headius]

I'll keep this open until we have the psych release integrated into JRuby.

[headius]

I'll keep this open until we have the psych release integrated into JRuby.