Ruby 2.7 Support

[georgie84]

Ruby 2.7 Support

This issue will track Ruby 2.7 support for JRuby 9.4

This checklist is based on MRI's NEWS file: https://github.com/ruby/ruby/blob/ruby_2_7/NEWS

I removed the clear C API related issues, but there should still be a few.

Changes since the 2.6.0 release

Language changes

Pattern matching

• Pattern matching is introduced as an experimental feature. [Feature
  #14912]

    case [0, [1, 2, 3]]
    in [a, [b, *c]]
      p a #=> 0
      p b #=> 1
      p c #=> [2, 3]
    end
  
    case {a: 0, b: 1}
    in {a: 0, x: 1}
      :unreachable
    in {a: 0, b: var}
      p var #=> 1
    end
  
    case -1
    in 0 then :unreachable
    in 1 then :unreachable
    end #=> NoMatchingPatternError
  
    json = <<END
    {
      "name": "Alice",
      "age": 30,
      "children": [{ "name": "Bob", "age": 2 }]
    }
    END
  
    JSON.parse(json, symbolize_names: true) in {name: "Alice", children: [{name: name, age: age}]}
  
    p name #=> "Bob"
    p age  #=> 2
  
    JSON.parse(json, symbolize_names: true) in {name: "Alice", children: [{name: "Charlie", age: age}]}
    #=> NoMatchingPatternError
  

• See the following slides for more details:

  • https://speakerdeck.com/k_tsj/pattern-matching-new-feature-in-ruby-2-d
    ot-7
  • Note that the slides are slightly obsolete.

• The warning against pattern matching can be suppressed with
  -W:no-experimental option. (Add support for -W:category flag #7436)

The spec of keyword arguments is changed towards 3.0

• Automatic conversion of keyword arguments and positional arguments is
  deprecated, and conversion will be removed in Ruby 3. [Feature #14183]

  • When a method call passes a Hash at the last argument, and when it
    passes no keywords, and when the called method accepts keywords, a
    warning is emitted. To continue treating the hash as keywords, add a
    double splat operator to avoid the warning and ensure correct behavior
    in Ruby 3.

      def foo(key: 42); end; foo({key: 42})   # warned
      def foo(**kw);    end; foo({key: 42})   # warned
      def foo(key: 42); end; foo(**{key: 42}) # OK
      def foo(**kw);    end; foo(**{key: 42}) # OK
    

  • When a method call passes keywords to a method that accepts keywords,
    but it does not pass enough required positional arguments, the
    keywords are treated as a final required positional argument, and a
    warning is emitted. Pass the argument as a hash instead of keywords
    to avoid the warning and ensure correct behavior in Ruby 3.

      def foo(h, **kw); end; foo(key: 42)      # warned
      def foo(h, key: 42); end; foo(key: 42)   # warned
      def foo(h, **kw); end; foo({key: 42})    # OK
      def foo(h, key: 42); end; foo({key: 42}) # OK
    

  • When a method accepts specific keywords but not a keyword splat, and a
    hash or keywords splat is passed to the method that includes both
    Symbol and non-Symbol keys, the hash will continue to be split, and a
    warning will be emitted. You will need to update the calling code to
    pass separate hashes to ensure correct behavior in Ruby 3.

      def foo(h={}, key: 42); end; foo("key" => 43, key: 42)   # warned
      def foo(h={}, key: 42); end; foo({"key" => 43, key: 42}) # warned
      def foo(h={}, key: 42); end; foo({"key" => 43}, key: 42) # OK
    

  • If a method does not accept keywords, and is called with keywords, the
    keywords are still treated as a positional hash, with no warning. This
    behavior will continue to work in Ruby 3.

     def foo(opt={});  end; foo( key: 42 )   # OK
    

• Non-symbols are allowed as keyword argument keys if the method accepts
  arbitrary keywords. [Feature #14183]

  • Non-Symbol keys in a keyword arguments hash were prohibited in 2.6.0,
    but are now allowed again. [Bug #15658]

      def foo(**kw); p kw; end; foo("str" => 1) #=> {"str"=>1}
    

• **nil is allowed in method definitions to explicitly mark that the
  method accepts no keywords. Calling such a method with keywords will
  result in an ArgumentError. [Feature #14183]

    def foo(h, **nil); end; foo(key: 1)       # ArgumentError
    def foo(h, **nil); end; foo(**{key: 1})   # ArgumentError
    def foo(h, **nil); end; foo("str" => 1)   # ArgumentError
    def foo(h, **nil); end; foo({key: 1})     # OK
    def foo(h, **nil); end; foo({"str" => 1}) # OK
  

• Passing an empty keyword splat to a method that does not accept keywords
  no longer passes an empty hash, unless the empty hash is necessary for a
  required parameter, in which case a warning will be emitted. Remove the
  double splat to continue passing a positional hash. [Feature #14183]

    h = {}; def foo(*a) a end; foo(**h) # []
    h = {}; def foo(a) a end; foo(**h)  # {} and warning
    h = {}; def foo(*a) a end; foo(h)   # [{}]
    h = {}; def foo(a) a end; foo(h)    # {}
  

• Above warnings can be suppressed also with -W:no-deprecated
  option. (Add support for -W:category flag #7436)

Numbered parameters

• Numbered parameters as default block parameters are introduced.
  [Feature Activating refinements in binding.eval doesn't stick outside of the evaluated string. #4475]

  [1, 2, 10].map { _1.to_s(16) } #=> ["1", "2", "a"]
  [[1, 2], [3, 4]].map { _1 + _2 } #=> [3, 7]

  You can still define a local variable named _1 and so on, and that is
  honored when present, but renders a warning.

    _1 = 0            #=> warning: `_1' is reserved for numbered parameter; consider another name
    [1].each { p _1 } # prints 0 instead of 1
  

proc/lambda without block is deprecated

• Proc.new and Kernel#proc with no block in a method called with a block is
  warned now.

    def foo
      proc
    end
    foo { puts "Hello" } #=> warning: Capturing the given block using Kernel#proc is deprecated; use `&block` instead
  

  This warning can be suppressed with -W:no-deprecated
  option.

• Kernel#lambda with no block in a method called with a block raises an
  exception.

    def bar
      lambda
    end
    bar { puts "Hello" } #=> tried to create Proc object without a block (ArgumentError)
  

Other miscellaneous changes

• A beginless range is experimentally introduced. It might be useful in
  case, new call-sequence of the Comparable#clamp, constants and DSLs.
  [Feature #14799]

    ary[..3]  # identical to ary[0..3]
  
    case RUBY_VERSION
    when ..."2.4" then puts "EOL"
    # ...
    end
  
    age.clamp(..100)
  
    where(sales: ..100)
  

• Setting $; to a non-nil value is warned now. [Feature #14240] Use of it
  in String#split is warned too. This warning can be suppressed with
  -W:no-deprecated option.

• Setting $, to a non-nil value is warned now. [Feature #14240] Use of it
  in Array#join is warned too. This warning can be suppressed with
  -W:no-deprecated option.

• Quoted here-document identifiers must end within the same line.

    <<"EOS
    " # This had been warned since 2.4; Now it raises a SyntaxError
    EOS
  

• The flip-flop syntax deprecation is reverted. [Feature Time.new with explicit timezone string will display as UTF with +00:00. #5400]

• Comment lines can be placed between fluent dot now.

    foo
      # .bar
      .baz # => foo.baz
  

• Calling a private method with a literal self as the receiver is now
  allowed. [Feature #11297] [Feature #16123]

• Modifier rescue now operates the same for multiple assignment as single
  assignment. [Bug #8279]

    a, b = raise rescue [1, 2]
    # Previously parsed as: (a, b = raise) rescue [1, 2]
    # Now parsed as:         a, b = (raise rescue [1, 2])
  

• yield in singleton class syntax is warned and will be deprecated later.
  [Feature #15575].

    def foo
      class << Object.new
        yield #=> warning: `yield' in class syntax will not be supported from Ruby 3.0. [Feature #15575]
      end
    end
    foo { p :ok }
  

  This warning can be suppressed with -W:no-deprecated
  option.

• Argument forwarding by (...) is introduced. [Feature #16253]

    def foo(...)
      bar(...)
    end
  

  All arguments to foo are forwarded to bar, including keyword and block
  arguments. Note that the parentheses are mandatory. bar ... is parsed
  as an endless range.

• Access and setting of $SAFE is now always warned. $SAFE will become a
  normal global variable in Ruby 3.0. [Feature #16131]

• Object#{taint,untaint,trust,untrust} and related functions in the C-API
  no longer have an effect (all objects are always considered untainted),
  and are now warned in verbose mode. This warning will be disabled even in
  non-verbose mode in Ruby 3.0, and the methods and C functions will be
  removed in Ruby 3.2. [Feature #16131]

• Refinements take place at Object#method and Module#instance_method.
  [Feature #15373]

Command line options

Warning option

The -W option has been extended with a following :, to manage categorized
warnings. [Feature #16345] [Feature #16420] (#7436)

• To suppress deprecation warnings:

    $ ruby -e '$; = ""'
    -e:1: warning: `$;' is deprecated
  
    $ ruby -W:no-deprecated -e '$; = //'
  

• It works with the RUBYOPT environment variable:

    $ RUBYOPT=-W:no-deprecated ruby -e '$; = //'
  

• To suppress experimental feature warnings:

    $ ruby -e '0 in a'
    -e:1: warning: Pattern matching is experimental, and the behavior may change in future versions of Ruby!
  
    $ ruby -W:no-experimental -e '0 in a'
  

• To suppress both by using RUBYOPT, set space separated values:

    $ RUBYOPT='-W:no-deprecated -W:no-experimental' ruby -e '($; = "") in a'
  

See also Warning in [Core classes updates]

Core classes updates (outstanding ones only)

• Added Array#intersection. [Feature #16155]

• Added Array#minmax, with a faster implementation than
  Enumerable#minmax. Bug #15929, add Array#minmax #6939

• Comparable#clamp now accepts a Range argument. [Feature #14784]

            -1.clamp(0..2) #=> 0
             1.clamp(0..2) #=> 1
             3.clamp(0..2) #=> 2
            # With beginless and endless ranges:
            -1.clamp(0..)  #=> 0
             3.clamp(..2)  #=> 2
  

• Added Complex#<=>. So 0 <=> 0i will not raise NoMethodError.
  [Bug #15857]

• Dir.glob and Dir.[] no longer allow NUL-separated glob pattern.
  Use Array instead. [Feature #14643]

• Added new encoding CESU-8. [Feature #15931]

• Added Enumerable#filter_map. [Feature #15323]

            [1, 2, 3].filter_map {|x| x.odd? ? x.to_s : nil } #=> ["1", "3"]
  

• Added Enumerable#tally. [Feature #11076]

            ["A", "B", "C", "B", "A"].tally #=> {"A"=>2, "B"=>2, "C"=>1}
  

• Added Enumerator.produce to generate an Enumerator from any custom
  data transformation. [Feature #14781]

            require "date"
            dates = Enumerator.produce(Date.today, &:succ) #=> infinite sequence of dates
            dates.detect(&:tuesday?) #=> next Tuesday
  

• Added Enumerator::Lazy#eager that generates a non-lazy enumerator
  from a lazy enumerator. [Feature #15901]

            a = %w(foo bar baz)
            e = a.lazy.map {|x| x.upcase }.map {|x| x + "!" }.eager
            p e.class               #=> Enumerator
            p e.map {|x| x + "?" }  #=> ["FOO!?", "BAR!?", "BAZ!?"]
  

• Added Enumerator::Yielder#to_proc so that a Yielder object can be
  directly passed to another method as a block argument. [Feature
  #15618]

• Added Fiber#raise that behaves like Fiber#resume but raises an
  exception on the resumed fiber. [Feature #10344]

• File.extname now returns a dot string for names ending with a dot
  on non-Windows platforms. [Bug #15267]

            File.extname("foo.") #=> "."
  

• Added FrozenError#receiver to return the frozen object on which
  modification was attempted. To set this object when raising
  FrozenError in Ruby code, FrozenError.new accepts a :receiver
  option. [Feature #15751]

• Added GC.compact method for compacting the heap. This function
  compacts live objects in the heap so that fewer pages may be used,
  and the heap may be more CoW (copy-on-write) friendly. [Feature
  #15626]

        Details on the algorithm and caveats can be found here:
        https://bugs.ruby-lang.org/issues/15626
  

• Added IO#set_encoding_by_bom to check the BOM and set the external
  encoding. [Bug #15210]

• Integer#[] now supports range operations. [Feature #8842]

            0b01001101[2, 4]  #=> 0b0011
            0b01001100[2..5]  #=> 0b0011
            0b01001100[2...6] #=> 0b0011
            #   ^^^^
  

• Method#inspect shows more information. [Feature #14145]

• Added Module#const_source_location to retrieve the location where
  a constant is defined. [Feature #10771]

• Added Module#ruby2_keywords for marking a method as passing
  keyword arguments through a regular argument splat, useful when
  delegating all arguments to another method in a way that can be
  backwards compatible with older Ruby versions. [Bug #16154]

• Module#autoload? now takes an inherit optional argument, like
  Module#const_defined?. [Feature #15777]

• Module#name now always returns a frozen String. The returned
  String is always the same for a given Module. This change is
  experimental. [Feature #16150]

• NilClass#to_s, TrueClass#to_s, and FalseClass#to_s now always
  return a frozen String. The returned String is always the same for
  each of these values. This change is experimental. [Feature
  #16150]

• ObjectSpace::WeakMap#[]= now accepts special objects as either key
  or values. [Feature #16035]

• Added Proc#ruby2_keywords for marking the proc as passing keyword
  arguments through a regular argument splat, useful when delegating
  all arguments to another method or proc in a way that can be
  backwards compatible with older Ruby versions. [Feature #16404]

• Added Range#minmax, with a faster implementation than
  Enumerable#minmax. It returns a maximum that now corresponds to
  Range#max. Bug #15807

• Range#=== now uses Range#cover? for String arguments, too (in Ruby
  2.6, it was changed from Range#include? for all types except
  strings). [Bug #15449]

• RubyVM.resolve_feature_path moved to
  $LOAD_PATH.resolve_feature_path. [Feature #15903] [Feature
  #15230]

• Update Unicode version and Emoji version from 11.0.0 to 12.0.0.
  [Feature #15321]

• Update Unicode version to 12.1.0, adding support for U+32FF SQUARE
  ERA NAME REIWA. [Feature #15195]

• Update Unicode Emoji version to 12.1. [Feature #16272]

• Added Symbol#start_with? and Symbol#end_with? methods. [Feature
  #16348]

• Added Time#ceil method. [Feature #15772]

• Added Time#floor method. [Feature #15653]

• Time#inspect is separated from Time#to_s and it shows the time's
  sub second. [Feature #15958]

• Added UnboundMethod#bind_call method. [Feature #15955]

        `umethod.bind_call(obj, ...)` is semantically equivalent to
        `umethod.bind(obj).call(...)`.  This idiom is used in some
        libraries to call a method that is overridden.  The added method
        does the same without allocation of an intermediate Method object.
  
            class Foo
              def add_1(x)
                x + 1
              end
            end
            class Bar < Foo
              def add_1(x) # override
                x + 2
              end
            end
  
            obj = Bar.new
            p obj.add_1(1) #=> 3
            p Foo.instance_method(:add_1).bind(obj).call(1) #=> 2
            p Foo.instance_method(:add_1).bind_call(obj, 1) #=> 2
  

• Added Warning.[] and Warning.[]= to manage emitting/suppressing
  some categories of warnings. [Feature #16345] [Feature #16420]

• Added $LOAD_PATH.resolve_feature_path. [Feature #15903]
  [Feature #15230]

Stdlib updates (outstanding ones only)

Bundler

• Upgrade to Bundler 2.1.2. See
  https://github.com/bundler/bundler/releases/tag/v2.1.2

CGI

• CGI.escapeHTML becomes 2~5x faster when there is at least one escaped
  character. See Optimize CGI.escapeHTML by reducing buffer extension and branches ruby/ruby#2226

CSV

• Upgrade to 3.1.2. See https://github.com/ruby/csv/blob/master/NEWS.md.

Date

• Date.jisx0301, Date#jisx0301, and Date.parse support the new Japanese
  era. [Feature #15742]

Delegator

• Object#DelegateClass accepts a block and module_evals it in the
  context of the returned class, similar to Class.new and Struct.new.

ERB

• Prohibit marshaling ERB instance.

IRB

• Introduce syntax highlighting inspired by the Pry gem to Binding#irb
  source lines, REPL input, and inspect output of some core-class
  objects.

• Introduce multiline editing mode provided by Reline.

• Show documentation when completion.

• Enable auto indent and save/load history by default.

JSON

• Upgrade to 2.3.0.

Net::FTP

• Add Net::FTP#features to check available features, and Net::FTP#option
  to enable/disable each of them. [Feature #15964]

Net::HTTP

• Add ipaddr optional parameter to Net::HTTP#start to replace the
  address for the TCP/IP connection. [Feature [ji] RubyString implements CharSequence #5180]

Net::IMAP

• Add Server Name Indication (SNI) support. [Feature #15594]

open-uri

• Warn open-uri's "open" method at Kernel. Use URI.open instead. [Misc
  #15893]

• The default charset of "text/*" media type is UTF-8 instead of
  ISO-8859-1. [Bug #15933]

OptionParser

• Now show "Did you mean?" for unknown options. [Feature #16256]

    test.rb:
  
        require "optparse"
        OptionParser.new do |opts|
          opts.on("-f", "--foo", "foo") {|v| }
          opts.on("-b", "--bar", "bar") {|v| }
          opts.on("-c", "--baz", "baz") {|v| }
        end.parse!
  
    example:
  
        $ ruby test.rb --baa
        Traceback (most recent call last):
        test.rb:7:in `<main>': invalid option: --baa (OptionParser::InvalidOption)
        Did you mean?  baz
                       bar
  

Pathname

• Pathname.glob now delegates 3 arguments to Dir.glob to accept base
  keyword. [Feature #14405]

Racc

• Merge 1.4.15 from upstream repository and added cli of racc.

Reline

• New stdlib that is compatible with the readline stdlib but is
  implemented in pure Ruby. It also provides a multiline editing mode.

Rexml

• Upgrade to 3.2.3. See https://github.com/ruby/rexml/blob/master/NEWS.md.

RSS

• Upgrade to RSS 0.2.8. See https://github.com/ruby/rss/blob/master/NEWS.md.

RubyGems

• Upgrade to RubyGems 3.1.2.

StringScanner

• Upgrade to 1.0.3. See https://github.com/ruby/strscan/blob/master/NEWS.md.

Compatibility issues (excluding feature bug fixes)

• The following libraries are no longer bundled gems. Install corresponding
  gems to use these features.

  • CMath (cmath gem)
  • Scanf (scanf gem)
  • Shell (shell gem)
  • Synchronizer (sync gem)
  • ThreadsWait (thwait gem)
  • E2MM (e2mmap gem)

• Proc

  • The Proc#to_s format was changed. [Feature #16101]

• Range

  • Range#minmax used to iterate on the range to determine the maximum. It
    now uses the same algorithm as Range#max. In rare cases (e.g. ranges
    of Floats or Strings), this may yield different results. [Bug #15807]

Stdlib compatibility issues (excluding feature bug fixes)

See #7015 for a complete listing of gemified stdlib in 9.4 and current status.

• Promote stdlib to default gems

  • The following default gems were published on rubygems.org

    • benchmark
    • cgi
    • delegate
    • getoptlong
    • net-pop
    • net-smtp
    • open3
    • pstore
    • readline
    • readline-ext
    • singleton

  • The following default gems were only promoted at ruby-core, but not
    yet published on rubygems.org.

    • monitor
    • observer
    • timeout
    • tracer
    • uri
    • yaml

  • The did_you_mean gem has been promoted up to a default gem from a
    bundled gem

• pathname

  • Kernel#Pathname when called with a Pathname argument now returns the
    argument instead of creating a new Pathname. This is more similar to
    other Kernel methods, but can break code that modifies the return
    value and expects the argument not to be modified.

• profile.rb

  • Removed from standard library. It was unmaintained since Ruby 2.0.0.

Implementation improvements

• Allow selecting different coroutine implementations by using
  --with-coroutine=, e.g.

        $ ./configure --with-coroutine=ucontext
        $ ./configure --with-coroutine=copy
  

• Replace previous stack cache with fiber pool cache. The fiber pool
  allocates many stacks in a single memory region. Stack allocation
  becomes O(log N) and fiber creation is amortized O(1). Around 10x
  performance improvement was measured in micro-benchmarks.
  Improve performance of fiber creation by using pool allocation strategy. ruby/ruby#2224

• File.realpath now uses realpath(3) on many platforms, which can
  significantly improve performance. [Feature #15797]

• Change data structure of small Hash objects. [Feature #15602]

• Monitor class is written in C-extension. [Feature #16255]

• VM stack memory allocation is now combined with native thread stack,
  improving thread allocation performance and reducing allocation
  related failures. Around 10x performance improvement was measured in
  micro-benchmarks.

• JIT-ed code is recompiled to less-optimized code when an optimization
  assumption is invalidated.

• Method inlining is performed when a method is considered as pure. This
  optimization is still experimental and many methods are NOT considered
  as pure yet.

• The default value of --jit-max-cache is changed from 1,000 to 100.

• The default value of --jit-min-calls is changed from 5 to 10,000.

• Per-call-site method cache, which has been there since around 1.9, was
  improved: cache hit rate raised from 89% to 94%. See
  reduce cache mishits ruby/ruby#2583

  Some methods are defined in *.rb (such as trace_point.rb). For example, it
  is easy to define a method which accepts keyword arguments.

[headius]

Looks like a great start on this checklist, thank you. I think some of these were nested items but didn't convert to markdown that way. Can you double-check the structure so we can be sure this checklist is easy to read?

[eregon]

Will JRuby 9.4 target Ruby 2.7 or Ruby 3.0?
https://rubyconf.org/program/sessions#session-1222 sounds like 3.0.
Is there an issue to track progress towards Ruby 3.0 compatibility?

[mohits]

Hi @eregon - I read in a statement by @enebo that 2.7 will be skipped and JRuby 9.4 will target Ruby 3. See: #6691

[headius]

There is no checklist yet for 3.0 features, but that would be a good step for us to take.

[headius]

@eregon I noticed you created oracle/truffleruby#2453 today with a 3.0 checklist. Since I am not a contributor, I can't access the raw markdown (seems weird, but ok), but it is basically the same format we have used for our past checklists. Could you copy it into a JRuby issue or post that markdown somewhere?

[headius]

@eregon Nevermind, I figured out a way to get the raw markdown (quote reply).

[eregon]

I think you can also copy it from ruby/spec#823 where you have write permissions.

[eregon]

Issue for Ruby 3.0 support is #6878

[headius]

GC.compact is added but does nothing (we do not have that level of control over JVM GCs).

[headius]

The optimized Array#minmax and Range#minmax would be pretty easy Java tasks for someone. They could be pure-Ruby, but that would not really gain us much over just using Enumerable#minmax.

Array#minmax is described in https://bugs.ruby-lang.org/issues/15929

Range#minmax is described in https://bugs.ruby-lang.org/issues/15807

[headius]

Comparable#clamp is there but does not appear to support endless or beginless ranges.

[ahorek]

Range#minmax (already implemented)

jruby/core/src/main/java/org/jruby/RubyRange.java

Line 1055 in 8030ff9

public IRubyObject minmax(ThreadContext context, Block block) {

Array#minmax #6939

[headius]

Updated based on current master.

[headius]

I think we can call this one done. The implementation improvements might be interesting for the performance work planned in 9.5, but we have implemented basically everything else in this list!

[thbar]

Extra information : the latest version of Nokogiri (1.14.0) requires "Ruby >= 2.7.0". Given the number of security issues on Nokogiri, this is one extra motivation to move forward!