JRuby performance regression for `CSV.parse_line`

[donoghuc]

When run with JRuby, CSV.parse_line (and any CSV#each iteration) has a noticeable performance regression because CSV#each pulls rows one at a time through Enumerator#next.

CSV.parse_line is defined as new(line, **options).each.first

csv/lib/csv.rb

Lines 1898 to 1899 in b36c160

	def parse_line(line, **options)
	new(line, **options).each.first

CSV#each iterates by repeatedly calling parser_enumerator.next

csv/lib/csv.rb

Lines 2689 to 2697 in b36c160

	def each(&block)
	return to_enum(__method__) unless block_given?
	begin
	while true
	yield(parser_enumerator.next)
	end
	rescue StopIteration
	end
	end

def each(&block)
  return to_enum(__method__) unless block_given?
  begin
    while true
      yield(parser_enumerator.next)
    end
  rescue StopIteration
  end
end

parser_enumerator is parser.parse (an Enumerator

csv/lib/csv.rb

Lines 2974 to 2975 in b36c160

	def parser_enumerator
	@parser_enumerator ||= parser.parse

Per the Ruby documentation, external iteration with next "differs significantly from internal iteration due to using a Fiber" and "the Fiber adds some overhead compared to internal enumeration" https://docs.ruby-lang.org/en/3.4/Enumerator.html

Change that introduced it

CSV#each previously called parser_enumerator.each(&block) (internal iteration, no Fiber). It was changed to the Enumerator#next loop in commit acc05116c55579b78925dd27024969b9daf69135 ("All Enumerable based methods consume the same lines with other methods", acc0511), to fix GH-260 (#260). That commit message itself notes: "This may have a performance penalty. We should benchmark this." It was first released in v3.2.6

Reproducer

require "csv"
require "benchmark"

ROW = (1..30).map { |i| "field_#{i}" }.join(",")
N = 50_000
def rate(n, s) = (n / s).round

N.times { CSV.parse_line(ROW) }
parse_line = rate(N, Benchmark.realtime { N.times { CSV.parse_line(ROW) } })
en = rate(N, Benchmark.realtime { N.times { Enumerator.new { |y| y << 1 }.next } })
eb = rate(N, Benchmark.realtime { N.times { Enumerator.new { |y| y << 1 }.each { |v| break v } } })

engine = defined?(JRUBY_VERSION) ? "JRuby #{JRUBY_VERSION}" : RUBY_ENGINE
puts "#{engine}  Ruby #{RUBY_VERSION}  csv #{CSV::VERSION}"
puts "CSV.parse_line             : #{parse_line} calls/sec"
puts "bare Enumerator#next       : #{en} calls/sec"
puts "bare Enumerator#each+break : #{eb} calls/sec"

Output, JRuby 10.0.5.0 (via jruby-complete-10.0.5.0.jar, csv 3.3.5):

JRuby 10.0.5.0  Ruby 3.4.5  csv 3.3.5
CSV.parse_line             : 12270 calls/sec
bare Enumerator#next       : 27411 calls/sec
bare Enumerator#each+break : 447421 calls/sec

Output, MRI CRuby 3.4.10 (csv 3.3.5), for contrast:

ruby  Ruby 3.4.10  csv 3.3.5
CSV.parse_line             : 46946 calls/sec
bare Enumerator#next       : 291157 calls/sec
bare Enumerator#each+break : 1159496 calls/sec