Program that works under C-Ruby fails under JRuby #1098

Closed
bobjalex opened this Issue Oct 8, 2013 · 6 comments

Projects

None yet

2 participants

@bobjalex
bobjalex commented Oct 8, 2013

Always works under C-Ruby, fails under some Windows situations, works under others.

Using JRuby 1.7.5, Windows Vista 32-bit and Windows 7 64-bit, Java 1.7.0_40-b43 and Java 1.8.0-ea-b109. I currently don't have access to a Unix environment so haven't tested that.

Works OK under Vista 32-bit Java 1.7 both server and client, fails under both Vista 32-bit and Windows 7 64-bit under Java 1.8. I could try to enumerate all the permutations, but the program is command-line and easy to run, so I'm including it and you can see for yourselves.

To run it, just invoke the enclosed program with the -q option:

ruby auto_sol.rb -q

If running normally, will continuously output lines until ^C, Fails in various ways, such as hanging or "nil has no method empty?", etc. (The lines are outcomes of playing solitaire card games -- don't ask....)

This bug has no impact on me -- just sending it to help you perfect JRuby!

Embedding the program since I don't see a way to attach a file:

# Copyright (C) 2010, Robert J. Alexander. All rights reserved.

#
#  Automatic Klondike Solitaire card game
#

#
#  Possible strategies to experiment with sometime:
#
#  Hold off on playing cards on aces unless evenness of ace piles can
#  be maintained (e.g.  no ace pile should rank more than 1 higher
#  than any other) unless necessary to play another non-ace card (e.g.
#  from deck or on board).
#
#  Hold off on moving a pile that has no down cards until there is a
#  king to put in its slot.
#
#  Implement rule of playing single cards from top of a pile to top
#  of another (probably only good for getting to a particular card to
#  play on an ace).
#

def usage(s = nil)
  $stderr.puts("auto_sol: " + s) if s
  $stderr.puts(
    "Args: [-options]",
    "  -n n   number of games to play (default: infinite)",
    "  -v     print each play",
    "  -q     play quietly (opposite of -v)",
    "  -t     number of cards to turn (default: 3)",
    "  -p     number of passes through deck (0 means unlimited",
    "         (default: 1 if -t 1, otherwise unlimited)",
    "  -d n   between-play-delay[:inter-game-delay]",
    "         (in seconds -- fractions okay)",
    "         (default: wait for user response)",
    "  -s     print only final summary (default: outcome of each game)",
    "  -w     play only until a victory",
    "  -W     play until a victory and print winning deck",
    "  -D file  load initial deck from file",
    "  -S n   use n to seed random number generator for first game",
    "  -a     for video terminals supporting ANSI escapes",
    "  -R     print random seed after each game",
  )
  exit 2
end

def options()
  require 'getoptlong'
  usage() if ARGV.length == 0
  $n_games = nil
  quiet_opt = verbose = false
  $turn = 3
  $passes_allowed = nil
  $delay = nil
  $inter_game_delay = 3.0
  $summary = false
  $until_win = false
  $ansi = false
  $print_deck = false
  $deck_file = nil
  $seed = nil
  $print_random_seed = false
  no_arg, req_arg = GetoptLong::NO_ARGUMENT, GetoptLong::REQUIRED_ARGUMENT
  opts = GetoptLong.new(
    ["-v", no_arg],
    ["-q", no_arg],
    ["-t", req_arg],
    ["-p", req_arg],
    ["-d", req_arg],
    ["-n", req_arg],
    ["-s", no_arg],
    ["-w", no_arg],
    ["-W", no_arg],
    ["-D", req_arg],
    ["-S", req_arg],
    ["-a", no_arg],
    ["-R", no_arg])
  begin
    opts.each do |opt, value|
      case opt
      when "-n" then $n_games = Integer(value)
        when "-q" then quiet_opt = true
        when "-v" then verbose = true
        when "-t" then $turn = Integer(value)
        when "-p" then $passes_allowed = Integer(value)
        when "-d" then
          $delay, $inter_game_delay =
              value.split(':').collect {|x| x ?  Float(x) : nil}
          $inter_game_delay ||= 3.0
        when "-s" then $summary = true
        when "-w" then $until_win = true
        when "-W" then $until_win = $print_deck = true
        when "-D" then $deck_file = value
        when "-S" then $seed = Integer(value)
        when "-a" then $ansi = true
        when "-R" then $print_random_seed = true
        else raise
      end
    end
    if (quiet_opt && verbose) || !(quiet_opt || verbose)
      usage("must specify one of -v or -q")
    end
    $quiet = quiet_opt
    $print_random_seed = true unless quiet_opt
    unless $passes_allowed
      $passes_allowed = $turn > 1 ? 0 : 1
    end
    if $deck_file && !$n_games
      $n_games = 1
    end
    if $n_games && $n_games < 0
      $n_games = nil
    end
  ensure
  end
end

def init()
  options()
  $up_deck = nil
  $board = nil
end

def main()
  init()
  won = games = max_up_deck_cards = passes = 0
  print("\033[2J\033[H") if $ansi
  begin
    while true
    @say_number = 0
      stats = play_game
      outcome =
        if stats.win
          won += 1
          if max_up_deck_cards < stats.max_up_deck_cards
            max_up_deck_cards = stats.max_up_deck_cards
          end
          "Won "
        else
          "Lost"
        end
      if passes < stats.passes
        passes = stats.passes
      end
      games += 1
      unless $summary
        puts("#{outcome} (#{won}/#{games} = #{percent(won, games)}, turn #$turn" \
            "#{$passes_allowed == 1 ?
            ", max up cards: #{stats.max_up_deck_cards}" :
            ", #{stats.passes} pass#{stats.passes != 1 ? "es" : ""}"}" \
            "#{$print_random_seed ? ", seed: #{stats.seed}" : ""})")
        sleep($inter_game_delay) unless $quiet
      end
      break if ($n_games && ($n_games -= 1) == 0) || ($until_win && won > 0)
    end
  ensure
    if games > 0
      print("#{won}/#{games} = #{percent(won, games)}")
      if $passes_allowed == 1
        puts("; max up cards in winners: #{max_up_deck_cards}")
      else
        puts(", max passes: #{passes}")
      end
    end
    if $print_deck
      $orig_deck.reverse_each do |card|
        puts(card.short_name)
      end
    end
  end
end

def play_game()
  win = false
  if $deck_file
    load_deck_file()
    $deck_file = nil
  else
    new_seed = shuffle()
  end
  if $print_deck
    $orig_deck = $deck.dup
  end
  $pass_nbr = 1
  deal()
  unless $quiet
    show_board()
  end
  max_up_deck_cards = 0
  while true
    cards_played = 0
    while true
      while make_play_on_board()
        cards_played += 1
        show_board() unless $quiet
      end
      break if $deck.empty?
      unless $quiet
        say("Turning over #{$turn == 1 ? "a card" : "#{$turn} cards"}")
      end
      get_next_card()
      max_up_deck_cards = [max_up_deck_cards, $up_deck.length].max
      show_board() unless $quiet
    end
    if $up_deck.empty?
      x = for pile in $board
        break if pile.face_up > 0
        win = false
      end
      if x
        win = true
        break
      end
    end
    break if $pass_nbr == $passes_allowed || cards_played == 0
    $pass_nbr += 1
    say("Recycling deck") unless $quiet
    $deck, $up_deck = $up_deck, $deck
    $deck.reverse!()
    show_board() unless $quiet
  end
  GameStats.new(win, $pass_nbr, max_up_deck_cards, new_seed)
end

def get_next_card()
  return nil if $deck.empty?
  [$turn, $deck.length].min.times do
    $up_deck << $deck.pop
  end
  $up_deck.last
end

def shuffle()
  unless $quiet
    say("Shuffling")
    sleep($delay || 1.0)
  end
  if $seed
    new_seed = $seed
    $seed = nil
  else
    new_seed = Random.new_seed
  end
  srand(new_seed)
  $deck = Card.new_deck
  i = $deck.length
  while (i -= 1) > 0
    j = rand(i)
    $deck[i], $deck[j] = $deck[j], $deck[i]
  end
  $up_deck = []
  new_seed
end

def load_deck_file()
  f = File.new($deck_file)
  puts("Loading deck from file \"#$deck_file\"") unless $quiet
  $deck = []
  while line = f.gets
    card = Card.from_short_name(line)
    unless card
      raise "Bad card in deck file, line #{f.lineno}: #{line}"
    end
    $deck.push(card)
  end
  f.close()
  if $deck.length != 52
    raise RuntimeError("Wrong number if cards in deck (" + len($deck) + ")")
  end
  deck_set = Hash.new(false)
  for card in $deck
    deck_set[card] = true
  end
  n = deck_set.length
  if n != 52
    raise "Duplicate card(s) in deck (#{52 - n} duplicates)"
  end
  $deck.reverse!()
  $up_deck = []
end

def deal()
  say("Dealing") unless $quiet
  $board = Array.new(7, nil)
  $board.collect! {Stack.new([])}
  j = nil
  $board.each_index do |i|
    for j in i ... $board.length
      $board[j].cards << $deck.pop
    end
  end
  $board.each {|x| x.face_up = x.cards.length - 1}
  $aces = Array.new(4, nil)
  $aces.collect! {[]}
end

def card_is_playable_on_ace(card)
  rank = card.rank
  stack = $aces[card.suit]
  rank == (stack.empty? ? 1 : stack.last.rank + 1)
end

def card_is_playable_on_board(card, stack)
  rank = card.rank
  stack_cards = stack.cards
  return rank == 13 if stack_cards.empty?
  top_card = stack_cards.last
  rank == top_card.rank - 1 && card.suit_color != top_card.suit_color
end

def make_play_on_board()

  #  Check the face-up card in the deck.
  unless $up_deck.empty?
    card = $up_deck.last
    if card_is_playable_on_ace(card)
      say("Playing #{card.card_name} from deck to aces") \
          unless $quiet
      $up_deck.pop
      $aces[card.suit] << card
      return true
    end
    $board.each_with_index do |test_stack, i|
      if card_is_playable_on_board(card, test_stack)
        say("Playing #{card.card_name} from deck to pile #{i + 1}") unless $quiet
        $up_deck.pop
        test_stack.cards << card
        return true
      end
    end
    false
  end

  #  Check for plays on the board.
  cards = face_up = card = ncards = msg = range_to_move = nil
  $board.each_with_index do |stack, j|
    cards = stack.cards
    unless cards.empty?
      face_up = stack.face_up
      card = cards.last
      if card_is_playable_on_ace(card)
        unless $quiet
          say("Playing #{card.card_name} from pile #{j + 1} to aces")
        end
        cards.pop
        if face_up >= cards.length
          stack.face_up = face_up - 1
        end
        $aces[card.suit] << card
        return true
      end
      $board.each_with_index do |test_stack, i|
        if face_up >= 0
          card = cards[face_up]
          if card_is_playable_on_board(card, test_stack)
            ncards = cards.length - face_up
            unless $quiet
              msg = "Playing #{card.card_name} "
              if ncards > 1
                msg += "(#{ncards} cards)"
              end
              msg += "from pile #{j + 1} to pile #{i + 1}"
              say(msg)
            end
            range_to_move = face_up ... cards.length
            test_stack.cards.concat(cards[range_to_move])
            cards.slice!(range_to_move)
            stack.face_up = cards.length - 1
            return true
          end
        end
      end
    end
  end
  false
end

$title = nil

def show_board(option = nil)
  unless $title
    $title = ""
    for i in 1 .. $board.length
      $title += "---#{i}"
    end
    $title += "---\n"
  end
  peek = option == "peek"
  print(" ")
  $aces.each_with_index do |pile, j|
    unless pile.empty?
      print("   " + pile.last.short_name)
    else
      print("    -" + "sHDc"[j, 1])
    end
  end
  puts
  print($title)
  i = 0
  while true
    found = false
    for pile in $board
      cards = pile.cards
      if cards.length > i
        card = cards[i]
        found = true
      else
        card = nil
      end
      if !peek && i < pile.face_up
        print("  ##")
      else
        if card
          print(" " + card.short_name)
        else
          print("    ")
        end
      end
    end
    break unless found
    puts
    i += 1
  end
  print("   ")
  unless $deck.empty?
    if peek
      print($deck[-1].short_name + " ")
    else
      print(" ## ")
    end
  else
    print("    ")
  end
  unless $up_deck.empty?
    print($up_deck[-1].short_name)
  else
    print("   ")
  end
  puts("  (#{$up_deck.length} up, #{$deck.length} to go, pass: #{$pass_nbr})")
  if peek
    spaces = " " * 31
    for i in 2 .. [$decklength, $up_deck.length].max
      print(spaces)
      if i <= $deck.length
        print($deck[-i].short_name)
      else
        print("   ")
      end
      if i <= $up_deck.length
        print(" " + $up_deck[-i].short_name)
      end
      puts
    end
  end
  unless option
    if $delay
      sleep($delay) if $delay > 0.0
    else
      while true
        print("? ")
        resp = gets || exit
        resp.strip!.downcase
        if resp.empty?
          break
        elsif %w(q quit e exit).include?(resp)
          exit
        elsif %w(p peek s show).include?(resp)
          show_board(resp)
        elsif %w(? /).include?(resp)
          print("quit, exit, peek, show, or <return>?\n")
        end
      end
    end
  end
end

def say(s)
  if $ansi
    print("\033[2J\033[H")
  end
  puts("\n#{@say_number += 1}. #{s}\n\n")
end

def percent(won, games)
  sprintf("%.2f%%", Float((won * 100)) / games)
end

class Stack

  attr_accessor :cards, :face_up

  def initialize(cards = nil)
    cards = [] unless cards
    @cards = cards
  end

end

class GameStats

  attr_accessor :win, :passes, :max_up_deck_cards, :seed

  def initialize(win, passes, max_up_deck_cards, seed)
    @win = win
    @passes = passes
    @max_up_deck_cards = max_up_deck_cards
    @seed = seed
  end

end

class Card

  attr_reader :value, :rank, :suit, :suit_color, :rank_name, :suit_name,
      :short_name, :card_name

  def initialize(value)
    @value = value
    suit, rank = value.divmod(13)
    @rank = rank + 1
    @suit = suit
    @suit_color = suit == 0 || suit == 3 ? :black : :red
    @rank_name = @@rank_names[rank]
    @suit_name = @@suit_names[suit]

    rank_abbrev = case rank
      when 1 ... 10 then (rank + 1).to_s
      else @rank_name[0, 1].upcase
    end
    @short_name = (rank_abbrev + suit_name[0, 1].tr("shdc", "sHDc")).rjust(3)

    @card_name = rank_name + " of " + suit_name
  end

  @@rank_names = %w(ace deuce three four five six seven eight nine ten
      jack queen king")

  @@suit_names = %w(spades hearts diamonds clubs)

  @@card_pool = Array.new(52) {|i| Card.new(i)}

  def Card.new_deck
    @@card_pool.dup
  end

  def Card.get(i)
    @@card_pool[i]
  end

  def Card.from_short_name(s)
    s = s.rjust(3)
    r = s[1, 1].downcase
    rank = case r
      when "a" then 1
      when "j" then 11
      when "q" then 12
      when "k" then 13
      else Integer(s[0, 2])
    end
    return nil unless rank.between?(1, 13)
    suit = "shdc".index(s[2, 1].downcase)
    suit ? get(rank - 1 + suit * 13) : nil
  end

  def <=>(other)
    @value.<=>(other.value)
  end

  def hash
    self.value
  end

end

begin
  main()
rescue Interrupt
end
@headius
Member
headius commented Oct 10, 2013

Hmm....I let this run for quite a long time on JRuby master and did not see any failures. Do you only see failures with JRuby on Windows?

@bobjalex

I've only tried this on Windows, as I currently don't currently have
convenient access to any other OSes. It fails reliably when used with
Java8, but works on Java7. When it fails, it fails pretty quickly, in just
a few seconds,

Here's a failure on Windows Vista 32-bit, client VM:

...
Lost (1/96 = 1.04%, turn 3, 2 passes)
Lost (1/97 = 1.03%, turn 3, 2 passes)
Lost (1/98 = 1.02%, turn 3, 2 passes)
Lost (1/99 = 1.01%, turn 3, 2 passes)
Lost (1/100 = 1.00%, turn 3, 2 passes)
Lost (1/101 = 0.99%, turn 3, 2 passes)
1/101 = 0.99%, max passes: 7
NoMethodError: undefined method `empty?' for nil:NilClass
card_is_playable_on_ace at C:/RubyLib/auto_sol_b4_refactor.rb:300
make_play_on_board at C:/RubyLib/auto_sol_b4_refactor.rb:341
each at org/jruby/RubyArray.java:1613
each_with_index at org/jruby/RubyEnumerable.java:954
make_play_on_board at C:/RubyLib/auto_sol_b4_refactor.rb:336
play_game at C:/RubyLib/auto_sol_b4_refactor.rb:195
main at C:/RubyLib/auto_sol_b4_refactor.rb:131
(root) at C:/RubyLib/auto_sol_b4_refactor.rb:576
load at org/jruby/RubyKernel.java:1101
(root) at C:\RubyLib\invoke_ruby.rb:23

Dies similarly but not exactly the same on server VM:

Lost (6/111 = 5.41%, turn 3, 5 passes)
Lost (6/112 = 5.36%, turn 3, 5 passes)
Lost (6/113 = 5.31%, turn 3, 5 passes)
6/113 = 5.31%, max passes: 8
NoMethodError: undefined method `cards' for nil:NilClass
make_play_on_board at C:/RubyLib/auto_sol_b4_refactor.rb:337
each at org/jruby/RubyArray.java:1613
each_with_index at org/jruby/RubyEnumerable.java:954
make_play_on_board at C:/RubyLib/auto_sol_b4_refactor.rb:336
play_game at C:/RubyLib/auto_sol_b4_refactor.rb:195
main at C:/RubyLib/auto_sol_b4_refactor.rb:131
(root) at C:/RubyLib/auto_sol_b4_refactor.rb:576
load at org/jruby/RubyKernel.java:1101
(root) at C:\RubyLib\invoke_ruby.rb:23

but on 1 server VM trial, hung after 27 games.

On several trials on my Windows 7 64-bit machine (whose Java has only a
server VM), always hung after either 25 or 26 games.

This program normally sets a new random seed for each run, but seed can be
fixed using the -S option. Trying a few runs with a fixed initial seed
still shows variable behavior, sometimes crashing, sometimes hanging, after
a different but fairly consistent number of games.

New data point: after submitting this issue I rediscovered a forgotten
refactor of this program to be "better Ruby", putting it in a module,
eliminating the global variables, etc. The refactored version does not fail
at all, suggesting that the failures could be connected with the globals
in the version I sent.

Note that there are no failures on Java7, only Java8. Assuming that your
successful trials are on Linux, could be a Java8-Windows JVM anomaly.

The "improved" program that does not fail is attached, in case it might be
helpful.

Bob

On Wed, Oct 9, 2013 at 11:23 PM, Charles Oliver Nutter <
notifications@github.com> wrote:

Hmm....I let this run for quite a long time on JRuby master and did not
see any failures. Do you only see failures with JRuby on Windows?


Reply to this email directly or view it on GitHubhttps://github.com/jruby/jruby/issues/1098#issuecomment-26031587
.

@headius
Member
headius commented Oct 10, 2013

Ah-ha... this looks like one of several JIT issues related to invokedynamic that we fixed in 1.7.5. However, I was able to reproduce issues on JRuby 1.7 head with Java 8. We have a bug!

Here's the output I got from three runs. Something's definitely wrong:

system ~/projects/jruby $ jruby -v game.rb -q
jruby 1.7.5 (1.9.3p392) 2013-10-10 73821ac on Java HotSpot(TM) 64-Bit Server VM 1.8.0-ea-b103 +indy [darwin-x86_64]
Lost (0/1 = 0.00%, turn 3, 2 passes)
Lost (0/2 = 0.00%, turn 3, 6 passes)
Lost (0/3 = 0.00%, turn 3, 2 passes)
...
Lost (3/97 = 3.09%, turn 3, 4 passes)
Lost (3/98 = 3.06%, turn 3, 4 passes)
Lost (3/99 = 3.03%, turn 3, 4 passes)
Lost (3/100 = 3.00%, turn 3, 4 passes)
Lost (3/101 = 2.97%, turn 3, 4 passes)
Lost (3/102 = 2.94%, turn 3, 4 passes)
3/102 = 2.94%, max passes: 10
Error: Your application used more memory than the safety cap of 500M.
Specify -J-Xmx####m to increase it (#### = cap size in MB).
Exception trace follows:
java.lang.OutOfMemoryError: Java heap space
    at org.jruby.RubyArray.realloc(RubyArray.java:366)
    at org.jruby.RubyArray.append(RubyArray.java:1144)
    at java.lang.invoke.LambdaForm$DMH/695682681.invokeVirtual_LL_L(LambdaForm$DMH)
    at java.lang.invoke.LambdaForm$MH/1773206895.convert(LambdaForm$MH)
...

system ~/projects/jruby $ jruby -J-Xmx2048M -v game.rb -q
jruby 1.7.5 (1.9.3p392) 2013-10-10 73821ac on Java HotSpot(TM) 64-Bit Server VM 1.8.0-ea-b103 +indy [darwin-x86_64]
Lost (0/1 = 0.00%, turn 3, 7 passes)
Lost (0/2 = 0.00%, turn 3, 3 passes)
Won  (1/3 = 33.33%, turn 3, 5 passes)
...
Lost (1/21 = 4.76%, turn 3, 8 passes)
Lost (1/22 = 4.55%, turn 3, 4 passes)
Lost (1/23 = 4.35%, turn 3, 6 passes)
^C (hung)
system ~/projects/jruby $ jruby -J-Xmx2048M -v game.rb -q
jruby 1.7.5 (1.9.3p392) 2013-10-10 73821ac on Java HotSpot(TM) 64-Bit Server VM 1.8.0-ea-b103 +indy [darwin-x86_64]
Lost (0/1 = 0.00%, turn 3, 5 passes)
Lost (0/2 = 0.00%, turn 3, 4 passes)
Lost (0/3 = 0.00%, turn 3, 5 passes)
...
Lost (1/23 = 4.35%, turn 3, 4 passes)
Lost (1/24 = 4.17%, turn 3, 6 passes)
Lost (1/25 = 4.00%, turn 3, 2 passes)
1/25 = 4.00%, max passes: 7
NoMethodError: undefined method `rank' for nil:NilClass
  card_is_playable_on_board at game.rb:308
         make_play_on_board at game.rb:355
                       each at org/jruby/RubyArray.java:1613
            each_with_index at org/jruby/RubyEnumerable.java:947
         make_play_on_board at game.rb:352
                       each at org/jruby/RubyArray.java:1613
            each_with_index at org/jruby/RubyEnumerable.java:947
         make_play_on_board at game.rb:336
                  play_game at game.rb:195
                 __ensure__ at game.rb:131
                       main at game.rb:128
                     (root) at game.rb:576
@headius
Member
headius commented Oct 10, 2013

Oops...submitted a little fast.

So yeah I think this is another bug in our invokedynamic logic, or possibly a bug in the JVM's invokedynamic logic (it's probably us). This one's for me.

@headius
Member
headius commented Feb 21, 2014

I am no longer able to get this to fail! There have been a number of fixes to our invokedynamic logic in the past four months, so it wouldn't surprise me if we managed to fix this one too.

I tested both JRuby 1.7.11 and JRuby master (9k) for 5 runs up past 10k iterations. All succeeded, and appeared to run very fast too.

Tested on Oracle's early access JDK 1.8.0-b128, which is likely to become the release of Java 8.

@headius headius closed this Feb 21, 2014
@headius headius modified the milestone: JRuby 1.7.11, JRuby 1.7.10 Feb 21, 2014
@bobjalex

Thanks for the note. If you can't get it to fail, that's good enough for
me. I'll bet it's fixed.

At the time I sent it in, the only platform I had available was Windows, so
I was only able to experience it with Windows.

This was never a problem that concerned me very much, since it was easy to
work around. I mainly sent it in so that your Ruby implementation could be
perfect :-)

Bob

On Fri, Feb 21, 2014 at 1:29 PM, Charles Oliver Nutter <
notifications@github.com> wrote:

I am no longer able to get this to fail! There have been a number of fixes
to our invokedynamic logic in the past four months, so it wouldn't surprise
me if we managed to fix this one too.

I tested both JRuby 1.7.11 and JRuby master (9k) for 5 runs up past 10k
iterations. All succeeded, and appeared to run very fast too.

Tested on Oracle's early access JDK 1.8.0-b128, which is likely to become
the release of Java 8.


Reply to this email directly or view it on GitHubhttps://github.com/jruby/jruby/issues/1098#issuecomment-35775603
.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment