YJIT: Fallback setivar if the next shape is too complex

[k0kubun]

Currently, setivar_too_complex is one of the most frequent exit ops on SFR (e.g. 21.7% overall). This exit happens when it needs to transition to a too-complex shape.

This PR hoists out the logic to get the new_shape_id (as well as other information that is used to get new_shape_id and reused later) and check if it's OBJ_TOO_COMPLEX_SHAPE_ID before generating any code. If it is, it fallbacks to the generic setivar C call.

[casperisfine]

I've witnessed YJIT specific crashes since this was merged.

Here's a relatively small repro script:

require 'bundler/inline'

require 'objspace'
# run with --yjit --yjit-call-threshold=1

gemfile do
  source 'https://rubygems.org'
  gem 'mocha', '2.0.4'
  gem 'minitest', '5.18.1'
end

module MyConfig
end

module OtherModule
end
223.times do |iv|
  MyConfig.instance_variable_set("@_iv_#{iv}", 42)
end

require "minitest/autorun"
require "mocha/minitest"

class Module
  def mocha(instantiate = true)
    if instantiate
      @mocha ||= Mocha::Mockery.instance.mock_impersonating(self)
    else
      defined?(@mocha) ? @mocha : nil
    end
  end
end

class BugTest < Minitest::Test
  def test_mocha
    10.times do
      OtherModule.mocha
    end
    puts ObjectSpace.dump(MyConfig)
    puts ObjectSpace.dump(MyConfig.singleton_class)
    MyConfig.mocha
  end
end

[casperisfine]

Ok, so from what I gathered, the interpreter doesn't run into this, because vm_setivar just bails out when on a T_MODULE or T_CLASS, so we end up in the slow path.

From why I understand of the code, YJIT seem to always use the fast path, even with T_MODULE / T_CLASS hence why it runs into that bug.