Heroku CI: undefined symbol: rb_data_typed_object_zalloc

[joshwlewis]

Some users will see this when Heroku CI is running their tests:

-----> Running Ruby buildpack tests...
/tmp/buildpacks/f6d48d8a14fccbb19c0c0402fca224929d18e9ee042b3e204bc5992612e990d3b3a0fb1f9627b1a3bae11e9fa20dffc96e136bd734a2f3cf92a0d05bedd42cfe/vendor/ruby/cedar-14/bin/ruby: symbol lookup error: /app/vendor/bundle/ruby/2.3.0/gems/json-1.8.6/lib/json/ext/generator.so: undefined symbol: rb_data_typed_object_zalloc
-----> Ruby buildpack tests failed with exit status 127

After a bunch of digging, it looks like these are the conditions needed to reproduce:

1. The app is not using the same ruby version that comes with the Heroku stack
2. The app has json in it's Gemfile
3. The app's app.json does not include a scripts.test entry

I've created a sample app that meets those conditions. You can find it at https://github.com/joshwlewis/heroku-ci-json-example. I've also setup a pipeline that showcases the failing test here.

I've traced down the error to this line.

My working theory:

1. During test-setup the buildpack runs require "json", this does not error, because it uses the default json library packaged with the stack Ruby.
2. test-setup installs json (with native extensions) with the user-specified ruby version.
3. test (running against the stack Ruby) attempts to require "json", but instead of grabbing the Ruby vendored version, it tries to load the bundled one (which was compiled against the user's specified Ruby version).

[joshwlewis]

Note that users can work around this by specifying the test script inapp.json. For example:

{
  "scripts": {
    "test": "bundle exec rspec"
  }
}

[schneems]

Thanks we can also replicate the error on a regular deploy if two ruby buildpacks are specified

# Gemfile
source "https://rubygems.org"

ruby "2.3.3"
gem "json", "1.8.6"

# Fails
$ heroku buildpacks
=== mysterious-lowlands-72135 Buildpack URLs
1. https://github.com/heroku/heroku-buildpack-ruby
2. https://github.com/heroku/heroku-buildpack-ruby#v155

# works
$ heroku buildpacks
=== mysterious-lowlands-72135 Buildpack URLs
1. https://github.com/heroku/heroku-buildpack-ruby
2. https://github.com/heroku/heroku-buildpack-ruby#v154

It seems something in v155 is causing issues, but we're not entirely sure what.

[schneems]

I've got a fix in #552 the failure mode is really...interesting.

[joshwlewis]

While #553 fixes the double rainbow issue on normal builds, it persists for CI runs. Do we also need to unset GEM_PATH in bin/test-compile?

[schneems]

10-4. I'm going to move all the bash code that sets up ruby into it's own file. Actually think we can re-organize a few things to make it a bit easier to maintain.

[joshwlewis]

Correction: the unset should probably live in bin/test. Unfortunately, this doesn't work, as we can't find bundler:

-----> Running Ruby buildpack tests...
/tmp/tmp.WINZyBNX4H/lib/ruby/2.2.0/rubygems/core_ext/kernel_require.rb:54:in `require': cannot load such file -- bundler (LoadError)
	from /tmp/tmp.WINZyBNX4H/lib/ruby/2.2.0/rubygems/core_ext/kernel_require.rb:54:in `require'
	from /tmp/buildpacks/3ed7ebcfb243df2305cc306e80f79df43cfa8f55b071f5a455ec3abfd8008adfd2f56585e76e39f12775ca1866aad7bedb12a59fb779894dc7d06136fc9dc998/lib/language_pack/helpers/bundler_wrapper.rb:37:in `install'
	from /tmp/buildpacks/3ed7ebcfb243df2305cc306e80f79df43cfa8f55b071f5a455ec3abfd8008adfd2f56585e76e39f12775ca1866aad7bedb12a59fb779894dc7d06136fc9dc998/bin/ruby_test:36:in `<main>'
-----> Ruby buildpack tests failed with exit status 1

[joshwlewis]

Ah yeah, the source of that error is here. If the GEM_PATH is unset, require 'bundler' will fail.

[joshwlewis]

Fixed by #555. Thanks!