Ubuntu Build Failure: Could not find 'minitar' (= 0.5.4) among 12 total gem(s)

[robbavey]

Builds are currently failing due to failed dependency resolution, which appear to be related to the recent move to jruby-9.1.13.0:

https://logstash-ci.elastic.co/job/elastic+logstash+master+multijob-unix-compatibility/os=ubuntu/135/console

14:00:04 Installing minitar 0.5.4 because the build process needs it.
14:00:04 mkdir -p build
14:00:04 mkdir -p build/bootstrap
14:00:04 [bootstrap] Fetching and installing gem: minitar (0.5.4)
14:00:04 rake aborted!
14:00:04 Gem::UnsatisfiableDependencyError: Unable to resolve dependency: user requested 'rake (= 12.1.0)'
14:00:04 /var/lib/jenkins/workspace/elastic+logstash+master+multijob-unix-compatibility/os/ubuntu/rakelib/gems.rake:39:in `block in (root)'
14:00:04 /var/lib/jenkins/workspace/elastic+logstash+master+multijob-unix-compatibility/os/ubuntu/rakelib/gems.rake:16:in `block in (root)'
14:00:04 /var/lib/jenkins/workspace/elastic+logstash+master+multijob-unix-compatibility/os/ubuntu/rakelib/dependency.rake:12:in `block in (root)'
14:00:04 /var/lib/jenkins/workspace/elastic+logstash+master+multijob-unix-compatibility/os/ubuntu/rakelib/vendor.rake:15:in `block in untar'
14:00:04 /var/lib/jenkins/workspace/elastic+logstash+master+multijob-unix-compatibility/os/ubuntu/rakelib/vendor.rake:87:in `block in (root)'
14:00:04 /var/lib/jenkins/workspace/elastic+logstash+master+multijob-unix-compatibility/os/ubuntu/rakelib/z_rubycheck.rake:28:in `(root)'
14:00:04 Gem::MissingSpecError: Could not find 'minitar' (= 0.5.4) among 12 total gem(s)
14:00:04 Checked in 'GEM_PATH=/var/lib/jenkins/workspace/elastic+logstash+master+multijob-unix-compatibility/os/ubuntu/vendor/bundle/jruby/2.3.0', execute `gem env` for more information
14:00:04 /var/lib/jenkins/workspace/elastic+logstash+master+multijob-unix-compatibility/os/ubuntu/rakelib/gems.rake:13:in `block in (root)'
14:00:04 /var/lib/jenkins/workspace/elastic+logstash+master+multijob-unix-compatibility/os/ubuntu/rakelib/dependency.rake:12:in `block in (root)'
14:00:04 /var/lib/jenkins/workspace/elastic+logstash+master+multijob-unix-compatibility/os/ubuntu/rakelib/vendor.rake:15:in `untar'
14:00:04 /var/lib/jenkins/workspace/elastic+logstash+master+multijob-unix-compatibility/os/ubuntu/rakelib/vendor.rake:87:in `block in (root)'
14:00:04 /var/lib/jenkins/workspace/elastic+logstash+master+multijob-unix-compatibility/os/ubuntu/rakelib/z_rubycheck.rake:28:in `<main>'

[jakelandis]

I dug into this a bit a found that rake bootstrap is blowing up while trying to install minitar with this error:

#<Gem::UnsatisfiableDependencyError: Unable to resolve dependency: user requested 'rake (= 12.1.0)'>

I believe that is because minitar has dependency on rake < 12
https://github.com/halostatue/minitar/blob/v0.6.1/minitar.gemspec#L35

but JRuby ships with 12.1.0

which rake
/Users/jake/.rvm/gems/jruby-9.1.13.0@global/bin/rake
/Users/jake/.rvm/gems/jruby-9.1.13.0@global/bin/rake --version
rake, version 12.1.0

...so this makes sense to why it doesn't work.

But JRuby 9.1.12 also ships with 12.1.0 !

/Users/jake/.rvm/gems/jruby-9.1.12.0@global/bin/rake --version
rake, version 12.1.0

so the better question may be why did this ever work ?

Anyway, I have the request in to revert the build system back to 9.1.12.0 so all should be happy again. @ph is working on a more strategic fix for how we interact with rake.

[jakelandis]

Also @original-brownbear found that the following will allow it work (assuming rake is installed elsewhere):

gem uninstall -i  ~/.rvm/gems/jruby-9.1.13.0@global rake

and @colinsurprenant with rbenv states BUNDLE_DISABLE_SHARED_GEMS: '1' may help.

 cat ~/.bundle/config
---
BUNDLE_PATH: vendor
BUNDLE_DISABLE_SHARED_GEMS: '1'

[jakelandis]

Closing this issue.

Root cause: #8220
PR that triggered the failure: #8165
Revert of PR: #8219

[anthraxx]

@jakelandis this is still the case for the 5.6 tree, i don't see any fix applied there.

rake is from ruby 2.4.2 (there is no way to downgrade as its a distro level build for packaging)
Advises how yo proceed would be appreciated.

==> Starting build()...
mkdir -p vendor
mkdir vendor/_
mkdir -p build
mkdir -p build/bootstrap
rake aborted!
Gem::UnsatisfiableDependencyError: Unable to resolve dependency: user requested 'did_you_mean (= 1.1.0)'
/build/logstash/src/logstash-5.6.1/rakelib/gems.rake:39:in `block (2 levels) in <top (required)>'
/build/logstash/src/logstash-5.6.1/rakelib/gems.rake:16:in `rescue in block (2 levels) in <top (required)>'
/build/logstash/src/logstash-5.6.1/rakelib/gems.rake:12:in `block (2 levels) in <top (required)>'
/build/logstash/src/logstash-5.6.1/rakelib/dependency.rake:12:in `block (2 levels) in <top (required)>'
/build/logstash/src/logstash-5.6.1/rakelib/vendor.rake:16:in `untar'
/build/logstash/src/logstash-5.6.1/rakelib/vendor.rake:86:in `block (2 levels) in <top (required)>'
/build/logstash/src/logstash-5.6.1/rakelib/z_rubycheck.rake:28:in `<top (required)>'
Gem::MissingSpecError: Could not find 'minitar' (= 0.5.4) among 6 total gem(s)
Checked in 'GEM_PATH=/build/logstash/src/logstash-5.6.1/vendor/bundle/ruby/2.4.0', execute `gem env` for more information
/build/logstash/src/logstash-5.6.1/rakelib/gems.rake:13:in `block (2 levels) in <top (required)>'
/build/logstash/src/logstash-5.6.1/rakelib/dependency.rake:12:in `block (2 levels) in <top (required)>'
/build/logstash/src/logstash-5.6.1/rakelib/vendor.rake:16:in `untar'
/build/logstash/src/logstash-5.6.1/rakelib/vendor.rake:86:in `block (2 levels) in <top (required)>'
/build/logstash/src/logstash-5.6.1/rakelib/z_rubycheck.rake:28:in `<top (required)>'
Tasks: TOP => gem:install
(See full trace by running task with --trace)
Downloading http://jruby.org.s3.amazonaws.com/downloads/1.7.27/jruby-bin-1.7.27.tar.gz
Installing minitar 0.5.4 because the build process needs it.
[bootstrap] Fetching and installing gem: minitar (0.5.4)
==> ERROR: A failure occurred in build().

[jakelandis]

@anthraxx - we didn't really 'fix' it, we just worked around it temporarily in our build systems.

The root cause here is a mismatch of dependencies w/r/t the required rake versions. Since it appears that you are using MRI Ruby, I believe that means you should ensure that you have a rake version less then v12 in your set of gems.

Perhaps try sudo gem install rake -v 11.3.0 ? FWIW, my Ubuntu 16.04 VM also uses system MRI Ruby (2.3), but has both 12.0.0 and 10.5.0 rake gems installed by default and is able to bootstrap 5.6.

There is also work currently underway to not require Ruby for the bootstrapping, but I doubt that will be back ported to 5.x.

[anthraxx]

@jakelandis the problem is, I can't do that. I'm speaking about packaging for Arch Linux and for that you can't run gem installs via sudo inside a clean build environment and rake must be provided through the ruby package that hips that version.
Right now this blocks the update to 5.6.1 and we are stuck with the old version in the distro.

[jakelandis]

@anthraxx - this should only be development/build environment for building from source right ? (e.g. running a Logstash distribution is not effected is it ?)

Can you update your clean environment to a different distro, or bake in rbenv or or a different rake version ?

FWIW, this is not a new issue, we just recently bumped into this when we upgraded our build environment to JRuby 9.13.0, and are unsure why it didn't bite us before.

[anthraxx]

@jakelandis We as a distribution aim to always build from source rather then preconstructed artifacts generated out of a source tree. They may contain precompiled things plus its harder to ensure that its really what the source is showing/claiming to the public inside the repository (has noting to do with mistrust so don't take it as offense, its just the 'build from source' approach).

The env is bound to arch, so no different distro is possible and rbenv is not available. i could possibly try to use ruby 2.3.4 instead of 2.4.2 but that feels non optimal.

[jakelandis]

@anthraxx - what is the highest version that your builds still work ?

We thought this issue was related to upgrade to JRuby 9.1.1.13 (6.0+) , but there may be other compounding factors if you are seeing a change in behavior with 5.6.1.

[anthraxx]

@jakelandis first, thanks that you are trying to resolve this problem, much appreciated!

looks like the old 5.6.0 also fails, so I assume this may be related to ruby 2.4.2 that brings rake 12.0.0.
This happens when doing the rake bootstrap call, which then also invokes the download of jruby-bin-1.7.27.tar.gz associated with this logstash version.

[jakelandis]

@anthraxx - it almost certainly rake 12.0.0 that is doing the bootstrapping that is the issue. For the 5.x stream of code, the requirement to have a rake <12.0.0 for bootstrapping will likely remain. (only production bug fixes are getting push back to 5.x as we are prepping for 6.x)

I think this will be formally fixed in 6.1 as we will use Gradle (not Ruby) for boot strapping. Here is the first (of many) change(s) to our build system: #8238

[anthraxx]

@jakelandis thanks a lot for the pointer and explenation, looking forward for the 6.x stuff. Meanwhile i try to make 5.x work with old rake until we get the fancy new stuff, will report back so you know if it worked 😛

[anthraxx]

@jakelandis it worked perfectly with an old rake as bootstrapping. Goess thats good enough until 6.x arrives, thanks again I very much appreciate the assistance. thanks for everything