1.7.14+ imposes intrusive jar-dependencies gem

[dekellum]

I was under the impression that the "default" gems shipped with the jruby distro where intented to still be opt-in, in the sense that a require or bundle dependency was needed to load them This does not appear to be the case with the new "jar-dependencies" gem added as default in 1.7.14: I have no references to it in my projects but it appears to be injecting and running at least Gem.post_install hooks that are causing me some trouble (which I will elaborate in specific bugs.) It may also be hooking into bundler?

This has me a bit disappointed because 1.7.14 should just be for bug fixes over 1.7.13. Here you have made seemingly significant feature additions which have real and potentially more negative consequences. #1965 and #1942 appear to demonstrate this already.

Can you offer a workaround for the 1.7.14 and 1.7.15 releases to disable this "default" jar-dependencies gem extension behavior at install and/or runtime? I haven't found a way. I've been trying to delete various files listed in the jar-dependencies-0.0.9.gemspec to no avail.

We are unable to benefit from the other 1.7.14 and 1.7.15 bug fixes given this regression. The right thing to do from a release management perspective would be to back these associated jar-dependences changes out and release a 1.7.16 without them. Uphold some modicum of semantic versioning and release this in a new feature release, e.g. 1.8.0 if you don't need to get cute with "9000".

[dekellum]

From #1975 @mkristian wrote:

there are a couple of gems you did not want to run like krypt, krypt-provider, jruby-openssl, jar-dependencies - these a defaults gems and a such part of jruby. the moment jruby requires any file of those default gems, rubygems either finds an installed gem or activates the default gem. this is also true for MRI for their default gems (not sure if they have any) and this logic is part of rubygems.

I obviously want HTTPS and thus krypt, ssl for rubygems.org access from bundler, and since jruby is a drop in replacement for MRI ruby, it needs to work. Bundler just uses the "stdlib" openssl. JRuby happens to implement this in terms of a "default" gem. Also this wouldn't even load unless I loaded bundler and the HTTPS URL to rubygems.

By comparison "jar-dependences":

• appears to always load and execute as part every Gem post install, without any user opt-in
• may run without opt-in in other cases? As part of gem load?
• has been introduced without warning in a JRuby patch release (1.7.14)
• has caused compatibly problems/regressions Gemfiles that use :path broken between 1.7.13 and 1.7.15 #1965 Bundle not working with two gemspecs - bug introduced in 1.7.14 #1942 nil TypError on bundle install, regression 1.7.14+ #1975, so far...
• has no-equivalent in MRI and could be viewed as a divergence from MRI Gem and Bundle behavior

The actual jar-dependencies gem might be a useful utility for projects that want it. Why not make it an opt-in facility (i.e. via require), and have those projects opt-in?

[dekellum]

The performance concern raised by @kares in #1942 would also be avoided if jar-dependencies was an opt-in facility. As it stands, its not very conducive to testing.

[kares]

@dekellum all very true, but ... JRuby 1.7.x has not been rolling in an absolutely "every commit should make it more stable" sense (well it has as much as tests guard for regressions but there are usually corner cases with refactorings - although invasive PRs usually mean to improve things so it's a 2 sided story as usual). there was at least one load service refactoring (I know of - caused a regression in 1.7.14) and certainly more seems planned ... e.g. there's some work that will change how loading works and might have impact on $LOADED_FEATURES.

I feel your pain, but that 9k is still pretty far from now and it's pretty hard to work on smt with "distant" feedback - do not get me wrong I'm on your side while as well I do understand the motivation of some of the "new feature" commiters (there certainly would be less work done otherwise).

as far as I know jar-dependencies is used by jruby-openssl (since 0.9.5 - maybe if you degrade to 0.9.4 it won't load ?) ... I can work on making sure it does not get used unless previously loaded, are there other cases/places where you have seen it getting pulled in ?

[dekellum]

@kares you are absolutely correct on the history of jruby release management. I'm saying this issue is just the latest example of how regression bug impacts are amplified by poor release management decisions. This "9000" thing is just a marketing code, and I'm not in marketing. JRuby could have released this jar-dependencies feature in a new "8.0.0" line and saved "9.0.0" for the long awaited "9000" feature set.

I was the one who contributed the backporting effort for the release of 1.6.8, which in my biased estimation was JRuby's most stable release in its history. Unfortunately I feel I won that battle but have lost the war, in terms of changing any release management approach.

Of course, different release numbering isn't enough: the necessary additional step is that 1.7.x gets maintained for a while (bug fixes only) and in parallel with 8.0.x, then 9.0.x.

It would make JRuby a more grown-up kind of project, to see it released with at least the same level of care and parallel patch releases, as for example: MRI Ruby!

cc @headius

[dekellum]

Back to the bug proper... Looks like krypt at least gets pulling in even with a local gem install. I don't know how to degrade jruby-openssl as I'm just using what's included in the various jruby versions tested.

[enebo]

@dekellum I fully admit our last few point releases have been rocky and I can also appreciate the approach you would like us to take. In truth, our numbering is botched for 1.7.x and downward as 1.7 is more like a major number and .x is a minor number. Our intention in each release is the equivalent of minor number of semantic versioning. 1.7.14 appears to have a went further than that and it was not intentional. With that said, if we followed your advice (and I am not saying it is poor advice) how many versions would our project have to support in parallel as we periodically add new non-backwards breaking behavior? Should adding ripper (added I think 1.7.4-5) have been a new number? We have tried to strike a balance between stability and backwards compat as much as we can while still being able to support putting out releases. With only two FTEs and a lot of other work we are perhaps not doing as good as we could but we are trying to provide the most benefit we can while still making forward progress.

For JRuby 9000, we will at least correct the labelling issue so we are inline with semantic versioning. Also, we will be in a better position to put out security or regression fixes as a quick turnaround so I think that will nudge a tiny bit closer to your ideal of what we should providing...(e.g. 9000.1.1 for a security fix).

[kares]

... and my jruby-openssl work #1543 would have been so over 9000! so I'm guilty as well :)

now, back to jar-dependencies I thought there's no way avoiding it with a gem install but I was wrong there's a check as well ... thus doing export JARS_SKIP=true or jruby -J-Djars.skip=true should (still load esp. when require 'openssl' runs, but) avoid using the gem completely - please do tell us more about your concrete concers if any that are not reported related to running with "jars skipped" ...

[dekellum]

On releases, @enebo, I really appreciate your nuanced response.

It would help JRuby's practical stability and perception greatly if users had 3-6 months to test and work out regressions with a new feature release (e.g. 9.1.0 or 10.0.0), without loosing bug fix support on a prior feature release (e.g. 9.0.3). I would guess that means maintaining concurrent releases for 2 release branches most of the time, but it depends on the frequency of feature releases.

I see other smaller and bigger projects doing this with benefit. There is certainly also cost as you note. More git topic branch conventions and discipline? Volunteer help with additional releases and back porting? I could find some time to help with this again. Its worth it to me to see it done.

[dekellum]

@kares I now see jar-dependencies#5 (for 1.1.2 of same). Having to opt-out (with JARS_SKIP environment var I need to circulate) isn't quite the same as having an explicit option required to opt-in to jar-dependencies. We are talking Gem and Bundle behavior here, so I think its correct to say this is a new workaround, not a fix.

To be fair, if jar-dependencies had been introduced as a new feature release instead of 1.7.14, I'd be less certain that opt-in was a necessity. Specifically I'd be more confident you would have all the edge cases fixed before this feature was forced upon me.

[dekellum]

...add #3074 and #3078 to list of unintended regressions and consequences of jar dependencies. In the #3078 case at least, JARS_SKIP was not an available workaround to disable this extension.

[kares]

we might need to have the "hard" dependency in jruby-openssl on by default to avoid bc.jar collisions ... these are quite annoying: jruby/jruby-openssl#74 ... and there's no other way around them.
please follow up on the issue and let us know if recent jar-dependencies is still causing you troubles. thanks.