Allow installing from non https sources

[deivid-rodriguez]

What was the end-user or developer problem that led to this PR?

OpenSSL is not mandatory but installing from non https sources fail.

What is your fix for the problem, implemented in this PR?

My fix is to allow it, since it's not hard.

But I think we should at least print a warning.

Fixes #4191.

Make sure the following tasks are checked

• Describe the problem / feature
• Write tests for features and bug fixes
• Write code to solve the problem
• Make sure you follow the current code style and write meaningful commit messages without tags

[sonalkr132]

I think we should definitely add a warning about the insecure source to discourage usage of HTTP.

I will add http://insecure.rubygems.org which doesn't redirect to https.

[deivid-rodriguez]

Thanks @sonalkr132, that makes sense to me.

I would propose the following as an alternative to this PR:

For rubygems.org sources

• If http://rubygems.org is specified as a source:
  • If openssl is available: Warn with something like, "Using http sources is discouraged, change your source to https://rubygems", and automatically use https://rubygems.org.
  • If openssl is not available: Warn with something like, "Using http sources is discouraged, install a ssl-capable ruby and use https://rubygems.org", and automatically use http://insecure.rubygems.org.

For custom sources

• Do nothing, except detecting if a http source has been specified and we've been redirected to a https source. In that case, fail with a proper error if openssl is not available.

[hsbt]

I agreed with @sonalkr132 's opinion.

[sonalkr132]

Using http sources is discouraged, change your source to https://rubygems"

I think this should mention that we redirected to https automatically, if we are transparently redirecting to https.

automatically use http://insecure.rubygems.org

I am not sure about this. I think we should always error. HTTP source has not been supported for over five years, which goes to show how few users must be affected by this. IMHO, it should remain "officially" unsupported. We can add a guide with detailed consequences, helpful commands to fix their problem (install openssl, links to fix cert guides etc), mention http://insecure.rubygems.org as last resort and config/flags they need to manually change to use it. We can print link to guide in the error message.

[deivid-rodriguez]

@sonalkr132 I agree with your comments, I can move forward with my proposal + your modifications once http://insecure.rubygems.org is in place 👍.

[sonalkr132]

so, just for update sake.. insecure.rubygems.org was added. only, ^/(info|versions|api/v1/dependencies) won't redirect to https if accessed over insecure.rubygems.org. It works for bundler and rubygems <= 3.2.2.
I have one clarification for a potential guide on this.. is missing openssl a genuine use case and something we want to support, given we enforce the following?

$ gem uninstall openssl
Gem openssl-2.2.0 cannot be uninstalled because it is a default gem

I am not sure if we should remove raise Gem::Exception.new .. unless Gem::HAVE_OPENSSL. It is one thing to allow installation over HTTP, but another to go out of the way to support something which seems like "won't fix".
cc: @marcandre

[deivid-rodriguez]

Oh, sorry @sonalkr132, just to clarify, I marked this PR as WIP because with the new proposal I won't remove this raise.

The alternative proposal was to restore legacy support for http sources. I didn't actually check when this stopped working, but nothing in the codebase indicates that this is unsupported, including the current error message recommending using non-https sources as an alternative.

[sonalkr132]

the new proposal I won't remove this raise

thanks for clarifying. I was under the assumption that you just won't redirect to http://insecure.rubygems.org

[deivid-rodriguez]

The idea is that we will intercept rubygems.org http sources first and never try to even request anything over https if openssl is not available.

As per the updated proposal:

For rubygems.org sources

• If http://rubygems.org is specified as a source:

  • If openssl is available, warn with something like

    Using http sources is discouraged, rubygems is automatically using https://rubygems.org instead. Change your source to https://rubygems.org to avoid this message

    And automatically use https://rubygems.org.

  • If openssl is not available, fail with something like

    Using http sources is no longer supported. Check https://<link_to_guide> to fix your setup.

    In the guide, mention everything relevant like openssl installation guides, ssl certificate troubleshooting, and http://insecure.rubygems.org as a last resort.

For custom sources

• Do nothing, except detecting if a http source has been specified and we've been redirected to a https source. In that case, fail with a proper error if openssl is not available. Actually, I believe this is exactly where this raise would he hit, and the current error message would be just fine. It could perhaps be tweaked to something like

  OpenSSl is not available. Install OpenSSL and rebuild Ruby (preferred) or use pure non-HTTPS sources i.e., http:// sources that don't redirect to https

[marcandre]

so, just for update sake.. insecure.rubygems.org was added. only, ^/(info|versions|api/v1/dependencies) won't redirect to https if accessed over insecure.rubygems.org. It works for bundler and rubygems <= 3.2.2.
I have one clarification for a potential guide on this.. is missing openssl a genuine use case and something we want to support, given we enforce the following?

$ gem uninstall openssl
Gem openssl-2.2.0 cannot be uninstalled because it is a default gem

I am not sure if we should remove raise Gem::Exception.new .. unless Gem::HAVE_OPENSSL. It is one thing to allow installation over HTTP, but another to go out of the way to support something which seems like "won't fix".
cc: @marcandre

It is possible to install Ruby without OpenSSL. It actually seems tricky to install it with OpenSSL (dunno why); I had troubles with legacy version and rvm, and switched to rbenv/ruby-install that actually recompiles OpenSSL each time to avoid such issues. Also, installing from source seem to install without OpenSSL by default (I imagine there's an option to ./configure but haven't looked it up yet). Hope this helps.

[deivid-rodriguez]

@marcandre I'm very surprised by what you said, haven't heard of any similar report by anyone since ruby 2.3 I think. I guess we should dig into your issues a bit more before changing anything here. Depending on how many people is affected we can make a decision. I'm also surprised by what you said about installing from source, my experience is completely the opposite. A while ago I tried to install ruby without openssl and it was quite hard to figure out (my shell history says --with-out-ext=+,openssl might've done the trick). I think perhaps you have a misconfiguration of the openssl library in your machine or something?

@sonalkr132 I noticed that currently you can visit https://insecure.rubygems.org/ just fine. Is that the issue you were mentioning about gem install unnecessarily making a root request? I feel the fact that https://insecure.rubygems.org/ works just fine could make a bit of a bad impression. Since it's served over https, it makes you think that it's the site that's insecure, not the protocol. Since the refactoring to remove this root HEAD request might not be trivial, maybe we should stick to the plan we designed,but without the "insecure.rubygems.org as a last restort". Basically, leave this broken, but give better error messages and resolution steps?

[sonalkr132]

I noticed that currently you can visit https://insecure.rubygems.org/ just fine.
leave this broken, but give better error messages and resolution steps?

This is expected. I have only disabled the https redirect on dependency API endpoints, otherwise, insecure.rubygems.org is the same as any other domain. We can potentially block everything on this except dependency endpoints, however as you are suggesting probably leaving this broken is the most straightforward option.

Depending on how many people is affected we can make a decision.

This may be relevant rbenv/ruby-build#377 I don't have first hand experience of this but what I can tell from dealing with support tickets is that macos users always had to struggle with openssl being outdated, misconfigured or regression (on OS updates).

[deivid-rodriguez]

Then maybe we should redirect http://insecure.rubygems.org to https://rubygems.org instead? I think urls that include "insecure" should be served over an insecure protocol, otherwise it could get confusing and not make a good impression for people. Otherwise I would revert this "last resort domain".

Regarding OpenSSL issues, yeah, being a MacOS thing explains why I never run into it. Still, things seem good now, at least that ticket you linked to suggests that since ruby 2.3 people seem to no longer have those issues.

[deivid-rodriguez]

@sonalkr132 I didn't manage to get to this yet, but I wonder about your opinion about my previous message. I still don't like the fact that we serve a url including the "insecure" word over https.

[sonalkr132]

maybe we should redirect http://insecure.rubygems.org to https://rubygems.org instead?

This should be doable. But only redirect it for non-dependency endpoints, right?

[deivid-rodriguez]

I would be fine with either completely dropping the http://insecure.rubygems.org last resort solution in favour of the others, or redirecting http://insecure.rubygems.org requests to the equivalent https://rubygems.org addresses unless it's a dependency endpoint in which case we shouldn't be redirecting it at all.

[sonalkr132]

leave this broken, but give better error messages and resolution steps?

the resolution would be that you need OpenSSL to use rubygems, so steps to install it?

[deivid-rodriguez]

Yeah, basically remove the "and http://insecure.rubygems.org as a last resort." part from the plan outlined in #4192 (comment).

[sonalkr132]

Got it. I will remove insecure.rubygems.org setup.

[deivid-rodriguez]

@simi Would you be ok with that? This has apparently been broken for a while now and we've got a single report so far, so I think it should be ok?

[simi]

Sure, let's do that.

[sushma-4]

Sorry what's the fix again?

ERROR:  While executing gem ... (Gem::Exception)
    OpenSSL is not available. Install OpenSSL and rebuild Ruby (preferred) or use non-HTTPS sources

[deivid-rodriguez]

There's no solution, really. The plan is to try print a better error, so that you can fix your ruby installation so that it can use https, but we haven't got to that yet.

[josienb]

There's no solution, really. The plan is to try print a better error, so that you can fix your ruby installation so that it can use https, but we haven't got to that yet.

I just tried to install a gem via http and was indeed puzzled by it not working:

 == $ gem install openssl --source 'http://rubygems.org'
ERROR:  While executing gem ... (Gem::Exception)
    OpenSSL is not available. Install OpenSSL and rebuild Ruby (preferred) or use non-HTTPS sources

I think a quick improvement would be to only change the error message and remove the "use non-HTTPS sources" part (even without doing anything else, like adding steps on how to install).

Would it make sense if I made a PR for just that small change?

[simi]

Can you try with --clear-sources? Feel free to include also --verbose for better tracing.

$ gem install ABO --clear-sources --source 'http://rubygems.org' --verbose

[josienb]

Can you try with --clear-sources? Feel free to include also --verbose for better tracing.

Thanks for the quick reply!

 == $ gem install openssl --clear-sources --source 'http://rubygems.org' --verbose
HEAD http://index.rubygems.org/
301 Moved Permanently
ERROR:  While executing gem ... (Gem::Exception)
    OpenSSL is not available. Install OpenSSL and rebuild Ruby (preferred) or use non-HTTPS sources

That's probably the http -> https redirect kicking in, right?

[simi]

Can you try with --clear-sources? Feel free to include also --verbose for better tracing.

Thanks for the quick reply!

 == $ gem install openssl --clear-sources --source 'http://rubygems.org' --verbose
HEAD http://index.rubygems.org/
301 Moved Permanently
ERROR:  While executing gem ... (Gem::Exception)
    OpenSSL is not available. Install OpenSSL and rebuild Ruby (preferred) or use non-HTTPS sources

That's probably the http -> https redirect kicking in, right?

Yes, there was insecure.rubygems.org, but we have removed this one at rubygems/rubygems.org#2657. I think currently there is no way to install rubygem.org gem without HTTPS. But it is possible with other (custom) sources.