Include headers for bundled libraries #279

Open
sodabrew opened this Issue Aug 3, 2015 · 10 comments

Projects

None yet

3 participants

@sodabrew
sodabrew commented Aug 3, 2015

Following up from a post by at https://groups.google.com/forum/#!topic/rubyinstaller/Bu-Aa49WvNk (by @FooBarWidget) requesting OpenSSL headers, I would also like to see these included with RubyInstaller, and generally any libraries that are bundled with RubyInstaller should have their headers included.

Per the README, these libraries should get their headers included in C:\RubyVER\include or similar:

At this time zlib, rb-readline, gdbm, iconv, pdcurses, and openssl are included.

I assert that it makes more sense to bundle these headers with RubyInstaller rather than DevKit, first because the libraries ship with the Ruby package, and are updated with the Ruby package, and second that the DevKit package tends to span across many revisions since it's basically just a compiler package.

Happy to help implement this, given some pointers in the right direction!

@Azolo
Member
Azolo commented Aug 3, 2015

Actually the headers aren't bundled with DevKit either. They are distributed through OpenKnapsack Project Repository. Which includes all relevant binaries and build information instead of just the things needed for RubyInstaller. Which keeps the Ruby packages nice and small.

On top of that packaging all of that still wouldn't make it easier to build many of those gems because Windows doesn't have a unified spot for library headers. We would still be telling people how to use them "inside" their Ruby installation versus how to get them and then use them.

In other words, you're going to need a REALLY compelling reason on why everyone needs them instead of how it could make people who may not want to use the static eventmachine or people who want to build puma lives marginally better. Or how I'm missing something and the increases would be better than a marginal increase.

I'm not trying to sound dismissive, because this has been a topic of discussion before, but I can't see it helping more that a few people and only marginally.

@sodabrew
sodabrew commented Aug 7, 2015

Right, so, the problem is that if someone wants to use an OpenSSL-able gem, they need RubyInstaller + DevKit + exactly the same version of OpenSSL from Knapsack as their Ruby includes. That's a huge pain. If they get a different version then you've introduced the user to 1990's "DLL Hell" for no particularly good reason.

The complete set of OpenSSL headers on my system is 412 KB. This isn't going to hurt anybody's disk space or download times.

There is already an include/ directory in RubyInstaller where ruby.h lives. This is necessarily the versioned ruby.h as corresponds with the ruby.exe binary in the same directory tree. There's a pattern here: you need to ship the headers with the binaries so that they match.

@Azolo
Member
Azolo commented Aug 7, 2015

You can use any OpenSSL you want. 😃

You can also replace the DLLs in the folder as well and it will function.

But if someone did that then the included headers would take precedence, UNLESS someone realizes to specify, and if someone was worried about vulnerability then it would be a bigger deal.

Also, what kind of license implications would bundling the OpenSSL headers have?

@sodabrew
sodabrew commented Aug 7, 2015

There's an important detail: you can replace the OpenSSL DLLs in the RubyInstaller directory with compatible DLLs. In very great likelihood they would have identical headers. If you replaced an OpenSSL 1.0.0k DLL with a 1.0.0m DLL, that is supposed to work, and if it didn't, that would be considered a bug by OpenSSL and a corrected version would come out later.

If you replaced an OpenSSL 1.0.0k DLL with a 1.0.2x DLL, the application may succeed or may fail as the functions and function signatures may have changed between versions, and this would not be considered a bug.

Licensing: https://www.openssl.org/source/license.html The only part that differentiates between source and binary redistribution is this:

 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions
 * are met:
 * 1. Redistributions of source code must retain the copyright
 *    notice, this list of conditions and the following disclaimer.

You should already be shipping the LICENSE file in your binary package, so there is no change required if you include the headers (considering them as sources) in the package as well.

@sodabrew
sodabrew commented Aug 7, 2015

Also, I cannot figure out how to install Knapsack packages. Am I just supposed to extract them to C:\ !?

@Azolo
Member
Azolo commented Aug 8, 2015

You can extract them where ever. They don't "install" per say, windows doesn't have an common place to store shared libraries, so you extract the dll files somewhere and tell rubygems where to look for them like in this example.

Now, I still don't think that we should ship with the headers, but I do think that there is probably a better way to distribute them and let people know how to use them.

The context for them being a dynamic-link library instead of a static-link library is for the stdlib, pretty much all of the "dependencies" are dynamic-link to make the stdlib work. That way you can call them on demand and they don't get loaded when you don't always need them. As you saw from the OpenKnapsack package there is a lot more in the bin folder than just the two dll files. We only ship the bare minimum needed to make Ruby work.

@sodabrew
sodabrew commented Aug 8, 2015

Your link proves my point. Please ship headers that match the binaries so that I can compile and link and run without requiring users to find, download, extract, and manually configure external items that will be identical to what you could ship with RubyInstaller in the first place.

@Azolo
Member
Azolo commented Aug 8, 2015

First off, you bring up some valid points. I'm sorry if it feels like I'm pushing back against you even though you have some good points. I honestly am glad that you brought this conversation up in this way. Sincerely, Thank you.

But the bottom line is, I don't know how people are going to use RubyInstaller and RubyInstaller is only intended to be a barebones implementation that doesn't add a bunch of extra stuff. As a result RubyInstaller is used as a starting point for a bunch of other installers, including RailsInstaller, Shoes, Chef, Puppet, and I think Heroku. I'm sure there are others out there as well. As a result when we package things we have to make sure only to package what is absolutely necessary for Ruby to run, which provides building blocks for others.

BUT, we do also handle a bunch of other little things. This is one of them and IT IS a problem spot. But because of the things mentioned in the previous paragraph, we can't make assumptions that it will be useful to everyone. In fact, I have to question if it would be harmful most of the time. As a result, most of us around here have prejudice, sometimes to a fault, of adding anything to the installer, DevKit, etc.

The end result is a pretty good clean installer. It might not meet the needs of everyone, but anyone can make it meet their needs. But I'll admit it requires knowledge that almost any new comer to Ruby probably doesn't have. So, while it probably is a solution to ship the headers with RubyInstaller, I feel like there's a better solution that would also correspond with what we're trying to accomplish.

@sodabrew

eventmachine/eventmachine#627 (comment)

@larskanis has offered to update rake-compiler-dock to help binary releases of Windows gems to link against the RubyInstaller libraries, if headers were provided. Also he may weigh in with a perspective on this conversation!

@luislavena
Member

@sodabrew is not just shipping of headers, but also linking libraries that is required.

More on that, it also requires modifications to rbconfig.rb so when compiling and linking extensions, it knows where to find those bundled headers/libs that are not in the standard MinGW/GCC search locations.

That also means if someone wants to test something against a different version of OpenSSL, zlib or something (like testing ruby/openssl gem) they will be affected by that forced search path used in rbconfig.rb

There is no win/win scenario for this, but if that definitely helps reduce the friction, a patch to the installer process (and the customization of rbconfig.rb) will be welcomed.

Cheers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment