Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support the openssl standard library #61

Closed
chrisseaton opened this issue Feb 15, 2017 · 23 comments
Closed

Support the openssl standard library #61

chrisseaton opened this issue Feb 15, 2017 · 23 comments
Assignees
@chrisseaton
Labels
enhancement

Comments

@chrisseaton
Copy link
Collaborator

We are implementing support for openssl using Sulong as described in our RubyConf talk Ruby's C Extension Problem and How We're Fixing It.

I'll comment on progress in this issue. Feel free to ask questions or contribute in any other way.
@chrisseaton chrisseaton self-assigned this Feb 15, 2017
This was referenced Feb 15, 2017
Closed
Closed
Closed
@ylluminate
Copy link

ylluminate commented Feb 15, 2017 •
edited
Loading

sulong looks tremendous. Where can we follow along the progress of getting OpenSSL to work via Sulong? From what I quickly gathered, and perhaps I'm wrong, there will be some kind of a fork of or commits to OpenSSL that will get it prepped to compile properly with sulong and this is where your effort is going to come in in terms of what you're saying here. Ultimately it would seem that this will be a support step for rvm installation as it will have to grab this version of OpenSSL or compile it itself via sulong during the install process...

This is quite a selfish inquiry as I'd love to follow along (and probably many others) so that we could get a feel for how to convert other programs or libraries over with sulong and this seems like a perfect opportunity! :)

@ylluminate ylluminate mentioned this issue Feb 15, 2017
Closed
@chrisseaton
Copy link
Collaborator Author

Where can we follow along the progress of getting OpenSSL to work via Sulong?

That's what this issue is for.

From what I quickly gathered, and perhaps I'm wrong, there will be some kind of a fork of or commits to OpenSSL that will get it prepped to compile properly with sulong and this is where your effort is going to come in in terms of what you're saying here.

Yes I may modify openssl in place here

https://github.com/graalvm/truffleruby/tree/master/truffleruby/src/main/c/openssl

But I then hope to go back and fix the need for any modifications later on.

Ultimately it would seem that this will be a support step for rvm installation as it will have to grab this version of OpenSSL or compile it itself via sulong during the install process...

It should come ready to use in the GraalVM bundle.

@chrisseaton
Copy link
Collaborator Author

Where can we follow along the progress of getting OpenSSL to work via Sulong?

If you mean how to build it yourself, see

https://github.com/graalvm/truffleruby/blob/master/doc/contributor/cexts.md

@ylluminate
Copy link

Yes, wanting to follow along with your code changes in order to get an idea of the workflow at converting our own C, etc. projects over to run via Sulong. Thanks!

@chrisseaton
Copy link
Collaborator Author

I'll post things here next time I do some commits so you can see concretely what I'm doing.

@eregon
Copy link
Member

eregon commented Feb 15, 2017

From what I quickly gathered, and perhaps I'm wrong, there will be some kind of a fork of or commits to OpenSSL that will get it prepped to compile properly with sulong and this is where your effort is going to come in in terms of what you're saying here

Just to clarify, we might need to modify the code of the openssl C extension, but we do not plan on touching the OpenSSL library itself (but rather use the system-installed library).

@chrisseaton
Copy link
Collaborator Author

Today I'm looking at how to implement rb_scan_args. It's a simple function but happens to trip up a few things that Sulong doesn't implement as well as we'd like.

#72

@chrisseaton
Copy link
Collaborator Author

Trying to implement rb_scan_args as a macro #74.

@chrisseaton
Copy link
Collaborator Author

chrisseaton commented Mar 4, 2017 •
edited
Loading

Related to C extensions, I'm looking at implementing fiddle in #85.

@chrisseaton
Copy link
Collaborator Author

We've been working on C extension specs for the last week, to hopefully make it easier to run openssl without encountering so many bugs.

@ylluminate
Copy link

It really looks like you're mowing down https://github.com/graalvm/truffleruby/tree/master/spec/tags/optional/capi rather quickly.

@chrisseaton
Copy link
Collaborator Author

We can now load the openssl C extension and the associated Ruby code that sets it up. We pass most of the specs for it, but can't even successfully run the setup for the MRI tests for it. And actually trying to use it doesn't work. Progress is pretty quick now though.

@ylluminate
Copy link

You're doing fantastic. Been tracking your updates and commits. Several of us have been here in the background silently and cautiously cheering you on and are very excited!

@chrisseaton
Copy link
Collaborator Author

We can now run the setup for the MRI tests, so we've got a large number of fine-grained tests we can now work through.

@chrisseaton
Copy link
Collaborator Author

All the openssl specs now pass. There's still lots of tests, and it still doesn't actually work.

I'm having some trouble with what I think is undefined behaviour in openssl which Clang happens to accept but Sulong is more strict about and detects as undefined.

fad82d0

@chrisseaton
Copy link
Collaborator Author

openssl seems to basically work now for making requests that use HTTPS. We still need to integrate so it's available by default.

@DemiMarie
Copy link

@eregon I am glad that the system-provided OpenSSL library is being used. A lot of the crypto stuff makes assumptions regarding timing that might break on a JIT, and besides much of it is in assembly anyway. And none of it accesses Ruby objects.

@chrisseaton
Copy link
Collaborator Author

@DemiMarie yes definitely right for OpenSSL, but I should just also mention that Sulong can actually handle inline assembly as well!

@DemiMarie
Copy link

@chrisseaton Nice! Does it treat it the same way as ex. GCC or Clang?

@chrisseaton
Copy link
Collaborator Author

I think it still interprets (and just-in-time compiles) the assembly instructions, rather than copying them into machine code. It all has to work on a standard JVM where you can't just emit machine code whenever you want.

@chrisseaton
Copy link
Collaborator Author

openssl is now enabled by default on master. We're now merging this into the next GraalVM release.

@ylluminate
Copy link

ylluminate commented Jul 29, 2017 •
edited
Loading

Great work guys. Really excited to see you forging ahead here.

Giving you a little love over on Reddit.

I think Rails users will be most interested in this at the moment for server purposes and I think that's your primary target right now from listening to your talks, etc. Please correct me if I'm wrong.

@chrisseaton
Copy link
Collaborator Author

The next version of GraalVM will include this by default.

chrisseaton added a commit that referenced this issue May 30, 2018
@chrisseaton
Merge pull request #61 in G/truffleruby from tck-prep-2 to master
5e705df 
* commit 'fa94243553b8ac15b5acc8f0e6223defea6ed212':
  Fixes
  Fix integer coercions in float
  Revert "Bignum should be IS_BOXED and UNBOX to a double"
  Documentation fix
  Lint
  Fix coercion guards
  Minor fixes
  Float#== specialisation too broad
  to_s and to_str on foreign objects
  Foreign objects should respond_to?(:inspect)
  Truffle::Debug.java_to_string becomes Truffle::Interop.to_string
  Coercion of foreign numbers to Ruby numbers works
  Truffle::Interop.unbox_if_needed
  Bignum should be IS_BOXED and UNBOX to a double
  Make the backtrace interleaver tolerant to running out of Java frames
  Let Truffle::Debug.print print anything
  Most objects with #[] shouldn't have KEYS, so people don't READ them
chrisseaton pushed a commit to Shopify/truffleruby that referenced this issue Nov 13, 2019
@shopify-shipitnext
Merge pull request oracle#61 from Shopify/process-status-equal
5a86886 
Process::Status#==: compare raw process status

Merge-Requested-By: chrisseaton
Merge-Queue-Digest: ded691629c2d53cdaf513f69b863a5ed00dc40f182ec0b71a1707f87a8fdef9d
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@chrisseaton chrisseaton

Labels
enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@eregon @pitr-ch @ylluminate @chrisseaton @DemiMarie