Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Embed API #1099

Closed
wants to merge 2 commits into
from

Conversation

Projects
None yet
8 participants

nasser commented Aug 5, 2011

This is an implementation of a basic embed API for Rubinius as originally discussed here. It builds on @gaffo's work by implementing the methods he stubbed out and adding a few more where they were needed.

The code passed my informal printf tests, but testing it formally is difficult as the current implementation crashes when a Rubinius context is stopped and restarted. This makes running multiple isolated tests from one processes impossible at the moment. I'm testing by using these files and statically linking to librubinius-static.a (generated from my fork). The program generates the following output.

** rbx_require_file
./req.rb required

** rbx_eval_file
a.rb
foo method called
caesar

** rbx_eval_file
b.rb
This is file b

** rbx_eval
Hello, World!
Hello, Woorld!
Hello, Wooorld!
Hello, Woooorld!
Hello, Wooooorld!
Hello, Woooooorld!
Hello, Wooooooorld!
Hello, Woooooooorld!
Hello, Wooooooooorld!
Hello, Woooooooooorld!
Hello, Wooooooooooorld!

This is a very early draft and everything is up in the air, subject to change. But it works well enough and I feel it is a good place to start the conversation around the nature of the API.

Owner

evanphx commented Oct 11, 2011

This can't be merged in because it doesn't yet deal with GC lifetimes of objects. We need to figure that out before we can offer this API.

nasser commented Oct 15, 2011

I imagined as much – creating a second context caused all sorts of crazy errors. What needs to be done to figure the GC lifetimes out? Does something need to change in the Rubiunus core, or is it the way I am handling things in this API implementation?

gaffo commented Oct 15, 2011

I believe he means we need to define an API for releasing and acquiring handles to objects we want to use outside of the vm

Ramsey Nasser reply@reply.github.com wrote:

I imagined as much – creating a second context caused all sorts of crazy errors. What needs to be done to figure the GC lifetimes out? Does something need to change in the Rubiunus core, or is it the way I am handling things in this API implementation?

Reply to this email directly or view it on GitHub:
#1099 (comment)

nasser commented Dec 3, 2011

Is this on the roadmap? Is there any way I can contribute to this effort? I'm super interested in this functionality.

gaffo commented Dec 3, 2011

As @evanphx said, the api for the lifetime of objects either referred to outside of or bound into the VM needs to be decided or prototyped. Also who is in charge of them needs to be figured out. http://donttreadonme.co.uk/rubinius-irc/rubinius.log.20110623.html is the conversation we had a while ago discussing it.

Contributor

headius commented Mar 22, 2012

I mentioned this briefly on IRC, but I would recommend looking at JNI for the more mechanical parts of the API (runtime lifecycle, object lifecycle). I also see no reason this couldn't be developed as a generic API for embedding any Ruby impl. The functions you have so far are all easily supportable outside of rbx, and your mechanism of passing around a context is exactly what JNI does with JNIEnv.

Hopefully we can collaborate on a single API, rather than having yet another C API that only works well on one impl.

nasser commented Apr 3, 2012

I agree completely. The lack of a robust embedding API is one of Ruby's biggest shortcomings. For a language so adept at implementing DSLs, it's a real shame that those DSLs can't be used to script an application!

I'm very happy to collaborate on the design of a generic API and an implementation for Rubinius. My fellowship at Eyebeam allows me to work on it pretty much full time. I would love to hear from you all about how to move this forward.

Contributor

headius commented Apr 3, 2012

I think the best way to proceed would be to collaborate on defining a header that has no direct connection with rbx, but which defines the basic functions we would want for embedding. We can look at implementing that in JRuby in parallel and help shake out issues.

It would probably also be worth your time to read the JNI (Java Native Interface) API set, which does a very good job of allowing JVMs to be embedded in native applications (and vice versa) while keeping the VM internals well-isolated. That pattern would advise our common Ruby embedding (and eventually, common C API replacement) very well.

nasser commented Apr 4, 2012

Sounds great. I started a gist with a an initial sketch of the header. It is based on the header used in this pull request, which is in turn based on this work by @gaffo. I just removed any Rubinius/C++ dependent code.

I'll read up on the JNI. I've also been looking at the Python and Lua embedding APIs for how they handle the dynamic features of those languages.

Contributor

tmornini commented Apr 4, 2012

@nasser, thanks for picking this up and running with it. A single embedding API for Rubinius and JRuby will be absolutely fantastic.

Thanks!

gaffo commented Apr 4, 2012

@headius do you have any links handy for the JVM / JNI api's just to make the collaboration easier? Putting them in this ticket would make things easier for anyone coming in late.

Contributor

headius commented Apr 5, 2012

@gaffo The JNI APIs are documented here: http://docs.oracle.com/javase/6/docs/technotes/guides/jni/spec/jniTOC.html

You'll want to read over the list of functions, which should give a good idea of where a Ruby embedding API should go. Especially interesting are the functions for managing the lifecycle of a given object reference, which is the biggest pain for JRuby/Rubinius C extensions right now (we have to assume "handles" live forever...or for a very long time).

I should also point out Wayne Meissner's xrbapi: https://github.com/wmeissner/xrbapi

It is rather C++ oriented, but the general principals behind it are the same as JNI: pass an opaque VM context around, control access to object references, use functions for all object-internal data manipulation.

This pull request passes (merged d53c858 into 151932a).

Contributor

chrisseaton commented Jun 23, 2015

#3446 will cause issues on Mac when embedded dynamically.

Owner

brixen commented Dec 28, 2015

@nasser Thanks for your work on this. I'm still interested in making Rubinius embeddable, but there's a substantial amount of work that needs to be done initially and it's not very high priority due to very little demand.

I'm going to close this because it will need to be rewritten anyway and it hasn't seen any activity in a long time.

@brixen brixen closed this Dec 28, 2015

nasser commented Dec 28, 2015

Understood!

Owner

brixen commented Dec 28, 2015

@nasser for a bit more context, please see this comment from a related ticket. These two concepts (embedding and C++ API) are closely related and will likely end up under the same milestone.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment