Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Embed API #1099

Open
wants to merge 2 commits into from

6 participants

@nasser

This is an implementation of a basic embed API for Rubinius as originally discussed here. It builds on @gaffo's work by implementing the methods he stubbed out and adding a few more where they were needed.

The code passed my informal printf tests, but testing it formally is difficult as the current implementation crashes when a Rubinius context is stopped and restarted. This makes running multiple isolated tests from one processes impossible at the moment. I'm testing by using these files and statically linking to librubinius-static.a (generated from my fork). The program generates the following output.

** rbx_require_file
./req.rb required

** rbx_eval_file
a.rb
foo method called
caesar

** rbx_eval_file
b.rb
This is file b

** rbx_eval
Hello, World!
Hello, Woorld!
Hello, Wooorld!
Hello, Woooorld!
Hello, Wooooorld!
Hello, Woooooorld!
Hello, Wooooooorld!
Hello, Woooooooorld!
Hello, Wooooooooorld!
Hello, Woooooooooorld!
Hello, Wooooooooooorld!

This is a very early draft and everything is up in the air, subject to change. But it works well enough and I feel it is a good place to start the conversation around the nature of the API.

@evanphx
Owner

This can't be merged in because it doesn't yet deal with GC lifetimes of objects. We need to figure that out before we can offer this API.

@nasser

I imagined as much – creating a second context caused all sorts of crazy errors. What needs to be done to figure the GC lifetimes out? Does something need to change in the Rubiunus core, or is it the way I am handling things in this API implementation?

@gaffo
@nasser

Is this on the roadmap? Is there any way I can contribute to this effort? I'm super interested in this functionality.

@gaffo

As @evanphx said, the api for the lifetime of objects either referred to outside of or bound into the VM needs to be decided or prototyped. Also who is in charge of them needs to be figured out. http://donttreadonme.co.uk/rubinius-irc/rubinius.log.20110623.html is the conversation we had a while ago discussing it.

@headius

I mentioned this briefly on IRC, but I would recommend looking at JNI for the more mechanical parts of the API (runtime lifecycle, object lifecycle). I also see no reason this couldn't be developed as a generic API for embedding any Ruby impl. The functions you have so far are all easily supportable outside of rbx, and your mechanism of passing around a context is exactly what JNI does with JNIEnv.

Hopefully we can collaborate on a single API, rather than having yet another C API that only works well on one impl.

@nasser

I agree completely. The lack of a robust embedding API is one of Ruby's biggest shortcomings. For a language so adept at implementing DSLs, it's a real shame that those DSLs can't be used to script an application!

I'm very happy to collaborate on the design of a generic API and an implementation for Rubinius. My fellowship at Eyebeam allows me to work on it pretty much full time. I would love to hear from you all about how to move this forward.

@headius

I think the best way to proceed would be to collaborate on defining a header that has no direct connection with rbx, but which defines the basic functions we would want for embedding. We can look at implementing that in JRuby in parallel and help shake out issues.

It would probably also be worth your time to read the JNI (Java Native Interface) API set, which does a very good job of allowing JVMs to be embedded in native applications (and vice versa) while keeping the VM internals well-isolated. That pattern would advise our common Ruby embedding (and eventually, common C API replacement) very well.

@nasser

Sounds great. I started a gist with a an initial sketch of the header. It is based on the header used in this pull request, which is in turn based on this work by @gaffo. I just removed any Rubinius/C++ dependent code.

I'll read up on the JNI. I've also been looking at the Python and Lua embedding APIs for how they handle the dynamic features of those languages.

@tmornini
Collaborator

@nasser, thanks for picking this up and running with it. A single embedding API for Rubinius and JRuby will be absolutely fantastic.

Thanks!

@gaffo

@headius do you have any links handy for the JVM / JNI api's just to make the collaboration easier? Putting them in this ticket would make things easier for anyone coming in late.

@headius

@gaffo The JNI APIs are documented here: http://docs.oracle.com/javase/6/docs/technotes/guides/jni/spec/jniTOC.html

You'll want to read over the list of functions, which should give a good idea of where a Ruby embedding API should go. Especially interesting are the functions for managing the lifecycle of a given object reference, which is the biggest pain for JRuby/Rubinius C extensions right now (we have to assume "handles" live forever...or for a very long time).

I should also point out Wayne Meissner's xrbapi: https://github.com/wmeissner/xrbapi

It is rather C++ oriented, but the general principals behind it are the same as JNI: pass an opaque VM context around, control access to object references, use functions for all object-internal data manipulation.

@travisbot

This pull request passes (merged d53c858 into 151932a).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Aug 5, 2011
  1. @gaffo @nasser

    added non-working initial c api for embedding

    gaffo authored nasser committed
  2. @nasser

    Implements initial embed api

    nasser authored
This page is out of date. Refresh to see the latest.
View
22 kernel/embed_loader.rb
@@ -0,0 +1,22 @@
+module Rubinius
+ class EmbedLoader
+ def self.load_compiler
+ l = Loader.new
+ l.preamble
+ l.system_load_path
+ CodeLoader.require_compiled "compiler"
+ end
+
+ def self.require_file file
+ Kernel.require file
+ end
+
+ def self.eval_file file
+ CodeLoader.load_script file
+ end
+
+ def self.eval_script code
+ CodeLoader.execute_script code
+ end
+ end
+end
View
63 vm/api/embed.h
@@ -0,0 +1,63 @@
+#include "environment.hpp"
+#include "builtin/object.hpp"
+
+using namespace rubinius;
+
+typedef Environment* rbx_ctx;
+typedef Object* rbx_obj;
+
+#define RUBINIUS_CONST(ctx, n) ctx->state->globals().rubinius.get()->get_const(ctx->state, ctx->state->symbol(n));
+
+extern "C"{
+/**
+ * Create a embedded rubinius environment
+ */
+rbx_ctx rbx_create_context();
+
+/**
+ * Requires a ruby file (like require 'file' in ruby)
+ *
+ * Returns true if it executed without exception, false if there was an error.
+ * Finding out what went wrong will be defined later.
+ */
+bool rbx_require_file(rbx_ctx ctx, const char * file);
+
+/**
+ * Evaulate a ruby file
+ *
+ * Returns true if it executed without exception, false if there was an error.
+ * Finding out what went wrong will be defined later.
+ */
+bool rbx_eval_file(rbx_ctx ctx, const char * file);
+
+/**
+ * Evaluate a ruby string
+ *
+ * Returns true if it executed without exception, false if there was an error.
+ * Finding out what went wrong will be defined later.
+ */
+bool rbx_eval(rbx_ctx ctx, const char * code);
+
+/**
+ * Send a message to an object
+ *
+ * Send +msg+ to object +rcv+ with arguments specified by +argc+ and the
+ * variadic parameters that follow. All arguments *must* be of type rbx_obj.
+ */
+rbx_obj rbx_send(rbx_ctx ctx, rbx_obj recv, const char* msg, int argc, ...);
+
+/**
+ * Acquire global lock
+ */
+void rbx_lock(rbx_ctx ctx);
+
+/**
+ * Release global lock
+ */
+void rbx_unlock(rbx_ctx ctx);
+
+/**
+ * Clean up a context
+ */
+void rbx_close_ctx(rbx_ctx ctx);
+}
View
88 vm/embed.cpp
@@ -0,0 +1,88 @@
+#include "api/embed.h"
+
+#include "builtin/object.hpp"
+#include "builtin/string.hpp"
+#include "builtin/array.hpp"
+#include "builtin/module.hpp"
+
+rbx_ctx rbx_create_context() {
+ Environment* ctx = new Environment(0, NULL);
+ ctx->setup_cpp_terminate();
+
+ const char* runtime = getenv("RBX_RUNTIME");
+ if(!runtime) runtime = RBX_RUNTIME;
+
+ const char* lib_path = getenv("RBX_LIB_PATH");
+ if(!lib_path) lib_path = RBX_LIB_PATH;
+
+ std::string runtime_str(runtime);
+ std::string lib_path_str(lib_path);
+
+ int i = 0;
+ ctx->state->set_stack_start(&i);
+
+ ctx->load_platform_conf(runtime);
+ ctx->boot_vm();
+ ctx->state->initialize_config();
+ ctx->load_tool();
+ ctx->load_kernel(runtime);
+ ctx->start_signals();
+
+ ctx->run_file(runtime_str + "/loader.rbc");
+ ctx->run_file(runtime_str + "/embed_loader.rbc");
+
+ rbx_obj loader = RUBINIUS_CONST(ctx, "EmbedLoader");
+ rbx_send(ctx, loader, "load_compiler", 0);
+
+ return ctx;
+}
+
+bool rbx_require_file(rbx_ctx ctx, const char * file) {
+ rbx_obj loader = RUBINIUS_CONST(ctx, "EmbedLoader");
+ rbx_send(ctx, loader, "require_file", 1, String::create(ctx->state, file));
+
+ return true;
+}
+
+bool rbx_eval_file(rbx_ctx ctx, const char * file) {
+ rbx_obj loader = RUBINIUS_CONST(ctx, "EmbedLoader");
+ rbx_send(ctx, loader, "eval_file", 1, String::create(ctx->state, file));
+
+ return true;
+}
+
+bool rbx_eval(rbx_ctx ctx, const char * code) {
+ rbx_obj loader = RUBINIUS_CONST(ctx, "EmbedLoader");
+ rbx_send(ctx, loader, "eval_script", 1, String::create(ctx->state, code));
+
+ return true;
+}
+
+rbx_obj rbx_send(rbx_ctx ctx, rbx_obj recv, const char* msg, int argc, ...) {
+ rbx_lock(ctx);
+ va_list args;
+ va_start(args, argc);
+
+ Array* arg_ary = Array::create(ctx->state, argc);
+ for(int i=0; i<argc; i++)
+ arg_ary->append(ctx->state, va_arg(args, Object*));
+
+ va_end(args);
+
+ rbx_obj ret = recv->send(ctx->state, 0, ctx->state->symbol(msg), arg_ary);
+ rbx_unlock(ctx);
+ return ret;
+}
+
+void rbx_lock(rbx_ctx ctx) {
+ ctx->state->global_lock().take();
+}
+
+void rbx_unlock(rbx_ctx ctx) {
+ ctx->state->global_lock().drop();
+}
+
+void rbx_close_ctx(rbx_ctx ctx) {
+ ctx->halt();
+ delete ctx;
+}
View
22 vm/test/test_embedding.hpp
@@ -0,0 +1,22 @@
+#include "vm/test/test.hpp"
+#include "api/embed.h"
+
+class TestEmbedding : public CxxTest::TestSuite {
+public:
+ void test_create_and_close_context() {
+ rbx_ctx ctx = rbx_create_context();
+ TS_ASSERT(ctx != NULL);
+ rbx_close_ctx(ctx);
+ }
+
+ void test_multiple_sequential_contexts() {
+ TS_FAIL("This is crashing");
+ rbx_ctx ctx1 = rbx_create_context();
+ TS_ASSERT(ctx1 != NULL);
+ rbx_close_ctx(ctx1);
+
+ rbx_ctx ctx2 = rbx_create_context();
+ TS_ASSERT(ctx2 != NULL);
+ rbx_close_ctx(ctx2);
+ }
+};
Something went wrong with that request. Please try again.