Rust: ops/extensions within a Worker?

[alshdavid]

I am embedding Deno within a Rust application to use as a JavaScript plugin engine and want to be able to send work to multiple Deno contexts (seperate threads) in a "round robin" fashion.

They don't need to be workers, they can be multiple independent instances of Deno.

To embed Deno I have vendored the cli folder into my project because the available Deno crates on crates.io are not sufficient to run a complete Deno instance with support for the Node.js standard library + Deno extras.

I don't need a lot of the functionality in the deno_cli crate (like the lsp, test runner, benchmark runner, etc) but it's very challenging to lean out the crate due to the many interdependencies + making changes to the vendored deno_cli package will make syncing with deno updates quite difficult.

Due to conflicting dependencies (swc, amongst others) - I have had to silo Deno into a seperate dylib crate and consume it from my main application via Rust's FFI (Rust's FFI is unstable but in my case the Deno lib is compiled specifically for the consumer using the same settings).

I have rewritten the entrypoint of Deno's CLI so that it exports the CLI commands as library entry points

deno run

becomes

mod deno_cli;

fn main() {
  std::thread::spawn(move || {
    deno_cli::deno_current_thread(async {
    
      let cli_main_worker = deno_cli::run(Flags{
        subcommand: DenoSubcommand::Run(RunFlags {
          script: "/file/path/main.js",
          watch: None,
        }),
        allow_all: true,
        custom_extensions: vec![] // <-- I added this to allow injecting my own custom extensions
        ..Default::default(),
      })

      cli_main_worker .run().await.unwrap();
      
    });
  }).join().unwrap();
}

reference

Initially, I simply tried to spawn multiple instances of Deno, each with its own channel which I use to send work to it.

fn main() {
  let mut senders = vec![];

  std::thread::spawn(move || {
     let (tx, rx) = channel::<DenoWorkerRequest>(); 
     deno_cli::deno_current_thread(async { /* as above */ });
     senders.push(tx);
  })
  
  std::thread::spawn(move || {
     let (tx, rx) = channel::<DenoWorkerRequest>(); 
     deno_cli::deno_current_thread(async { /* as above */ });
     senders.push(tx);
  })

  let deno_worker_farm = DenoWorkerFarm::new(senders);

  deno_worker_farm.send(DenoWorkerRequste::HelloWorld);
  deno_worker_farm.send(DenoWorkerRequste::HelloWorld);
}

Running multiple instances of Deno like this resulted in a segfault when more than one instance is launched (I assume this is because Deno CLI is using static globals for some internal state).

So I figured, if multiple instances of Deno can't coexist, then I should try to spawn multiple worker threads and talk to them from the host process.

I am using my custom ops to communicate to/from the JavaScript context so I figured I could just access them from within a worker however the create_web_worker_callback() function in Deno CLI does not accept custom extensions.

I tried to add custom extensions and run custom_extensions.clone() to pass them into the web worker, however this didn't work because Extension cannot be cloned.

So I tried to create a Vec<Vec<Extension>> where I would create multiple copies of my extensions (equivalent to the number of threads I will spawn) and each worker will grab their extensions with custom_extensions.pop(). This didn't work because something doesn't implement Send, so I wrapped the extensions in an Arc<Mutex<Vec<Vec<Extension>>>> but this also didn't work.

In the end, I modified the Deno CLI crate to hard code my extensions into the worker init options

This compiled and the ops were available in the workers - but obviously any changes to the vendored deno_cli package makes it harder to sync with Deno when there are updates.

However - this didn't work at runtime. Each worker calls my "hello world" extension which prints "hello world" - but it only prints once despite running on all workers.

My next experiment is to spawn multiple instances of the deno_embed.so dynamic library via libloading multiple times in the hope that I can create multiple independent Deno instances that don't share state.

Any advice would be welcome

[bartlomieju]

I am using my custom ops to communicate to/from the JavaScript context so I figured I could just access them from within a worker however the create_web_worker_callback() function in Deno CLI does not accept custom extensions.

We can accept a PR that adds ability to pass additional extensions to web workers.

However - this didn't work at runtime. Each worker calls my "hello world" extension which prints "hello world" - but it only prints once despite running on all workers.

That's most likely caused by the fact that you only initialize once and then clone. Should be solved if you can pass additional extensions to web worker and initialize them when passing.

[alshdavid]

Haha sorry for spamming your repo with questions, I'm really trying my best to get Deno integrated into my project. Here's the same question with some examples.

For the first part:

So I can't pass Vec<Extension> to the create_web_worker_callback function because Extension cannot be cloned. It looks like it's not easy to modify to make it cloneable (because of op_state_fn and middleware_fn).

I have an example with one hello_world() extension that just prints "Hello world" from Rust.

To make the op available to my workers I have to hardcode it into the WorkerOptions in:

• worker.js#592 (create_custom_worker)
• worker.js#783 (create_web_worker_callback)

Initially I tried to simply pass custom_extensions down into create_web_worker_callback however the fails to compile because Extension cannot be cloned. worker.js#531

Hardcoding the ops into deno_cli works but adds an extra step in syncing with upstream which is not ideal. Making Extension clone-able would allow me to reuse the custom_extensions argument in create_custom_worker().

The second problem:

The problem is my ops supply values into state, and I can't see a way to access the JsRuntime of a WebWorker to retrieve the values from the state.

I am passing a callback to my ops from JavaScript and will call that function from Rust at a later point in time.

Extension.register_callback(() => {
  console.log('called from Rust')
})

Normally, from your op, you'd put the callback into the op state

pub type CallbackState = Option<v8::Global<v8::Function>>

#[op2]
pub fn op_register_callback(
  #[state]: state: &mut CallbackState,
  #[global] callback: v8::Global<v8::Function>,
) {
  state.replace(callback);
}

deno_core::extension!(
  custom_register_callback,
  ops = [op_register_callback],
  esm_entry_point = "ext:custom_register_callback/src/custom_extensions/register_callback/register_callback.js",
  esm = ["src/custom_extensions/register_callback/register_callback.js"],
  state = |state| {
    state.put(CallbackState::None);
  },
);

Then later call the Js function by pulling it out of the op state within JsRuntime

let function: CallbackState = js_runtime.op_state().borrow_mut().take();

let request = {
    let mut scope = &mut js_runtime.handle_scope();
    let request = serde_v8::to_v8(scope, &[]).unwrap();
    Global::new(&mut scope, request)
  };

  let call = js_runtime.call_with_args(&function.unwrap(), &[request]);

  js_runtime
    .with_event_loop_promise(call, PollEventLoopOptions::default())
    .await.unwrap();

However when I have multiple workers storing their callbacks into their respective op states - the abstractions within deno_cli (CliMainWorker MainWorker) don't offer any obvious means to access the JsRuntime instances for the workers.

This means I can't access the JsFunctions from workers to later call from Rust.

So my workaround has been to have the JS callback remain in the op closure but block the completion of the function with a loop over a channel.

I then send the channel sender to the main thread to handle orchestrating messages.

• main.rs#L14
• main.rs#29
• worker_farm.rs#L31

This works but is a bit weird. I generally like to avoid static globals where possible.

Another question - Do workers run on their own threads? I don't see where another thread is spawned for the worker context. Is that handled internally within v8?

[mlwilkerson]

I don't have answers to this, but I've been on a similar journey without finding workable solutions.

I started with the roll-your-own-javascript-runtime repo. That one works directly with a JsRuntime in deno_core. But I want my system to have similar capabilities with respect to Node (deno_node extensions) and NPM (bring your own node modules) as I'd get from the Deno CLI.

It's been very challenging to figure out how to build up from the base in the roll-your-own-javascript-runtime example to add these additional capabilities.

So one of my attempts was to simply build at the level of deno_runtime constructing a MainWorker. I can see that a MainWorker as constructed by deno_runtime includes a lot more capabilities than a plain JsRuntime from deno_core.

The roadblock I hit there is that the ops are not available in the JavaScript. I can see that they are registered when doing something like this in the Rust code with the worker that is built from MainWorker::bootstrap_from_options():

    println!("DEBUG: worker op names:");
    for op_name in worker.js_runtime.op_names() {
      println!("\t{op_name}");
    }

The ops are listed there. And I confirmed that that skip_op_registration: false in my WorkerOptions. But if one of my ops were named, say, op_bark, then in runtime.js I add:

globalThis.bark = () => {
  core.ops.op_bark();
};

...and in my test JS code, I invoke like this:

bark();

I get the error that op_bark() is not a function. And if I were to output the value of typeof core.ops.op_bark, it's undefined.

So it seems that there's something with the MainWorker, or how it's configured, that I haven't discovered, that results in the op being apparently registered from the Rust side, yet not available on the JavaScript side.

My next experiment was to simply fork the deno CLI repo and hardcode my own extension to be added to the custom_extensions that are used in this invocation of MainWorker::bootstrap_from_options().

That worked better, after hardcoding skip_op_registration: false in the WorkerOptions there.

However, I found that within my runtime.js, if I implemented like this, core.ops.op_bark() was not available at runtime:

const { core } = Deno

globalThis.bark = () => {
  core.ops.op_bark();
};

Yet this did work:

const { core } = Deno

const { op_bark } = core.ops

globalThis.bark = () => {
  op_bark();
};

When I saw this work in the context of the forked deno CLI, I thought it might work if I took it back to my expanded roll-your-own-javascript-runtime that was based on a MainWorker from deno_runtime. But no, it still didn't work.

So it seems clear that there's something I'm missing about what enables the ops to show up in the extension's runtime.js. I had thought maybe there's something fundamental about how a MainWorker works that disallows it, but when I saw it work in the context of the hacked deno CLI, I knew that must not be the case. So it must be possible to do it with MainWorker, but I haven't figured it out.

[bartlomieju]

For security purposes Deno CLI removes most of the ops from the global object for security purposes:

deno/runtime/js/99_main.js

Lines 514 to 561 in 4f27d7c

	// NOTE(bartlomieju): remove all the ops that have already been imported using
	// "virtual op module" (`ext:core/ops`).
	const NOT_IMPORTED_OPS = [
	// Related to `Deno.bench()` API
	"op_bench_now",
	"op_dispatch_bench_event",
	"op_register_bench",
	"op_bench_get_origin",
	// Related to `Deno.jupyter` API
	"op_jupyter_broadcast",
	"op_jupyter_input",
	// Used in jupyter API
	"op_base64_encode",
	// Used in the lint API
	"op_lint_create_serialized_ast",
	// Related to `Deno.test()` API
	"op_test_event_step_result_failed",
	"op_test_event_step_result_ignored",
	"op_test_event_step_result_ok",
	"op_test_event_step_wait",
	"op_test_op_sanitizer_collect",
	"op_test_op_sanitizer_finish",
	"op_test_op_sanitizer_get_async_message",
	"op_test_op_sanitizer_report",
	"op_restore_test_permissions",
	"op_register_test_step",
	"op_register_test",
	"op_test_get_origin",
	"op_pledge_test_permissions",
	// TODO(bartlomieju): used in various integration tests - figure out a way
	// to not depend on them.
	"op_set_exit_code",
	"op_napi_open",
	];
	function removeImportedOps() {
	const allOpNames = ObjectKeys(ops);
	for (let i = 0; i < allOpNames.length; i++) {
	const opName = allOpNames[i];
	if (!ArrayPrototypeIncludes(NOT_IMPORTED_OPS, opName)) {
	delete ops[opName];
	}
	}
	}

In your own code you should prefer to use

import { op_bar } from "ext:core/ops";

instead

[Yohe-Am]

@mlwilkerson hey, so we have need to embed deno in a similar manner. You can best see some of the work we've done to enable this at https://github.com/metatypedev/ghjk. Happy to answer questions you might have.

Note: we had to fork deno to https://github.com/metatypedev/deno to make some of the modules public. Otherwise, we're for the most part using the CliFactory directly and try to avoid modifying too much to simplify keeping up-to-date to upstream Deno. A very important point for our project since we intend to expose an unmodified Deno target directly.

[mlwilkerson]

@bartlomieju and @Yohe-Am thank you both for your responses. You helped me get unstuck and I completed a working demo of this multi-threaded task runner using deno_runtime::worker::MainWorker instead of a deno_core::JsRuntime as in the original roll-your-own-javascript-runtime.

Ideally, this embedded runtime would be able to handle NPM package resolution. But I'd prefer not to have to fork the deno CLI to accomplish that. For my purposes (an internal tool), I think pre-bundling according to the Deno 2.x migration guide here will suffice.

So this seems to get me most of what I wanted.