New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Suggestion: Look into porting to Rust and using Tokio #205

Closed
brucod opened this Issue Jun 7, 2018 · 31 comments

Comments

@brucod
Copy link

brucod commented Jun 7, 2018

Alex Crichton, from the Rust Core team, gave a great presentation on Concurrency in Rust with Async I/O using Tokio at code::dive 2017. As soon as I watched your Deno talk, I thought about his. It is long, but I highly recommend watching it.

From his talk: Using a simple hello world benchmark, they achieved almost 2mil req/sec - nearly 250k more than the next best implementation and 500k more than Go's fasthttp. Out of the box, Tokio supports 'TCP, UDP, Unix sockets, Named pipes, processes, signals, http, http2, web sockets, ...'

Some observations (Some may be wrong and definitely missing some 😃):

  • Mio crate handles all the OS async abstractions for you for each major platform bundled into Tokio. No re-inventing the wheel.
  • They don't use callbacks like Node but futures and task-scheduling.
  • Actual threadsafe concurrent/multi-threaded async operations and servers. See Rayon as well
  • Async/await Rust syntax that might map well to V8/JS's async/await implementation.
  • Rust is already poised for WebAssembly. Writing Deno in Rust, developers won't have to know Go as well.
  • Rust is young but growing - voted most loved language for third year in a row Stack Overflow's yearly survey
  • Tokio's thoughtful design appears to lend to extensible use cases and crates.
  • Tokio is written and supported by members of the Rust Core team.
  • No garbage collection

Thanks for the cool, new project and your time

Edit: Forgot to include no GC

@arjunyel

This comment has been minimized.

Copy link

arjunyel commented Jun 8, 2018

One block is Deno uses Protobufs for communication but they are not officially supported in Rust, There are third party libraries tho.

I think Rust would be a great fit, it seems Mozilla is putting a lot of work into WebAssembly 😄

@FrankFang

This comment has been minimized.

@fluxxu

This comment has been minimized.

Copy link

fluxxu commented Jun 8, 2018

I am a huge Rust fan, and I agree with the advantages you listed. But I don't think now is the best time for deno to invest to Futures/Tokio, considering

These are not deal breakers, and Rust core team is resolving them quickly. It's just not sensible to add these complexities to deno now, which itself an experimental project.
I think the better approach is to implement low-level modules using C++ now, and later port them to Rust after Rust's async io story settled.

@matiasinsaurralde

This comment has been minimized.

Copy link
Contributor

matiasinsaurralde commented Jun 10, 2018

I've been playing around with Rust and wrote this wrapper of libv8worker2, might not be useful but I wanted to explore some of the language capabilities for this: https://github.com/matiasinsaurralde/rust-v8worker2

@alexhultman

This comment has been minimized.

Copy link

alexhultman commented Jun 11, 2018

The biggest problem is not going to be making the native part of deno fast - there are tons of fast native servers. Where everything is going to be determined is how lightweight the JS wrapper of this server will be. Clearly Node.js completely failed at this, even though their native components are not all that slow.

A fast native server pushing 5 million pipelined hello world HTTP requests per second can be completely butchered by an inefficient JS wrapper, capping out at only a few thousand req/sec (see Node.js) while a well thought out wrapper with minimal amount of dynamic JS resources per request may very well keep up to 20% of the native performance, landing at 1 million req/sec from within JS (see "uws" for Node.js and Japronto for Python cases). The JS wrapping is by far the most perf. sensitive component.

@alexhultman

This comment has been minimized.

Copy link

alexhultman commented Jun 13, 2018

What I would like to see as a highly prioritized item on the agenda is a benchmark of Deno as is, with Golang. Because if Deno cannot properly retain even the perf. of the built-in Golang HTTP module then it will make no difference at all spending a lot of time rewriting the native parts in Rust. We really need to see a benchmark of Deno in action as soon as possible. This should be priority nr. 1 before every other decision really. Otherwise we end up with the same story of Node.js: an overly optimized HTTP parser written in C that does 50 billion req/sec on paper but when put into Node.js caps out at barely 18k req/sec.

@matiasinsaurralde

This comment has been minimized.

Copy link
Contributor

matiasinsaurralde commented Jun 13, 2018

@alexhultman I've been running a few benchmarks on deno with Go and also extended the profiling options in order to make it easier to find potential bottlenecks (#193). The Go profiling tools provide a great overview and you can visualize stuff like this:
captura de pantalla 2018-06-07 a la s 11 05 19
I've been working with Go interop for around two years, integrating Python, Lua and other runtimes with a HTTP server (actually an API gateway that people use to handle many requests) and I would say that the overhead of cgo calls is pretty big.
I think that the @ry minimal V8 bindings are pretty good and helps on keeping this overhead as low as possible, at the same time I've been hacking around Rust (and also learning it!) and recently ended up porting this component. I've also started to hack on a minimal deno port that uses std::sync::mpsc (have also tried crossbeam, really exciting stuff) in order to be able to benchmark and potentially compare both implementations. From my Rust experiments it seems that interop is lighter here and this could represent a big performance gain.
You may also find these benchmarks interesting, of course this is experimental and I'm sure there are many ways to optimize this.

@alexhultman

This comment has been minimized.

Copy link

alexhultman commented Jun 14, 2018

@matiasinsaurralde Your PR displays exactly what I feared: Deno will end up clogging up at the JS wrapper performing with horrible throughput. Not even beating Node.js, swapping to Tokio is completely pointless until that wrapper is fixed. The entire Deno project is completely pointless if it's just going to be a worse performing copy of Node.js.

@ry

This comment has been minimized.

Copy link
Collaborator

ry commented Jun 14, 2018

@alexhultman Removing Go is to avoid a double GC - which I think everyone can appreciate. And we're experimenting with interfaces - first pass implementations like the HTTP patch let us test. No one is claiming any release or interface commitment.

@matiasinsaurralde

This comment has been minimized.

Copy link
Contributor

matiasinsaurralde commented Jun 20, 2018

In case anyone is interested in hacking around, I've pushed a few tweaks for rust-v8worker2 and a limited implementation of Deno code in Rust here (currently you can boot up the runtime and execute a program, only readFileSync is implemented at the moment) 👍

@brandonros

This comment has been minimized.

Copy link

brandonros commented Jun 22, 2018

How do the HTTP benchmarks look when using reno? @matiasinsaurralde

@matiasinsaurralde

This comment has been minimized.

Copy link
Contributor

matiasinsaurralde commented Jun 22, 2018

@brandonros I haven't implemented the HTTP module yet, there are many options for this. I've been doing some quick benchmarks with this system call readFileSync (calling this 5000 times).

  1. Running the Deno version that uses Go:
% ./deno readfile.ts 
elapsed =  190
  1. Running reno using a debug build:
% ./target/debug/reno readfile.ts 
elapsed =  250
  1. Running reno using a release build:
% ./target/release/reno readfile.ts 
elapsed =  140
  1. NodeJS:
% node readfile_node.js 
elapsed =  77

The main advantage of Rust in this scenario is that we can interact directly with the C data structures, etc.
I've been trying to measure the overhead of using Protocol Buffers as described in #269 and started to migrate our Protocol Buffers definition to Cap'n Proto, implemented the Rust side of it but got stuck trying to parse these messages on the V8 side, found issues with the Cap'n Proto TS library that require some serious debugging.

@brandonros

This comment has been minimized.

Copy link

brandonros commented Jun 22, 2018

@matiasinsaurralde If you had to estimate, do you have faith that switching to Cap'n Proto will bring the elapsed time down from 140, close to Node's 77?

For perspective, it might make sense to show how long the pure C/pure Rust version of the same benchmark would take?

Much like @alexhultman, I'm struggling to see the benefit of running a message-passing layer on top of V8, given the performance. How does node.js currently solve this issue?

@alexhultman

This comment has been minimized.

Copy link

alexhultman commented Jun 23, 2018

For perspective, it might make sense to show how long the pure C/pure Rust version of the same benchmark would take?

For HTTP Deno would currently be about 13x off.

@ssokolow

This comment has been minimized.

Copy link

ssokolow commented Jun 23, 2018

Running reno using a releasebuild:

% ./target/release/reno readfile.ts 
elapsed =  140

NodeJS:

% node readfile_node.js 
elapsed =  77

NPM encountered something similar to this during a reliability-oriented rewrite of one of their components and wound up discovering that a surprising amount of the difference was down to how much optimization effort had been put into Node.js in areas you'd underestimate the impact of, like default buffer sizes.

@kentonv

This comment has been minimized.

Copy link

kentonv commented Jun 25, 2018

@matiasinsaurralde Can you tag me on any capnp-ts issues you're running into? I can't make any promises but I'm trying to make some time to play with it myself soon and I hope to help fix issues while I'm at it. (I created Cap'n Proto and the C++ reference implementation but not capnp-ts.)

@alexhultman

This comment has been minimized.

Copy link

alexhultman commented Jun 26, 2018

I just wanted to insert a few points.

It's easy to "dislike" comments that goes against ones gut feeling and fanatism. However disliking someone's comments does not make any change on reality, just because you don't want to accept reality for what it is. You can't pull the plug on reality and the reality as in, what's measurable with scientific methods shows a very clear picture.

If a project stems from the acknowledgement of prior mistakes in Node.js but still clings to it as the only source of reference then you're not going to get any further than a few baby steps past.

Crystal, Golang, Rust, Swift - they all have servers that outperform Node.js by multiple significant times over. You really need to stop comparing with Node.js and raise the stakes if you want to get anywhere real.

I have actual code running inside of reality in this very moment doing about 6x of Node.js, inside of Node.js via V8 native addons. It properly communicates with JavaScript callbacks and methods, has proper URL routing and SSL. It does about 1.5x of Crystal and Golang's fasthttp.

My point is that solely comparing with, and settling for, Node.js is only going to lead you to yet another Node.js. Clearly it is possible to raise performance up to Golang levels from inside V8, just do it already. Comparing with Node.js and accepting even a 4/5th the perf. is simply unacceptable.

Can we please start comparing with Golang's fasthttp or similar?

@matiasinsaurralde

This comment has been minimized.

Copy link
Contributor

matiasinsaurralde commented Jun 26, 2018

@kentonv Sure, I will continue my experiments this week and will be able to report the detailed issues on the repo. Thanks.

@brandonros

This comment has been minimized.

Copy link

brandonros commented Jun 26, 2018

@alexhultman

I have actual code running inside of reality in this very moment doing about 6x of Node.js, inside of Node.js via V8 native addons. It properly communicates with JavaScript callbacks and methods, has proper URL routing and SSL. It does about 1.5x of Crystal and Golang's fasthttp.

Link to said code? How does it compare to Rust?

@alexhultman

This comment has been minimized.

Copy link

alexhultman commented Jun 26, 2018

You always pay a quite hefty (but still somewhat acceptable) fine calling JS functions from C++. So of course it is impossible to measure up all the way to a native-only implementation. I lose about 30% req/sec when running inside of Node.js compared to as a stand-alone C++-only build.

Still, 70% of a good C++ implementation is enough to measure up to the fastest available for Golang.

@brandonros

This comment has been minimized.

Copy link

brandonros commented Jun 26, 2018

@alexhultman I want to learn Rust. Let's work together on what you think this project should be. I've tried e-mailing you but you never wrote back.

@alexhultman

This comment has been minimized.

Copy link

alexhultman commented Jun 26, 2018

I don't know a single line of Rust though, so I can't help

@brandonros

This comment has been minimized.

Copy link

brandonros commented Jun 27, 2018

@alexhultman

  1. You ignored my comment about the e-mail

  2. You ignored my comment about the link to your code that is 1.5x as fast as fasthttp.

  3. It sounds like you are saying protocol buffers/Cap 'n Proto are the wrong thing to do... proven by your fast JavaScript code, which uses callbacks and methods and native modules? If you remove node.js out of the picture and take the v8 runtime by itself, how would you compile native modules that hook into the v8 runtime but are still callable by Javascript?

  4. I also have full faith you could learn Rust to help

@robbym

This comment has been minimized.

Copy link
Contributor

robbym commented Jun 27, 2018

I've been watching this project since Ryan gave the talk, and have been wanting to contribute. I'm fairly familiar with Rust, but not with V8, Cap'n Proto or protobufs (I get the concept though). Is someone taking the helm on the Rust experiment and needs help?

@brandonros

This comment has been minimized.

Copy link

brandonros commented Jun 27, 2018

I'm currently researching what C++ code is required to hook a function into Javascript. So far, I pieced this together from things I found online:

void call_javascript_from_cpp() {
    HandleScope handle_scope;

    Persistent<Context> context = Context::New();
    Context::Scope context_scope(context);
    Handle<String> source = String::New("var testFunction = function() { return arguments[0] + arguments[1]; };");
    Handle<Script> script = Script::Compile(source);
    Handle<Value> result = script->Run();

    Handle<v8::Object> global = context->Global();
    Handle<v8::Value> value = global->Get(String::New("testFunction"));
    Handle<v8::Function> func = v8::Handle<v8::Function>::Cast(value);

    Handle<Value> args[2];
    args[0] = v8::String::New("1");
    args[1] = v8::String::New("1");

    Handle<Value> js_result = func->Call(global, 2, args);

    context.Dispose();
}

void call_cpp_from_javascript() {

}

Once I figure out the second function... I'm guessing we could use https://github.com/alexcrichton/rust-ffi-examples/tree/master/cpp-to-rust

I obviously don't know half as much as other people in this thread, but I was thinking about something simple. Send a message from Javascript to C++, get a response back. Kind of like... pubsub? For... network connections and filesystem operations? I'm sure that'll go great..........

@robbym

This comment has been minimized.

Copy link
Contributor

robbym commented Jun 28, 2018

@brandonros Looking at the roadmap, it seems like the portion that will be interacting with the VM, called libdeno, will be written in C++, which will expose a C api so that Rust can bind to it. The beginnings of it are here: https://github.com/ry/deno/blob/master/src/main.rs

As far as the messaging protocol you are mentioning, they already decided to use protobuf (Cap'n Proto in future?) and pub/sub.

@Azareal

This comment has been minimized.

Copy link

Azareal commented Jun 28, 2018

I would be careful about hello world benchmarks, they're notoriously... Useless lol
I suppose, they are alright as a rough indicator, but I wouldn't read into them too much.
In practice, it's largely how well optimised the application is rather than the stack below it.

Go seems to have a fair bit of room for improvement, at-least as far as performance is concerned (there's always some new idea bouncing around their Github), but the GC is probably always going to be a problem lurking round the corner.
The best you can probably get on that front is probably to reuse memory across requests ala fasthttp, I believe one thing fasthttp recommended there were the sync.Pools.

There are a couple of proposals including a percpu sharded value proposal which might also help with performance in some areas, but I'm not sure how much momentum it has behind it.

And there are ideas being bounced around to drastically reduce allocations in Go 2 (e.g. the conversions between []byte and string which can sometimes be costly), although I can't really say much on that, as the details seem a little vague and I'm not sure when it may be released.

All and all though, Rust was practically built for speed, while Go focuses more on productivity, so Go is probably always going to be slower, there is only so much people can do. It really depends on what you're really going for.

@alexhultman

This comment has been minimized.

Copy link

alexhultman commented Jun 28, 2018

I would be careful about hello world benchmarks, they're notoriously... Useless lol
I suppose, they are alright as a rough indicator, but I wouldn't read into them too much.
In practice, it's largely how well optimised the application is rather than the stack below it.

What is a "hello world benchmark" to you? If we're talking about HTTP pipelining of the string "hello world" then I can easily outperform Node.js by 80x (thousands of times over when counting ExpressJS). That's unlikely to ever be a useful number though. The 6x comes from a far more useful case and is not a "hello world" benchmark. Or maybe you suggest we all go write our servers in Ruby on Rails? Then what was the point of Node.js in the first place? What is the point with Deno with that opinion? Just go use Apache/PHP then.

@Azareal

This comment has been minimized.

Copy link

Azareal commented Jun 28, 2018

Edited it as I didn't articulate my thoughts as well as I should have, there are pros and cons to everything. That said, if you're on this side of the VM, then you're probably going for speed more than productivity, otherwise just write the thing in TypeScript lol

As for the performance bit, I was going off:
From his talk: Using a simple hello world benchmark, they achieved almost 2mil req/sec - nearly 250k more than the next best implementation and 500k more than Go's fasthttp. That.

https://www.techempower.com/benchmarks/#section=data-r16&hw=ph&test=json
I've seen benchmarks which put PHP as faster than Rust / C++ before, actually. There's a thing called Swoole which claims to be really fast. It's a very curious thing. Probably thanks to C in that particular case and the narrow thing it benched.

@alexhultman

This comment has been minimized.

Copy link

alexhultman commented Jun 28, 2018

Yeah those 2 mil are complete nonsense. They are achieved using HTTP pipelining which basically only benchmarks the parser/formatter. I get 5 million req/sec in my implementation when I do that.

@ry

This comment has been minimized.

Copy link
Collaborator

ry commented Aug 7, 2018

Yes we will likely use Tokio. It's about to land in #434

@ry ry closed this Aug 7, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment