Async client (unchanged API)

[ghost]

Okay here's a quick sketch of what a new, single-threaded client based on smol might look like.
Right now, this simple client can connect to demo.nats.io and publish messages.

@spacejam What do you think and how would you like to move forward from here?

[derekcollison]

Let's make sure we are set on our high level goals.

I think what we want to achieve is a runtime agnostic client that can easily plug into smol, tokio or async-std. Maybe default to smol and have a mush easier out of the box experience but you can swap out runtimes as needed.

That make any sense? You both are the experts of course ;)

[derekcollison]

don't need nats:// or the 4222 which is default.

nats-sub -s demo.nats.io hello

[spacejam]

it's nice that so far this is pretty tight. I think the next steps to sketch out for architectural soundness assessment are:

1. transition from connected -> disconnected state
2. transition from disconnected -> connected state if a reconnect happens within the retry bound
3. transition from disconnected -> closed state if a reconnect does not happen within the retry bound
4. transition from connected -> closed if the close method is called

[ghost]

This is just a sketch and a draft, don't look too much into the details right now (:

That make any sense? You both are the experts of course ;)

Totally. So the goal is to make using this client a seamless experience - and this client works with tokio, async-std, smol, what have you.

How? If you look at the code, this client spawns a thread that calls smol::run(), thus creating a tiny executor for its own use, which is independent from the end-user's executor. This way the user doesn't need to worry about runtimes at all - everything just works, no matter what runtime the user is running.

Really, all we need in the NATS client is a thread that polls the client() future and waits on the network using epoll.

[derekcollison]

I like that, very clean. I was worried that in the effort to support other runtimes we would have to supply a lower level future based api..

[spacejam]

@derekcollison we will be compatible with every executor, because none of the user-facing types that implement Future will have runtime assumptions.

we can make all functions async without requiring an async runtime, except for the flush commands, which we may add a flush_async method for to be non-blocking to play better with runtimes. So, this won't impact the existing API at all, it will just spin up fewer threads and rely on no mutexes in the core state machine, removing concurrency which has been at the root of the bugs I've been finding. Maybe just 1 additional method (or 2 for flush_timeout_async) in addition for non-blocking flush support.

But at this point, it's a high-level sketch, and after testing the high-level state transition architecture, we will drill down on the granularity of efforts as we move things over.

[derekcollison]

Thanks. Excited to see how this plays out and to have Rust+NATS+Smol powered services.

[ghost]

No locks on subscriptions, the whole client state is contained inside this client() function :)

[ghost]

I pushed some more changes. The client can now do basic handling of SUB, UNSUB, MSG, PUB.

The architecture is pretty clean:

• messages from the server are modeled as a Stream<Item = io::Result<ServerOp>>
• the whole client state is contained inside the client() function (no mutexes!)
• the public API (subscribe(), next_msg(), publish()) communicates with the client thread only through channels

[derekcollison]

We want to check how this impacts performance. NATS is known for performance, and so is Rust, so this needs to be a top goal. My original pass was about on par with Go more or less but needed some improvement.

[ghost]

My original pass was about on par with Go more or less but needed some improvement.

I'm curious - what tests did you use? I'd like to run the same thing.

[derekcollison]

I did a simple pub and pub/sub for baseline. We have nats-bench under the Go client that is more involved.

[derekcollison]

On my simple test we have lost about 3M msgs/sec in performance.

[redbaron]

How does it behave if it can't flush whole buffer? If it blocks instead of returning success on partial flush, this operation can deadlock. If recently flushed command make server to send huge amount of data back, it might end up blocking on on it's own writes, waiting for client to read data from TCP stream.

Timeout on flush can help to resolve deadlock

[spacejam]

this is still a draft. deadlock doesn't happen in this system because of the nats-server's flush logic that ejects slow consumers. duplex progress is incoming.

[spacejam]

@derekcollison perf will come quite easily after the control loop is minimized. are you running against the new client? on my beefy laptop, the benches for 32 byte messages top out at 2.3 million/s. did you run your baseline before the test without disabling frequency throttling and turbo boost? this can cause massive perf loss while running the identical workload twice.

[ghost]

I had to bump minimum required rust version (MSRV) to 1.40.0 because 1.39.0 is simply buggy. In particular, it can't compile this:

impl<T> Unblock<T> {
    pub async fn get_mut(&mut self) -> &mut T {
        // ...
    }
}

fixed

[spacejam]

I've renamed ConnectionOptions to Options internally, and then I added a deprecated public type alias from ConnectionOptions to Options, which allows code to keep working without any changes, but there will be a deprecation warning that people see that encourages them to move over to the new name. So, this could be released as a point release 0.5.1 without changing any existing APIs.

The addition is that Options now has a connect_async method, which returns an async-capable Connection. We've put this into the asynk module because the name async is a reserved keyword, and we figured we would follow the common pattern in other languages of renaming c's in reserved words to k, like klass etc...