Conversion to tokio 0.2

[dbcfd]

Convert to tokio 0.2

[dbcfd]

currently hung up on tokio-dns, looks like the future it returns isn't correct, Box<Future> rather than impl Future or Pin<Box<Future>>

[najamelan]

Have you filed an issue at tokio-dns to resolve the blocker?

[dbcfd]

Master has the required updates, so using that to keep moving forward.

…

On Tue, Sep 10, 2019, 7:07 AM Naja Melan ***@***.***> wrote: Have you filed an issue at tokio-dns to resolve the blocker? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#68?email_source=notifications&email_token=AALIKFGWLR6I3CVQD2WTRKLQI6L2TA5CNFSM4IVBHKW2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6LAOPA#issuecomment-529925948>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AALIKFEKHTFMHI4M7MJYOK3QI6L2TANCNFSM4IVBHKWQ> .

[najamelan]

Just asking some questions, I haven't fully reviewed everything.

I do feel this lacks integration testing, verifying every possible scenario, but then again, tokio-tungstenite didn't have those before either.

[dbcfd]

Just asking some questions, I haven't fully reviewed everything.

I do feel this lacks integration testing, verifying every possible scenario, but then again, tokio-tungstenite didn't have those before either.

Will look at add at least a couple more once I get something that seems like it might work.

[dbcfd]

@najamelan @application-developer-DA

Everything compiling, existing tests passing. Examples upgraded as well. Will be seeing what I can come up with for tests.

[CryZe]

This doesn't build without the tls feature. In particular here: https://github.com/dbcfd/tungstenite-rs/blob/expose-machine/src/client.rs#L49

error[E0432]: unresolved import `error`                                                                    
  --> C:\Users\Christopher Serr\.cargo\git\checkouts\tungstenite-rs-0e70558ccb2effd7\02684b4\src\client.rs:49:9
   |
49 |     use error::{Error, Result};
   |         ^^^^^ help: a similar path exists: `crate::error`
   |
   = note: `use` statements changed in Rust 2018; read more at <https://doc.rust-lang.org/edition-guide/rust-2018/module-system/path-clarity.html>

error[E0432]: unresolved import `stream`
  --> C:\Users\Christopher Serr\.cargo\git\checkouts\tungstenite-rs-0e70558ccb2effd7\02684b4\src\client.rs:50:9
   |
50 |     use stream::Mode;
   |         ^^^^^^ help: a similar path exists: `crate::stream`
   |
   = note: `use` statements changed in Rust 2018; read more at <https://doc.rust-lang.org/edition-guide/rust-2018/module-system/path-clarity.html>

[dbcfd]

@CryZe This was actually an issue in tungstenite and is fixed in snapview/tungstenite-rs#73. cargo update and this branch should build now.

[CryZe]

Nice, another issue I ran into was that the pin-project 0.4.0-alpha.10 pulled in here is causing a cargo update failure when the new hyper is also in the dependency graph. But I don't think it's reasonable to ask for you to downgrade to their version. That's a general problem with all these alpha versions because they are all semver incompatible (to the point where you can't even have a dependency on multiple, even if they don't interact with each other). Although maybe one can specify 0.4.0-alpha.*? Idk, I haven't tried, but maybe that would be better.

[dbcfd]

Matching hyper for now, since one of the reasons for this work was to use it in warp.

[CryZe]

If I'm sending two messages before trying to receive any I get:

thread 'main' panicked at 'assertion failed: !self.context.is_null()', C:\Users\Christopher Serr\.cargo\git\checkouts\tokio-tungstenite-42c59291ea04b35c\bea935e\src\compat.rs:44:13 

[dbcfd]

@CryZe Added a test which reproduces this behavior, attempting to determine cause and fix it.

[dbcfd]

The multiple write failure is due to tungstenite write_message calling flush. This means that Sink cannot be used for send, since it does not include a context.

[CryZe]

That seems to have fixed the bug for me :)

[CryZe]

Is there a reason this doesn't use Sink for sending messages?

[dbcfd]

@CryZe Sink cannot be used because tungstenite write_message is also calling flush. Sink write does not provide a context, so no context is present when write_message then calls flush.

One possibility is to separate the the write and flush into separate functions in tungstenite (that write_message uses), so then we could implement Sink in tokio-tungstenite.

[dbrgn]

Async/await is now available on Beta, so we don't even need nightly anymore! 🎉

https://blog.rust-lang.org/2019/09/30/Async-await-hits-beta.html

[CryZe]

Yeah looks like tokio and futures have released versions that are compatible with the beta, so I'd appreciate if this PR would be updated to those versions as well.

I did the changes locally here: CryZe@d4e5872

[stkfd]

Just wanted to say I'm pretty excited about this PR, I've been trying the most recent version out for a side project the last few days. It's just very smooth integration, I'd say currently this is shaping up to become the go-to for anyone building a websocket based application using async/await.

The only issue I've noticed so far is due to not implementing Sink in this version, as mentioned above - it means you can't split the stream into Sender and Receiver because StreamExt::split requires Self: Sink, which can be a killer for some applications

[anlumo]

Can someone point me to a way to read and write to the same Websocket with this implementation without Sink? Both poll_next and send need a &mut self, so I can't do both without splitting.

[agalakhov]

It is doable via .into_future() of the Stream trait for reading. The whole is likely to be called from unfold() or loop_fn(). This technique is useful for implementing state machines.

[anlumo]

I just discovered that it does work when I remove my call to map on the stream. However, I need that to be able to select on two different streams with different item types. The idea is that I have an futures::channel::mpsc as the incoming message channel from other parts of the program, so these other parts can communicate with the websocket.

I also can't use forward, since that requires a Sink as well.

Seems like this is one of the applications where the Sink issue is a killer, as @skaufhold mentioned.

[dbcfd]

@skaufhold I will look about adding a split method to this, so that Sink is not required.

@anlumo It will look something like this

while let Some(try_msg) = websocket_stream.next().await {
    let msg = try_msg.expect("Failed to receive message");
    sender.send(msg);
    if let Some(resp) = receiver.next().await {
        websocket_stream.send(resp).await.expect("Failed to receive message");
    }
}

You can also do something like

while let Some(try_msg) = websocket_stream.next().await {
  let msg = try_msg.expect("Failed to receive message");
  let response = transform(msg);
  websocket_stream.send(response).await.expect("Failed to send response");
}

The examples (especially autobahn) show this off better https://github.com/snapview/tokio-tungstenite/pull/68/files#diff-9da84c2001004ed41d861e7b1fc0850fR37

[mcseemk]

I just discovered that it does work when I remove my call to map on the stream. However, I need that to be able to select on two different streams with different item types. The idea is that I have an futures::channel::mpsc as the incoming message channel from other parts of the program, so these other parts can communicate with the websocket.

I also can't use forward, since that requires a Sink as well.

Seems like this is one of the applications where the Sink issue is a killer, as @skaufhold mentioned.

I use the following workaround:

let (mut ws_stream, _) = match connect_async(...).await {
    Ok(x) => x,
    Err(e) => {}
};

let mut heartbeat = Interval::new(Duration::from_secs(3));

'inner: loop {
    let either1 = select(receive_channel.next(), select(heartbeat.next(), ws_stream.next())).await;
    match either1 {
        Either::Left((msg, _)) => {
            // received message via channel, process it
        }
        Either::Right((either2, _)) => {
            match either2 {
                Either::Left(_) => {
                    // heartbeat, do whatever
                }
                Either::Right((msg, _)) => {
                    // websocket message received, process it
                }
            }
        }
    }
}

[nemasu]

Hello @dbcfd , I'm fiddling with the server and client examples.
The description says it broadcasts to all connected clients, but it seems to only echo messages back to the client that sent it.

[nemasu]

Hello @dbcfd , I made a PR to update the server example. Seems like you need to hit return in the client terminal(s) before any new messages show up, I'm pretty sure that's the clients fault though.

[CryZe]

What's the status of this?

[bstrie]

I believe that the acceptance of this PR was blocked on the next stable hyper release? That's just happened ( https://seanmonstar.com/post/189594157852/hyper-v013 ), so are there any remaining issues here?

[dbcfd]

@bstrie Nope :D

[dbcfd]

@application-developer-DA ready for review/merge

[bstrie]

Nit, but tokio's Cargo.toml has:

# Include nothing by default
default = []

so default-features = false is a no-op.

[CryZe]

Well that's assuming they won't add some. But I guess that's somewhat unlikely.

[vorot93]

@application-developer-DA Any chance this could be merged ASAP?

[kellytk]

What, if anything, could be done to progress this?

[bstrie]

I would suspect that the maintainers are reasonably predisposed during the holidays, although I suppose it wouldn't hurt to ping @agalakhov as well.

[agalakhov]

I'll check this during the holidays.

[vorot93]

@application-developer-DA while examples are definitely important, I strongly suggest merging this as is if it works. There is too much downstream blocked from upgrading to Tokio 0.2 because this PR has not been merged yet.

[agalakhov]

Checked this partially. This pull request requires more cleanup to be merged. I'm working on it. Sorry, it can't be merged in its current state.

[vorot93]

@agalakhov If it's in the working state it could be possible to merge as is and keep iterating in subsequent commits. You can slap an -alpha tag if you feel uncomfortable about the current quality of the port. At this point some working release is much better than no release at all!

[agalakhov]

I have to check for some potential security and stability issues first. Especially handshake code has to be carefully checked. Now it is much better than initially (no more unsafe), but I'm still suspicious about some places. I just don't want to see any CVE filed.

[agalakhov]

Ok, I checked this. While handshake submodule could be improved, I did not found any security related problems yet. It seems to be safe to merge. Thank you.