-
Use-Case: Large buffers (~ 1MB) and large number of them (~millions) are to be transferred by the client and exploring hyper for it. Need a way to limit the network bandwidth used. I see a similar ticket 2163 which was closed and hence opening a new one. Is there a way to control n/w bandwidth in hyper? |
Beta Was this translation helpful? Give feedback.
Replies: 5 comments
-
As @seanmonstar said (#2163 (comment)), hyper uses streamed bodies so you should be able to control the rate of reads and writes. |
Beta Was this translation helpful? Give feedback.
-
How on earth did I miss that comment? It seems I read it but did not process it. |
Beta Was this translation helpful? Give feedback.
-
I explored it and I am still confused. My understanding is that hyper Client has a pool of tcp connections. So at a client level (i.e. This implies that consumer of the client will specify say "do not use more than 20 MB/s". This needs to be split across underlying connections, which is further complicated by the fact that number of connections is dynamic. |
Beta Was this translation helpful? Give feedback.
-
How about this? I have used the limiter in the crate: async-speed-limit Now we:
The The Reference gist: here Usage use hyper::{self, Body, body::HttpBody};
let builder = hyper::client::Client::builder();
let connector = MyConnector::with_limit(102400.0); // ~ 100KB/s Limit
let client = Arc::new(builder.build::<MyConnector, Body>(connector)); Does it work?Initial results does seem to suggest that it is working |
Beta Was this translation helpful? Give feedback.
-
Hey, although I haven't tried it yet, your solution looks good to me. Since all connections use the same limiter, it would be able to limit the overall bandwidth rate perfectly. |
Beta Was this translation helpful? Give feedback.
How about this?
I have used the limiter in the crate: async-speed-limit
Now we:
MyStream
which wrapstokio::net::TcpStream
and implementsAsyncWrite
&AsyncRead
MyConnector
similar toHttpConnector
which helps to create newMyStream
MyConnector
MyConnector
to attach clone ofLimiter
to every newMyStream
(done viafn call()
inMyConnector
)The
Limiter
is thread-safe & cheaply clone-able. They all share the same bucket wrapped in Arc.The
MyStream
'sfn poll_write
calls the limiter to "consume" and wait if not enough tokens.Reference gist: here
Usage