New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
box.session.push rework #6107
Comments
I think I found a way to marry
|
The problem is that in case you can't call |
It's not a problem of pushes itself, but of notifications implementation. Imagine, a client sends a netbox request and surely knows it won't get any response until there's some event on the server. How could it tell network problems from the absence of an event? It can't happen without some additional conventions. For example, the server may always push approval at the beginning of the In other words, we can speak of it as of a netbox push-based sub-protocol and start designing it. Such an inspiring opportunity, isn't it? :-) |
Exactly. And the ticket is about problems of
Doesn't it prove my point? With the example you created you won't get any error. Regardless of its type.
That means you need a fiber to wait for the first push synchronously. I wanted to have 0 additional fibers for the subscriptions, at least on the client. Besides, it won't help if after the first push the network gets broken or the instance dies - the connection breaks then and you won't notice it. Because you don't have any fiber doing
Probably I misunderstood something? It is not a sub-protocol. It is only a matter of the API. Both on the storage and on the client (netbox). On the netbox the final goal is to never block and never create additional fibers. |
similar to #6257 |
Decided to get rid of box.session.push in 3.0+ |
There is a lot of feedback that
box.session.push()
is broken and not usable. Not many details though. This ticket attempts to gather the issues in one place.Push requires a fiber on the storage side
Push can be done only from inside of a request. Request always uses a fiber and a message object in iproto thread. This means a subscription becomes quite expensive: it wastes fiber which takes at least kilobytes of memory for its stack and it wastes a unit from
net_msg_max
.It would be probably much easier to use if the pushes could be done for any session not necessary inside of a request.
Netbox is_async and pushes are not very compatible
is_async
does not work withon_push
callback. It can only work withfuture:pairs()
. Which means it is not possible to handle the pushes in a fully async way. Either the request itself is blocking and supportson_push
, or it is not blocking but the pushes can only be consumed in the end.Second part of the async problems is that I can't detect when the subscription ends. For example, here is a function implementing subscription:
When I couldn't register it, I want to return an error and end the sub using
return nil, err
. But on the client side I won't receive anything until callfuture:wait_result()
or unless I pollfuture:result()
. Both options are not very async.I need a way to implement a fully async sub. Without having to care about the future object at all. And to be able to notice when the sub ends.
Back-pressure
Using
box.session.push
for the same session multiple times has no back-pressure control. A one can call push hundreds of times on the same session without waiting for the data to be sent. It might result into high memory usage and even OOM if there are many events.Would be good to be able either to be able to wait for the pushes flush somehow, or implement one-shot subscription. When you push something into a session and close the subscription. The client will re-subscribe when receives the previous event.
The latter looks interesting. It can be done even without
box.session.push
using normal requests. But to make them async netbox needs to be able to consume response of a request asynchronously.is_async
is cool, but it still requires to callwait_result()
in the end. It can't invoke a callback instead.One-shot sub could work if
is_async
also would allow to specify a callback and never wait for the response in a blocking way. Re-sub would happen from the callback if needed.Although if it is not
box.session.push()
, it will keep one fiber on the storage waiting for an event.This lead to a bigger question. Can we actually make possible to send a response on a request asynchronously on the server side? Without blocking one fiber for that?
The text was updated successfully, but these errors were encountered: