Skip to content

Conversation

@dmdashenkov
Copy link
Contributor

@dmdashenkov dmdashenkov commented Oct 28, 2021

In this PR, we change the client-server interaction protocol around entity state and event subscriptions.

When a client subscribes to server updates, the server allocates some resources in order to keep the subscription working. Unfortunately, we cannot trust the client to close all the subscriptions (because of network errors, page closures, programming mistakes). Thus, the server will eventually close the subscription by itself.

In order to ensure that the subscription is alive as long as the client needs it, the client sends keep-up requests as a way of saying that the client is still there and the subscription is still relevant.

Batch requests

Previously, such requests were sent individually for each subscription. This created an unnecessary load on the server.
From now on, the requests are merged into one, so that the server can handle them in bulk. The same change is applied to the subscription cancellation requests.

Explicit lifespan

Previously, the clients figured out when to send the keep-up requests based on a convention. Now, the subscription and keep-up requests include the time for which the client needs the subscription, and the responses to those requests contain the actual expiration date & time for each subscription. So, the clients may choose to request a longer or a shorter prolongation and the server may choose to fulfill the request or set its own expiration time to a subscription no matter what.

In the current version, the JS client still sends the keep-up requests every n seconds, effectively ignoring the available expiration date & time data. However, now there is a possibility to alter the client lib to work with that data without extra changes to the server lib.

Compatibility

These changes to the protocol are breaking and aren't built to be compatible with the previous protocol. This means that the updates to the client and the server must be done in sync.

There are also breaking changes to the Java server library. In particular, the generic parameters of the subscription servlets and SubscriptionBridge are removed.
The API of the JS client library, on the other hand, is unchanged.
Also, the 3 subscription HTTP endpoints, Subscribe-, SubscriptionKeepUp-, and SubscriptionCancelServlet are not changed. In most cases, the users will not have to alter any code around those servlets. The SubscriptionBulkKeepUpServlet and the SubscriptionCancelAllServlet, introduced in the latest patch to the v1.x, are removed.

@dmdashenkov dmdashenkov self-assigned this Oct 28, 2021
}

/** Executes a command in the project [rootFolder]. */
private fun execute(vararg command: String): String = Cli(rootFolder).execute(*command)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SpreadOperator: In most cases using a spread operator causes a full copy of the array to be created before calling a method which has a very high performance penalty.
(at-me in a reply with help or ignore)

@dmdashenkov dmdashenkov marked this pull request as ready for review November 1, 2021 17:42
@dmdashenkov dmdashenkov requested a review from armiol November 1, 2021 17:43
@dmdashenkov
Copy link
Contributor Author

@armiol, PTAL.

Copy link
Contributor

@armiol armiol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dmdashenkov please see my comments, mostly on naming and documenting. Other than that, LGTM.

this._keepUpOrGiveUp();
}

_cancelClosed() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would document this one and the one below.

The first one is pretty mysterious by name. And the second requires some clarifications on that "Or".

private final StoredJson originalJson;

private NodeValue(JsonObject value) {
private NodeValue(JsonObject value, @Nullable StoredJson originalJson) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please document the ctor. It's now a bit odd to see the value and the original JSON out of the context of any operation.

return originalJson.as(cls);
} else {
String jsonMessage = value.toString();
return fromJson(jsonMessage, cls);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would put this fromJson into @see section of the method's Javadoc — as more clarification is required on how the value is parsed etc.

It will be also helpful to explain how this originalJson (which I have previously requested to document along with the ctor) plays here.

*/
@Internal
public <M extends Message> M as(Class<M> cls) {
return Json.fromJson(value(), cls);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similarly to the one above, let's reference fromJson in @see.

Also, let's have them both either statically imported, or both used with the class name. Now, it's half-and-half.

default boolean isExpired() {
Timestamp validThru = getValidThru();
Timestamp now = Time.currentTime();
return compare(now, validThru) > 0;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Someone should explain this > (and not >=).

Ideally, that could be somewhere in the Protobuf definitions. But I cannot see any changes on that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added notes about when does the time elapse in Java and in Protobuf.
It is also hinted in the name "valid thru", (not "valid until").

It might be important for the users to understand this detail for the full picture, but, in practice, I don't think it matters much if the subscriptions become stale at the moment or after the moment. Unlike, say, delivery, this kind of granularity should not be expected, as the staleness is validated at the server but the prolongation requests are created at the client. I doubt there will be a case when this really matters to the API user.

import "spine/client/subscription.proto";
import "spine/core/ack.proto";

// A subscription initiated by a web client.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's have a file-level comment explaining how these definitions are different from the one provided by the subscription service.


// A subscription initiated by a web client.
//
// The subscription may have `extra` properties required by the web client.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How these properties are used? Do they characterise the subscription in case the web client "forgets"?

Please have some usage example for those. It's now a bit blurry and generic.

// The server may add this time or another duration. The updated subscription expiration date
// is provided in the response.
//
// If the client wishes to *reduce* the lifetime of the subscription, it can do so by providing
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh my god :) What if we submit something really-really negative? I.e. Duration.of(-65535, YEAR)?

I would not support negative values ever, as it will bring much more issues down the road. And — I suspect — the increased volume of the code to handle the edge cases. It is going to out-weight the benefits
— from all cases, it's probably <1% that will use these negative values.

Also, negative Durations looks really weird.

}

// The response to the `Cancel` request.
message Cancelling {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would name this Cancellations — it's much more clear.

}

// The response to the `KeepUp` request.
message KeepingUp {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if we name all responses and outcomes as events? I mean, the requests are already very commanding.

E.g. —

SubscriptionsKeptUp,
SingleSubscriptionKeptUp,
SubscriptionsCancelled.

That would eliminate all the dance with "KeepingUp" and "Cancelling" — which are really the same Outcomes, but "..damn it, the word is used already". And honestly, to me "Cancelling" is a process, not a result, or a bunch of results.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've applied this suggestion partially.
The "top-level" types, i.e. Subscribing, KeepingUp, and Cancelling are renamed to sound more event-ish. However, I don't want to use SingleSubscriptionKeptUp because it's a bit misleading. It might be kept up, or it not be due to an error.

In a normal command-event schema, we would throw a rejection here. I guess, the HTTP analog of that is an error code. However, the point here is to compose operations on multiple subscriptions under a single request/response pair. Thus, we return an "OK" response with an error as one of the outcomes.

@dmdashenkov
Copy link
Contributor Author

@armiol, PTAL again.

@dmdashenkov dmdashenkov requested a review from armiol November 2, 2021 14:59
Copy link
Contributor

@armiol armiol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dmdashenkov LGTM with a single comment.

I suggest returning to the discussion on the naming when we are back to the office. It seems to be the case for brain-storming.

*
* The server responds with status for each subscription. If a subscription could not be found on
* the server (e.g. because it was already closed), the server responds with an error.
* In this case, the client removes the subscription. The callbacks will no longer receive
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we please not leave "updates" alone? Maybe, "receive" could join?

@dmdashenkov dmdashenkov merged commit b94249b into master Nov 2, 2021
@dmdashenkov dmdashenkov deleted the bundle-keep-up branch November 2, 2021 18:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants