Report protocol errors that occur after subscriptions to Events and Property updates

[danielpeintner]

see discussions in eclipse-thingweb/node-wot#232 (comment)

[zolkis]

See comment.

[zolkis]

Changed the name, since it was suggesting a solution that needs debate.

The solution proposal was this: extend WotListener to include a mechanism to report errors as well. For instance a success callback and an error callback, or a success data and error data.

However, the API design has moved away from that Node.js pattern towards idioms used in browser APIs. Another proposal is to extend ConsumedThing (not WotListener) with an error event and define the data attached to it, for modeling protocol errors that happen during subscription (i.e. not during subscription, since that is already handled by the Promise). Possibly in the future this could also model other kinds of errors we currently don't handle, but without breaking function signatures.

Also, we need to extend the algorithms, for instance add error handling to the observeProperty() algorithm in step 10.

[zolkis]

In the Scripting call today, we discussed the following options:

1. extend WoTListener with a second argument error
   Con: scripts/callbacks first need to check if error is not null or undefined or is an Error, then handle that, otherwise do the main callback stuff. Not considered good programming practice.
   Pro: simple

2. extend the observe...() and subscribe...() calls with error callbacks as well
   Con: same argument as above.
   Pro: simple

3. resolve observe() and subscribe() with a subscription object that has

• an error event (for subscribing to errors)
• a cancel() or abort() method (unobserve or unsubscribe) or just implement AbortController
  there are 2 variants here:
  a) WotListener is still passed to observe() and subscribe()
  b) the notification event is moved to the subscription object as well (named like onchange) and then the observe() and subscribe() methods could become synchronous as well, returning the control object right away and handle scripting + protocol errors together on the onerror event.

4. include an onerror event on ConsumedThing that would report subscription errors (with an Error subclass defined in Scripting).

People in the call looked to prefer 3 - whether a. or b. that was not discussed, but I assume 3a was in the scope of discussions.
Please state preferences in this issue.

[zolkis]

There is a problem though with 3. and 4. above, namely that we've tried to avoid DOM Events for a reason (among which the fact that the reference implementation is node-wot).

So there is a 5th option, to use a subscription object very much like to the ThingDiscovery interface. The subscribeEvent() and observeProperty() will just take the InteractionOptions, and return the control object, which has a method that returns a Promise (error or in case of success, the value of the last registered change). In fact it's async, but with Promises, not events.

partial interface ConsumedThing {
 Promise<ThingSubscription> observeProperty(DOMString name, 
                              optional InteractionOptions options = null);

 Promise<ThingSubscription> subscribeEvent(DOMString name,
                              optional InteractionOptions options = null);
};

then we can define ThingSubscription as

[SecureContext, Exposed=(Window,Worker)]
interface ThingSubscription {
  Promise<InteractionOutput> next();
  undefined cancel();

  readonly attribute boolean active; 
  readonly attribute Error? lastError;
};

or even simpler,

[SecureContext, Exposed=(Window,Worker)]
interface ThingSubscription {
  Promise<InteractionOutput> next();  // also conveys errors
  undefined cancel();
};

EDIT: of course we could have separate PropertyUpdateSubscription and EventSubscription declarations.

[zolkis]

Option 6 would be to change the algorithm for observe() and subscribe() - as suggested in the original issue, namely that they would only resolve when canceled (unobserve() / unsubscribe() is called) and would reject when an irrecoverable error occurs in the subscription.

However that is not the intended use of Promises AFAICT.
IMHO subscribe() should resolve when the subscription was successfully created.

Based on offline discussions so far it seems 3b is the most preferred option (but using EventTarget, not Node's EventEmitter), maybe a more refined option 5 is still in the play.

[danielpeintner]

or even simpler,

[SecureContext, Exposed=(Window,Worker)]
interface ThingSubscription {
  Promise<InteractionOutput> next();  // also conveys errors
  undefined cancel();
};

I like this most. It also seems very close to "RxJS Subscriptions".
Maybe we can think about changing cancel() to unsubscribe().... but no strong feelings either.

Mhh, I did not have time to explore/implement it any further ... this might help also I think.

EDIT: of course we could have separate PropertyUpdateSubscription and EventSubscription declarations.

If there is no need I would not do so (at least I do not see any reason for doing that at the momemnt).

[danielpeintner]

FYI: node-wot internally still uses rxjs Subscription, see here

[zolkis]

So let's have examples:

try {
  let observe = await thing.observe("temperature"); 
  while (observe.active) {
    let value = await observe.next();
  }
} catch (e) {
   // ... log: observing "temperature" failed
};

I'd argue we need at least the active field, but it could be replaced by error:

try {
  let subscription = await thing.subscribe("temperaturechange"); 
  while (!subscription.error) {
    let value = await subscription.next();
  }
} catch (e) {
   // ... log: event subscription for "temperaturechange" failed
};

While the above seems okay for Property observes, it feels awkward for Events.

Compare with this:

let subscription = thing.subscribe("temperaturechange"); 
subscription.onchange = (value) => { ... };
subscription.onerror = e => { ... };
subscription.start();
// ...
subscription.stop();

[danielpeintner]

Some more generic comment:

• I don't think we need a start() call.. if I subscribe I indicated already my interest

w.r.t. code I envision something like we have in node-wot

// I can pass along next and error handlers 
let subscription = thing.subscribe("temperaturechange", next, error); 
// once I am not interested any longer I call unsubscribe
subscription.unsubscribe();

I am not event sure we need something like active/closed?

Another listener could be also complete to the initial thing.subscribe() call that may indicate that the observable itself has been shutdown...

Does that that look better for you or are we turning in circles now ;-)

[zolkis]

If we put callbacks in the observe()/subscribe() signature then it makes little sense to create a subscription object, then we could keep the unobserve()/unsubscribe() method.

Using callback + error callback in the signature is kind of lame. Also, using error as first argument (Node.js convention).

So let's see my preferences:

• I would prefer returning a subscription object and have a cancel() or stop() method on that (or implementing AbortController)
• since the implementation can stop a subscription for a number of reasons, either a status or an error should be exposed on the subscription object (the latter is simpler, but having both would be the best, since there could be a non-critical error reported and the subscription still active)
• the error could be an Error property representing the last error occured, or an error event (making the subscription an EventTarget)
• the main notification could be represented either as a callback (like it is now, option 3a), or as an event (option 3b). That means I don't prefer the next() method on the subscriber at least for events, but I'm okay using it for observe and discovery.

We're used with going in circles... and that is fine, if the radius is diminishing :).

[relu91]

I'm answering to various points presented above:

2. extend the observe...() and subscribe...() calls with error callbacks as well
   Con: same argument as above.
   Pro: simple

I do not think that this solution has the same cons as the first one. If we add another callback the developer should not check if the error is defined or do other controls. He just provides two functions. If the error is undefined we can define our own default (i.e. throw an async exception). However, I personally like this solution 3 on 5 five stars 😃 . So I kinda agree with your comment:

Using callback + error callback in the signature is kind of lame. Also, using error as first argument (Node.js convention).

On the other hand, it is exactly how RxJs Subscription is designed. Not saying that we have to blindly follow that design but just to clarify a little bit for the discussion:

RxJsObservable.subscribe(
        res => console.log('HTTP response', res),
        err => console.log('HTTP Error', err),
        () => console.log('HTTP request completed.')
    );

We can do the same as mentioned in the daniel's comment:

// First variant use function complete to know if the subscription was accepted (?)  
let subscription = thing.subscribe("temperaturechange",  
        async (res) => console.log('Temperature changed', await res.value()),
        err => console.log('Subscription connection error', err),
        () => console.log('Subscribed')
    ); 
// once I am not interested any longer I call unsubscribe
subscription.unsubscribe();

// Second variant remove the complete and matain subscribe to be async
// i.e. resolve only if the subscription was accepted (websocket connected, mqtt client connected ...)  
let subscription = aync thing.subscribe("temperaturechange",  
        async (res) => console.log('Temperature changed', await res.value()),
        err => console.log('Subscription connection error', err),
    ); 
// once I am not interested any longer I call unsubscribe
subscription.unsubscribe();

Finally, solution 3b or 3a might look promising. However, we cannot create the subscription asynchronously. Following that solution a subscription will look something like this:

// note: we cannot have an async creation of subscription
let subscription = async thing.subscribe("temperaturechange");
subscription.onsubscribe = (data)=>{/*do something*/}
subscription.ondata = (data)=>{/*do something*/}
subscription.onerror = ()=>{/*do something*/}
subscription.unsubscribe = ()=>{/*do something*/}
// once I am not interested any longer I call unsubscribe
subscription.unsubscribe();

I don't prefer the next() method on the subscriber at least for events, but I'm okay using it for observe and discovery.

👍

[zolkis]

We need to decide the behaviour of a subscription. Using 2 callbacks, after subscription one can get either notifications or errors, mixed as pleased. That is not a good representation of subscriptions (of course the algorithm may say that after error there are no more notifications if the error was non-recoverable, but still every little error will be popping up and that's not what the subscriber is after.

IMHO a subscriber is interested mainly in the notifications, or otherwise to know they will never come - eventually why they are not coming.
But what can a subscriber do if they get a subscription error? Nothing, really, other than canceling and trying to subscribe again. Which the impl could do anyway and fail only after a few attempts.

That should be taken into account when designing the API + the algorithm.

Currently what we are missing is the "sorry, notifications won't come, there is an error".
The other part, "oh, there was this error, but never mind, the subscription is still active" - I am not sure how useful that is and whether should we make a difference from the previous case and how.

If we only allow critical error reporting (subscription stopped after error), then we can go along with 1) an error event 2) or callback, or 3) status + last error.

If we allow non-critical error reporting as well, then we need the active or closed status flag on the subscription, plus the error event or callback.

If we choose the RxJS design, I'd prefer returning a Promise for the subscribe call and then the callback + error callback are added to the subscription as internal slots.

If it was a browser API, this would be the recommended solution:

try {
  const controller = new AbortController();
  let subscription = await thing.subscribe("temperaturechange", { signal: controller.signal } );
  
  subscription.onchange = data => { ... };
  subscription.onerror = error => { ... };
  
// stop subscription after 60 seconds
  setTimeout(() => controller.abort(), 60000);
} catch (e) {
  // ... subscription failed
}

Using AbortController would enable canceling multiple subscriptions with one call.

[jak0bw]

It probably does not affect the currently pending decision but I want to add a consideration to this statement:

IMHO a subscriber is interested mainly in the notifications, or otherwise to know they will never come - eventually why they are not coming.
But what can a subscriber do if they get a subscription error? Nothing, really, other than canceling and trying to subscribe again. Which the impl could do anyway and fail only after a few attempts.

That should be taken into account when designing the API + the algorithm.

Currently what we are missing is the "sorry, notifications won't come, there is an error".
The other part, "oh, there was this error, but never mind, the subscription is still active" - I am not sure how useful that is and whether should we make a difference from the previous case and how.

If the subscriber can choose between different subscription providers (e. g. different sensors for redundancy that ultimately measure the same thing) and he experiences non critical connection problems, he still might want to change his subscribtion to another one of those redundant subscription providers and thus perhaps wants to know about a non fatal connection error.

[zolkis]

If we need to convey both critical and recoverable errors, then we need to add status to subscriptions.
In rest, the jury is out there on which design we choose.
The editors had a chat today and seems that we'd be fine with

• events: subscription object is returned synchronously, extends EventTarget, and has a start() and stop() method
• or callbacks: subscription object is resolved in Promise, callbacks provided either directly as args to subscribe(), or we move them in SubscribeOptions which will contain the callbacks and InteractionOptions and has a stop() method (or equivalent cancel(), or AbortSignal.

[jak0bw]

My example might be too rare/not "viable enough" to deserve any adaptions in the Scripting API design. In the end if the subscription recovered it still should work and the Thing can switch its subscription provider when a fatal error occurred...

I just wanted to add this thought to the conversation and let the more experienced people decide if this or another similar use case is worth considering.

[zolkis]

The question always is - what are the use cases. Since quite valid use cases have been presented, we will discuss how to adapt the API, on next Monday's call.

[danielpeintner]

Scripting Call Aug 31 - Summary

• thing.subscribe("name", subscriptionOption) should return Promise<Subscription>
  • subscriptionOption contains callbacks (data/closed/errors) etc (due to increased flexibility and clarity)
  • the return object for thing.subscribe() namely Subscription should have
    • a cancel/stop method that can be used for both, property observations and event subscriptions
    • properties telling about status like still active et cetera

Please comment if summary is not appropriate @zolkis @relu91