Discovery API - Alternatives

[JKRhb]

When trying to implement the current discovery API I got the impression that its current design could be simplified by using a structure similar to the subscription/observation of events/properties. In particular, I had problems with implementing step two of the next() method:

If the active property is true, wait until the discovery results internal slot is not empty.

Correct me if I am wrong, but it seemed to me as if this would only be possible by continuously checking the results, using a function like setTimeout or within a while loop (are there any implementation examples?). As this didn't seem that optimal to me, I tried out a callback based approach instead (similar to the Subscription interface and the observeProperty/subscribeEvent methods) which worked quite well so far.

I therefore wanted to suggest changing the signature of the discover method (or its potential successors which are discussed in #222) to something like

ThingDiscovery discover(DiscoveryListener listener, optional ThingFilter thingFilter = null);

where the DiscoveryListener function signature could look like the following:

type DiscoveryListener = (thingDescription: ThingDescription) => void;

A minimal example could look like this:

async function handleDiscovery(thingDescription: ThingDescription) {
    const consumedThing = await wot.consume(thingDescription);
    await consumedThing.invokeAction("foo");
}


wot.discover(handleDiscovery, { method: "direct", url: "coap://example.org/td" );
// Calling stop() can probably be omitted when the "direct" method is being used.

Here, the discovery process is initialized using the direct method. When a Thing Description has been retrieved successfully, the handleDiscovery function is called which consumes the TD and interacts with the Thing. If I am not mistaken, a similar approach (using a class called Observable) was already part of the specification some time ago but I still wanted to open this issue to discuss it as an alternative to the current discover method.

I am looking forward to your feedback!

[danielpeintner]

Scripting Call 2022-01-17

• collect pros and cons
• collect/write examples?
• align with other API style, subscription mechanism
• in case of same power: difficulty of implementing one or the other (e.g, blocking until new data is available)

[danielpeintner]

I have been thinking about this proposal and I like it. It well aligns with what we have for Subscription.

w.r.t. "power" I think one can achieve the same, e.g.,

• the discovery listener can stop() if the desired result has been found

[JKRhb]

@danielpeintner Thank you for the feedback!

w.r.t. "power" I think one can achieve the same, e.g.,

• the discovery listener can stop() if the desired result has been found

I would think so, too :) The only aspect that is probably not covered at the moment is pagination, but maybe this could be handled by adding additional parameters to the DiscoveryListener type definition. I guess we need an example to figure out how this use case could be handled.

Also, I had two ideas which could probably be further improvements to the approach from above. The first one would be to add a stop() callback parameter to the DiscoveryListener type definition, so that you can easily stop the discovery process once a certain condition is fulfilled. In the example below, the discovery would be stopped once a TD with a "foo" action is found. Maybe this approach could also be used to handle pagination in the background -- another "page" of TDs is only fetched from a directory if the discovery hasn't been cancelled by the user based on their own decision criteria.

The other idea I had was to turn the start() method of the ThingDiscovery object into a restart() method which could be used to reinitiate a discovery process. A separate start() method, so I thought, is probably not needed as the discovery process should start once the ThingDiscovery is initialized. Let me know what you think about this :)

async function handleDiscovery(thingDescription: ThingDescription, stop: () => void) {
    if (thingDescription.actions.has("foo") {
        stop();
        const consumedThing = await wot.consume(thingDescription);
        await consumedThing.invokeAction("foo");
    }
}


const discovery = wot.discover(handleDiscovery, { method: "directoy", url: "coap://example.org/tdd" );
// ... some time passes, then the discovery process is started again
discovery.restart();
// Then the discovery process is cancelled "manually".
discovery.stop();

[danielpeintner]

Some questions...

Also, I had two ideas which could probably be further improvements to the approach from above. The first one would be to add a stop() callback parameter to the DiscoveryListener type definition, so that you can easily stop the discovery process once a certain condition is fulfilled.

What is the difference of having stop() on ThingDiscovery ?
For subscriptions we have stop() also on Subscription.

The other idea I had was to turn the start() method of the ThingDiscovery object into a restart() method which could be used to reinitiate a discovery process.

The alternative would be to create yet another ThingDiscovery OR is your assumption that restart() does not report again the matches it found before being stopped?

[JKRhb]

What is the difference of having stop() on ThingDiscovery ?
For subscriptions we have stop() also on Subscription.

The additional stop() would mostly be for convenience as (if I am not mistaken) you would need a construct like this to access the discovery object from the handler function:

let discovery;
async function handleDiscovery(thingDescription: ThingDescription) {
    if (discovery != null) {
        discovery.stop();
    }
    ...
}

discovery = wot.discover(handleDiscovery, { ... );

This stop() callback would be the same as the stop() method, though. However, adding it as a callback would avoid the need for the null check and make stopping the process from the handler a bit more elegant.

The alternative would be to create yet another ThingDiscovery OR is your assumption that restart() does not report again the matches it found before being stopped?

That would also be an option :) I thought that restart() would behave exactly as the initial discovery. This could be useful if you don't store the discovered TDs but perform a single action on them and then throw them away, for example. Or if you a common discovery pattern that will be restarted frequently.

[JKRhb]

I've been thinking about the restart() question a bit more and now arrived at the conclusion that the start() already behaves like a restart. As you might want to instantiate a ThingDiscovery object "manually" I think keeping the name start() could therefore make more sense.

[zolkis]

For API design here, let's also keep in mind browser event loop considerations. Probably known content, but anyway :).
What the heck is the event loop anyway? (youtube)
In the loop (youtube)

Edit. For instance, we should check every place we call callbacks in the spec, as they need to move to "queue a task for..." constructs for that.

[danielpeintner]

relates to #222

[danielpeintner]

Scripting Call 2022-01-31

• @JKRhb creates PR (or updates the issue) with WebIDL changes
• consecutively we discuss the updates

[zolkis]

A possible variant for a callback based discovery API uses the Subscription object, that already has an active flag and a stop() method that can also accept interaction options.
However, the algorithms should have steps on how exactly the Subscription should be used.

callback DiscoveryHandler = undefined(ThingDescription);

partial namespace WOT {
  Subscription discover(DiscoveryHandler handler, optional ThingFilter filter = null);
};

typedef DOMString DiscoveryMethod;

dictionary ThingFilter {
  DiscoveryMethod method = "directory";
  USVString? url;
  object? fragment;
 };

[zolkis]

A possible API with direct functions.

callback DiscoveryHandler = undefined(ThingDescription);

partial namespace WOT {
[SecureContext, Exposed=(Window,Worker)]
interface WotDiscovery {
    Promise<ThingDescription> direct(USVString url);
    Promise<Subscription> directory(DiscoveryHandler handler, USVString url);
};

With examples like:

let discovery = new WotDiscovery();
let td = await discovery.direct("https://mythings.my/thing2");
let subs = await discovery.directory(handler, "https://directory.mythings.my/");
...
if (subs.active) {
   await subs.stop();
}

That allows for a small variation.

callback DiscoveryHandler = undefined(ThingDescription);

partial namespace WOT {
[SecureContext, Exposed=(Window,Worker)]
interface WotDirectDiscovery {
    Promise<ThingDescription> start(USVString url);
};

[SecureContext, Exposed=(Window,Worker)]
interface WotDirectoryDiscovery extends Subscription {
    Promise<undefined> start(DiscoveryHandler handler, USVString url);
};

The example becomes:

let discovery = new WotDirectDiscovery();
let td = await discovery.start("https://mythings.my/thing2");

let discovery = new WotDirectoryDiscovery();
await discovery.start(handler, "https://directory.mythings.my/");
...
if (discovery.active) {
    await discovery.stop();
}

Edit: changed the direct discovery to a promise, as we don't need a handler there, and it also manages the errors.
Note that the directory method is missing ways to handle errors during discovery (however, I don't see necessary to expose those in discovery). We could pass an error callback as well, if needed.

@relu91 PTAL

[zolkis]

Yet another variation. I like that direct() returns a promise that resolves with a TD.
If somehow we could expose also directory discovery with that pattern, that would also solve the follow-links problem.
That means it could return an iterable , which brings back the next() method we wanted to get rid of :).

So it might be that the current discovery API is still the most idiomatic and flexible:

• it implements an iterable, i.e. a standard and same interface for all discovery methods;
• therefore it has flow control (script calling next()), unlike with callbacks or events (though this is not really a problem because of how JS engines work)
• it handles errors both before and during discovery (the callback alternative would need to pass 2 callbacks); however, the way it is specified now, is the same as the promise version of directory discovery (only the last error available that was so severe as to stop discovery).
• extensible with more discovery methods without an API/ABI break (the others can do the same).

Now we have a lot of options to discuss :).

[relu91]

Yet another variation. I like that direct() returns a promise that resolves with a TD. If somehow we could expose also directory discovery with that pattern, that would also solve the follow-links problem. That means it could return an iterable , which brings back the next() method we wanted to get rid of :).

So it might be that the current discovery API is still the most idiomatic and flexible:

• it implements an iterable, i.e. a standard and same interface for all discovery methods;
• therefore it has flow control (script calling next()), unlike with callbacks or events (though this is not really a problem because of how JS engines work)
• it handles errors both before and during discovery (the callback alternative would need to pass 2 callbacks); however, the way it is specified now, is the same as the promise version of directory discovery (only the last error available that was so severe as to stop discovery).
• extensible with more discovery methods without an API/ABI break (the others can do the same).

Now we have a lot of options to discuss :).

I agree that the current API can cover the points above. Still, it is an overkill for the simplest use-case:

• Return the list of TDs at this location

I agree there's still a lot to discuss, taking also in mind that the Discovery specification is still ongoing (even though close to the feature freeze).