Add ExposedThing property observe, unobserve and event handler support

[zolkis]

• fixes [ExposedThing] Not possible to handle event subscriptions in Scripting level #210 and [ExposedThing] Not possible to handle property observations in Scripting level  #211
• updated ExposedThing algorithms
• added missing ExposedThing algorithms

Signed-off-by: Zoltan Kis zoltan.kis@intel.com

---

Preview | Diff

[zolkis]

Needs more work. This commit is quite old and since then there has been development on how to use InteractionData. Need to check on that.

[zolkis]

@danielpeintner, @relu91, @egekorkan the last commit is a substantial update on ExposedThing algorithms taking into account the latest developments in ConsumedThing. Please check.

[relu91]

As a general comment, I think we should wait until we clarify the issue #219. I made some points throughout the review, but no major revision is needed. Great job! 👍

[relu91]

This comment may be applied also to the other handlers. I think we can move the complexity of creating interaction data from the application to the runtime. If the read handler returns any then the runtime can execute the create interaction data algorithm on that object. Application will still be able to represent payloads that does not have data schema (i.e. raw images) by returning InteractionData objects. On the other hand it could simply return a valid javascript value (i.e. true, an object etc.).

[zolkis]

Actually I should not summarize an algorithm in that paragraph, just say what does it return.
It is the algorithmic steps that should specify what to do. Removing it.

[relu91]

Ok, but I actually meant is that the promise should return any. So that the application does not need to worry about how to create an InteractionData in trivial cases. Am I missing something?

[zolkis]

We have agreed in the calls that reading functions always return InteractionData that has the value() convenience function and is created by the impl, while for writing functions we accept any, in order to relieve apps from creating InteractionData for trivial cases.

[zolkis]

That of course on ConsumedThing side.

On the ExposedThing side, the handlers also return InteractionData, and - now I get the question :) - you're asking why not allowing to return any in the handler functions and let the implementation bother with InteractionData.

Well, a handler provides total control - an app may actually want to define how to create InteractionData. In fact you're right, we could apply the same principle we applied at ConsumedThing writes, i.e. allow returning any, and if that is an InteractionData, impls should just use it as it is, otherwise should use whatever is returned in order to construct a reply, possibly involving a Body-like object (which is InteractionData-like).

So all-in-all I think this is fair. I will modify the handler signatures.

[zolkis]

It is consistent: when data comes from the implementation to the script, it is InteractionData, except eventually for Actions (discussed here) where we'd use any for the reasons explained above.

When data comes from scripts to implementation, we use any.

IMHO this is the best way and most developer friendly.
The algorithms, notes and examples should provide clarity on that.

[zolkis]

With this, apps can just pass values like 5 or five instead of being required to create streams and InteractionData out of those.
That also applies for ExposedThing handlers. The implementation can work fine with the values provided by the script, following the algorithms and can create streams if necessary. However, apps do have the options to override that by providing InteractionData themselves.

[zolkis]

Alternatively, we can provide a constructor to InteractionData that takes any data as input, and use InteractionData everywhere in the API. But that is one step more complex for apps.

[zolkis]

@relu91 what's your opinion?

[zolkis]

As discussed on the call on 15.6.2020, we leave the current interfaces and include an explanatory section on how InteractionData vs any are used.

[zolkis]

Thanks @relu91 for the meaningful comments. I have made some fixes, and explained the rest.
Please resolve the conversations that are handled.

[relu91]

I replied to some comments all the other are resolved, however it seems that I cannot mark them as resolved because I do not have the write access (see here) . It is funny that I made the review but I cannot mark it as resolved 😄

[zolkis]

Huhh. Now it should be symmetric wrt InteractionData between ConsumedThing and ExposedThing, provided I didn't mess it up :). Maybe needs a re-read :).

[relu91]

Just one last change and I think it is ok!

[relu91]

So following this argument:

Done. Actually it's the exact opposite of how we do it in ConsumedThing.
When data comes from the implementation, it's InteractionData, when it goes there, it's any.

Shouldn't params be InteractionData?

[zolkis]

Now that is tricky. In ConsumedThing the params can be a single value, array or an object (used for multiple parameters).

Here in an action handler we may get a single value, array, InteractionData, or an object with {simple values, array, InteractionData or other objects}, recursively.

This part is simply underspecified - please make suggestions how to handle it. Maybe we can fix that in a separate PR as well.

If we use InteractionData, all of the parsing falls on the app, which may be OK, but that functionality is already implemented in the ConsumedThing implementation underneath, so we'd force the app to do something that's also done by the impl - in which case why should an app bother using a WoT Scripting implementation, just make a shim over a comms lib.

If we use any, perhaps we should make a note about what to expect, and specify an algorithm on what to do when an action is invoked - it will be similar to the write steps. This seems to be the more developer-friendly option, so I'd prefer this.

The TODO is then not changing the signature, but completing the algorithm on action requests, if you agree.

[relu91]

Now that is tricky. In ConsumedThing the params can be a single value, array or an object (used for multiple parameters).

Maybe I miss this: How can we specify different parameters on the TD? As far as I know, we have only the input property to specify the Dataschema of the input. In this sense, I do not see any difference with the writeHandler.

[zolkis]

The TD spec has always had trouble with multiplicity :). The idea was that one input is enough since it can describe multiple inputs as an object, and it can also describe a single param just as a single param. It would have been more clear if input was an array of DataSchema's, but it's this way now. :)

[zolkis]

Though I admit that representing multiple params as an object is better since if all of them are optional, you can name the ones used, instead of filling the sig with nulls. A bit awkward, perhaps not well explained in the TD spec, but works.

[relu91]

So should we move this discussion in a new issue/PR? it is fine to leave the PR as it is and change it later.

[zolkis]

No, I will fix the algorithm a bit later, maybe on Friday. We'll need to review it again it seems.
About multiple properties handling, it is underspecified now (it's implementation specific how do they tell it's a multiple properties request and how they deserialize the property names). We can handle that in another PR.
Anyway, it's better to comment in the PR than on the commits directly, since this discussion will be buried :).

[zolkis]

Looks like after checking the TD, it is possible to use InteractionData when passing args to the handler, which I did.

[zolkis]

@danielpeintner @relu91
I think using InteractionData vs any should be more clear now.
Please re-read again :).
You can also read the rendered version on https://zolkis.github.io/wot-scripting-api/

[zolkis]

This PR became so huge that it does not diff any more by default, and it's anyway unusable.
It needs to be checked commit by commit for diffs.
Let's check if it's mergeable and do the further changes in separate PR's.

[danielpeintner]

I agree

• it's getting more clear
• it's getting also more and more difficult to see the diffs

Hence, it might make sense to merge.

BTW. Thanks for the figures. I believe this is useful for many people!

I have 2 additional comments/ideas:

1. WebIDL improvements (which may be a bug in WebIDL):
   I played around with WebIDL union types to make clear that a value may be of type any and InteractionData. Simply putting typedef (any or InteractionData) does not work.
   However the following does work.

    <pre class="idl">
		typedef any tdAny;
		typedef (tdAny or InteractionData ) AnyOrInteractionData;
    </pre>

I think using the type AnyOrInteractionData when appropriate makes it rather clear what to expect (I think it is better than using plain any).
Also, if we don't want to go along this path creating a typedef any AnyOrInteractionData and using this name makes it better than any.
Any thoughts?

2. Nit-Picking
   Most of the handlers have speaking names such as "readHandler" or "writeHandler". Others again are just named "handler" e.g., for setPropertyObserveHandler(). maybe we align the wording in one way or the other. But as said this is nit-picking.

[zolkis]

Most of the handlers have speaking names such as "readHandler" or "writeHandler". Others again are just named "handler" e.g., for setPropertyObserveHandler()

The observe handler is basically a read handler, so from one hand it would be misleading to name it observeHandler, and from the other hand it would be confusing to to name it readHandler.
If alignment is desired, then all of them will become handler, just as they figure in the Web IDL.

However the following does work.

Nice finding. Though what we need is any or ReadableStream when data goes from script to implementation, but it works the same way.
(From impl to script now it's always InteractionData).
Now we need 2 good names.

In the end I am not sure what do we gain, since the role inversion in ConsumedThing vs ExposedThing messes things up, and then in the algorithms we cannot save naming the types what they are, ReadableStream or any (conforming to DataSchema).

I would leave this discussion for later PR and will fix the nits now.

[zolkis]

Maybe we could try the following:

typedef any DataSchema;  // explained in algorithms
typedef (ReadableStream or DataSchema) InteractionInput;

And rename InteractionData to InteractionOutput.
But then ExposedThing will become interesting, with the role inversion.
Better names needed.

[zolkis]

We have another problem, the any in actions is not the same as any in writing, for instance.
It would be the same issue with the type shuffling.
It is not possible to just look at the WebIDL and know how data is used.
It is indeed the algorithms that explain how data is to be used.

[zolkis]

Another way would be to be able to construct InteractionData with ReadableStream or any.
But that would require full compliance from implementation for always constructing a valid object with a stream inside, even when a plain number or string would do, which would be a plain overkill for most of the use cases, for questionable esthetic benefits.

[zolkis]

Actually InteractionInput and InteractionOutput proposed above might work, together with the diagrams, and an updated section on Interaction Data, comprising InteractionInput and InteractionOutput plus the figures.

Should I update this PR with that, or should we merge this and make it in a separate PR?
(I would prefer the latter, if we can merge this one soon enough, e.g. today :)).

[zolkis]

In the end I have updated this PR, since it belongs here.
So far it looks quite good to me, better than it was with InteractionData and any, so thanks @danielpeintner for insisting. :)
Let me know if there are errors, or if it's not clear or downright confusing.

[zolkis]

Again, here you can read the clean rendered version:
https://zolkis.github.io/wot-scripting-api/

[danielpeintner]

I also think this is much better.

The only thing that still bothers me are the terms "input" and "output" in InteractionInput and InteractionOutput.

e.g., emitEvent with *input does not "read" right to me.
void emitEvent(DOMString name, InteractionInput data)

Also ActionHandler returnig Promise<InteractionInput> looks strange.

I wonder whether we can find better names...

[zolkis]

e.g., emitEvent with *input does not "read" right to me.
void emitEvent(DOMString name, InteractionInput data)
Also ActionHandler returnig Promise looks strange.

Actually it makes sense, since emitEvent provides data to the implementation (an input to the interaction response) and an action handler returns the data from the HW, i.e. provides data to the implementation (input to the interaction response). The figures make that clear as well.

The Promise returned from handlers has been discussed in the call and IIRC @relu91 requested these to be a Promise (easing server side programming). Made sense to me.

I suggest we use these as working names and discuss it later.
I am still working on a commit to improve the text on algorithms vs these new interfaces, so please don't merge it yet.

[danielpeintner]

The Promise returned from handlers has been discussed in the call and IIRC @relu91 requested these to be a Promise (easing server side programming). Made sense to me.

It was again about the name ... meaning that a return value is of type InteractionInput.
Anyhow, naming can be fixed later also.

[zolkis]

It was again about the name ... meaning that a return value is of type InteractionInput.

Again, it's not a bug, it's a feature :).
It is an input to the interaction response created by the UA.

[relu91]

Disclaimer I only read the diffs of the last commits since I already accepted the previous changes.

Great job @zolkis I think the new pictures and the renaming helps a lot to understand what is going on in the infrastructure. We did well to take time and review what we had done, thank you @danielpeintner for suggesting this👍 .

One last comment, as well about a naming issue (sorry 😃 ). The document define interactionInput as DataSchema or ReadableStream. In my opinion, the name DataSchema is quite confusing (ok less than any). The problem is the clashing with DataSchema defined in the TD spec. So one reader can interpret it as an instance of a DataSchema and not a value that satisfies a DataSchema (as correctly written later on in the document). Hence, if everyone agrees, I'll do one final change to this biblical PR: rename DataSchema to DataSchemaValue (or something better?) I guess we are finishing the names 😄 .

[zolkis]

Yes, indeed I hesitated about DataSchema, and in the past we used to have something like DataSchemaValue, so I will do that.

[zolkis]

Made some typos and amended the last commit twice.
If no objections in the next nanosecond, I will merge. :)