Clarifying how readmultipleproperties work

[egekorkan]

The TD spec is not clear on how to implement readmultipleproperties from the Consumer side. The corresponding paragraph is:

If not specified otherwise (e.g., through a TD Context Extension), the request data of the readmultipleproperties operation is an Array that contains the intended PropertyAffordances instance names, which is serialized to the content type specified by the Form instance.

The words request data exist only here in the document which is not clear where this array should be in the request. This op value raises another concern since in protocols like HTTP and CoAP it is not a good practice to send a body/payload with a GET request if this is what is hoped for. In MQTT it would not be possible at all as far as I know.

I am guessing that request data was written explicitly and it would be the job of protocol bindings document to explain how to incorporate it?

[sebastiankb]

request data = response payload

[egekorkan]

But request payload means body for the HTTP request which would then conflict with the default assumption of using the GET method which is recommended to not have a body. Maybe it is even better to leave it not defined since it would differ for protocols?

[sebastiankb]

well it is not the best practice, however, http spec does not forbid using body on a GET

https://tools.ietf.org/html/rfc7231#section-4.3.1

In MQTT it would not be possible at all as far as I know.

I would expect that there will be separate topic defined where you can provide the desired properties.

[egekorkan]

Ok, in this case, I would propose to close this issue and keep it not exactly specified since it would depend on protocols and the binding templates document can explain how it would work for different protocols.

[takuki]

In 2020-02-28 TD call, it was suggested to keep this issue open till we have a consensus on how to properly implement this feature.

[zolkis]

Do have enumerable concrete use cases for the feature?, or just filling a theoretical gap?
If we have these use cases, we should already know the feasibility and how to do it.
If we don't, IMHO we better remove it until we figure out how can it work transparently over multiple protocols.

Is there guaranteed semantics, like is readmultipleproperties supposed to always fetch multiple Properties with a single request? Or can implementations simulate it by issuing subsequent requests? (in which case it doesn't make any sense to have it).

Apps would not lose much since they still have the readproperty and readallproperty ops, where a Property can be a compound structure (array or object). So all properties desired to be fetched with a single request could be encapsulated by an object or array of Properties. This pattern can also encapsulate whether the server supports transactions.

[egekorkan]

One way to do readmultipleproperties would be if a Thing level form has "op":"readmultipleproperties" (so only readmultipleproperties) and somehow describes that the response will contain properties 1, 3 and 5. This is of course not as specified by the specification, i.e. the Consumer doesn't have the option to choose the desired properties with an Array.

Do have enumerable concrete use cases for the feature?, or just filling a theoretical gap?

There was a mention in the call of 28/2 that Mozilla had this implemented but I couldn't find it in their documentation. @benfrancis do you have an opinion?

[egekorkan]

There is also a problem if Thing wants the Consumer to provide a list of property names in uriVariables. For example, the request needed is the following:

HTTP GET to http://example.com/?props=property1,property2

which returns the values of property1 and property2.

I would expect that the Thing level form would be something like:

    "forms": [{
        "op": "readmultipleproperties",
        "href": "http://example.com/{?props}",
        "contentType": "application/json",
        "htv:methodName": "GET"
    }]

How can we specify that the values of props should be property names? Do we need uriVariables in the Thing level? Even then, would the following be understood as a list of property names by every implementation?:

"uriVariables": {
  "props": { 
    "type": "array",
    "items":{
      "type":"string",
      "@type":"properties"//this is not the correct one but something similar
      }
  }
}

[egekorkan]

Yet another aspect to think about:
Does the Consumer have to provide a list of properties or could it be that that form returns a constant subset of the properties without any input? For example, a TD has property1, property2 and property3 where the response of a form like the following contains the values of property1 and property2:

{
// ...
"forms":[
  {
    "href":"http://example.org/myfavouriteproperties",
    "op":"readmultipleproperties",
    "contentType":"application/json",
    "htv:methodName":"GET",
    "description":"returns property 1 and 2"
  }
]
// ...
}

Also, this would raise an issue on how the Consumer would know that only property 1 and 2 would be returned.

[zolkis]

At the moment the readmultipleproperties op is not specified in a protocol-independent manner in the TD spec, therefore an algorithm cannot be specified, therefore it cannot be implemented in Scripting. This should be fixed by introducing a field in Form that could carry an array of strings.

[zolkis]

But I don't consider this an important use case, as said before properties in interest could be collected in an object and exposed as a single Property. We could apply the narrow waist here.

In most GObject use cases as well, getting all properties was far more used than just getting some of them. Do we have documented use cases for it?

[egekorkan]

This should be fixed by introducing a field in Form that could carry an array of strings.

Should this array contain the name of properties whose values would be returned?

[zolkis]

Should this array contain the name of properties whose values would be returned?

Of course :). It should be an array of strings.
If one name fails, the whole request fails.

[benfrancis]

@egekorkan wrote:

There was a mention in the call of 28/2 that Mozilla had this implemented but I couldn't find it in their documentation. @benfrancis do you have an opinion?

Sorry I missed this question. Mozilla's implementation has the equivalent of readallproperties but not readmultipleproperties.

Mozilla's Web Thing Description provides a Link to a Properties resource in the top level links member of the Thing Description. E.g.

  "links": [
    {
      "rel": "properties",
      "href": "/things/lamp/properties"
    }

The Web Thing REST API specification says that a GET on the Properties resource will return the values of all properties as a map of property names to values. E.g.

200 OK
{
  "temperature": 21,
  "humidity": 50,
  "led": true
}

There is no equivalent to readallproperties or readmultipleproperties in the Web Thing WebSocket API because all property changes are pushed to the client in a propertyStatus message as they occur. The values of all properties are also pushed to the client when a WebSocket is first opened, though the spec hasn't been updated to reflect that yet.

@zolkis wrote:

Do have enumerable concrete use cases for the feature?, or just filling a theoretical gap?
If we have these use cases, we should already know the feasibility and how to do it.
If we don't, IMHO we better remove it until we figure out how can it work transparently over multiple protocols.

+1

[relu91]

IF we want to state the list of properties returned in a read/writeMultiple... I think that it becomes quite tricky to design a TD. What it comes to my mind is how I would choose the group of property I want to return, should I put every possible combination of them? So I would probably end up with a TD with N-2 read/writeMultiple... forms (N stays for the number of properties).

[zolkis]

It's quite clear the list of properties should be provided as parameters by the app, and we need the infrastructure for that.

However, I am not convinced the whole use case is needed, or useful at all. Some real examples would be needed for one app that wants to get just some properties in a single request while another app wanting a different set, but neither one wanting the full set of properties (and why not).

[relu91]

However, I am not convinced the whole use case is needed, or useful at all. Some real examples would be needed for one app that wants to get just some properties in a single request while another app wanting a different set, but neither one wanting the full set of properties (and why not).

I do agree that we need a use case here. The only thing that it comes to my mind is that one client might want to really optimize the network usage, reading only one subset of properties instead of all of them. It is really a corner case...

It's quite clear the list of properties should be provided as parameters by the app, and we need
the infrastructure for that.

Agree

[zolkis]

To summarize, the current formulation in the TD spec needs to be more specific:

If not specified otherwise (e.g., through a TD Context Extension), the request data of the readmultipleproperties operation is an Array that contains the intended PropertyAffordances instance names, which is serialized to the content type specified by the Form instance.

It should define

• what is "request data" (in different protocol bindings), (IMHO it should be an explicit Form field of type array of string),
• ok, contentType is defined by the Form, but IMHO that is not enough; what content types are suggested
• how the encoding, language are specified
• is it possible/desired to define a DataSchema for this, or should the individual properties' schemas be "reused"?
• in a note, examples for existing use cases and how are they currently solved in various protocols.

[takuki]

I think readmultipleproperties is a readallproperties with a filter specified.

When there are 100 properties, but if the Consumer only needs 50 among them, it may be useful if the Consumer can specify the filter in the data of the request (i.e. request data).

How this data in the ' readmultipleproperties` operation is encoded should be up to the protocol binding.

I think the data schema is explained in the following clause, and there is not a way to override it.

If not specified otherwise (e.g., through a TD Context Extension), the request data of the readmultipleproperties operation is an Array that contains the intended PropertyAffordances instance names,

At the time, the proposers argued this is needed for describing OPC-UA.

[egekorkan]

If this should be handled by the protocol, why is there the clause that prescribes that the needed property names should be an Array? This makes it somewhat impossible to use uriVariables, which would be the only way to do this in HTTP without breaking best practices.

Additionally, how would I specify the contentType of this Array?

Regarding:

At the time, the proposers argued this is needed for describing OPC-UA.

Then the proposers should have showed a TD or an implementation, now we have no record of how this feature was even needed in the first place.

My recommendation would be to simply remove this clause with the Array and at the same time introduce uriVariables to the Thing level, see comment above.

Furthermore, this "feature" somehow got into the official specification even though there aren't two implementations supporting it on the Thing side. Not sure what are the 2 implementations shown in the implementation report but if I search for readmultipleproperties in the wot-testing repository, I find 3 TDs that are all node-wot devices and node-wot doesn't really implement this as far as I know. (Still haven't found how to to readmultipleproperties operation with Postman on http://plugfest.thingweb.io:8083/TestThing).

[takuki]

In 2020-06-12 telecon, it was suggested to discuss in F2F meeting whether there is an use case for readmultipleproperties (for example, using CoAP FETCH). It was also pointed out readmultipleproperties may involve an filter specified by Consumer, which in general, is not easy to implement. Some protocols can package several commands in one.

[takuki]

In the virtual F2F TD session on 2020-06-26 (see minutes), the WG noted that readallproperties is sufficient, and one possibility is to remove readmultipleproperties in the coming version.

In the meantime, it was suggested to investigate more for now even though it seems removing it makes more sense.

[JKRhb]

Just to add one thought to this issue: Maybe you could simply specify the list of properties in an array within the form?

For example:

"forms": [{
    "op": "readmultipleproperties",
    "href": "http://example.com/multipleproperties",
    "properties": ["status", "brightness"],
    "contentType": "application/json",
    "htv:methodName": "GET"
}]

I think one condition for using this approach would be that that data has to be returned as an object as described by @benfrancis, like so:

{
    "status": "on",
    "brightness": 254
}

I think one possible use case for this approach could be having a Thing that consists of multiple parts. Using multiple readmultipleproperties form elements you could then differentiate between those parts and get all the relevant information from one of them while ignoring the rest.

[zolkis]

I think one possible use case for this approach could be having a Thing that consists of multiple parts.

When it's the Thing that exposes a multiple-part structure, its TD can define properties as objects (of properties) as well.
For instance, OCF provides collections, which is a resource that contains 1+ references (links) to other resources, used for creating groups, indexes, hierarchies, etc.

Even more, one can define separate Things that just re-structure the resources available on that platform, in order to present different Thing-defined views.

I think the purpose of readmultipleproperties ops would be a generic mechanism to "select a view" over the Thing, where the view is defined by the apps, not by the Thing.

One question (from IoT side) is how much support would this have from the various underlying protocols, and what complexity would it introduce to a WoT runtime / bindings to implement an explicit mapping when an implicit support is not there.

But I sympathize more with the apps side, i.e. "make IoT accessible for the Web platform" - which in principle requires simplicity of interactions, which is why the solution outlined by @benfrancis seems elegant.

If we start from the apps side API, the approach outlined by @takuki makes sense: a read ops could have filters, where

• a filter can be absent (read all, implemented at the Thing side), or
• one property name (read property, Thing side), or
• multiple property names (read multiple, Thing or app side, depending on protocol) or
• an expression/algorithm acting as a filter (either at the Thing side, or at the app side bindings, or programmatic in the app).

And why not, the result can be specified as an object (property bag).

However, because of the significant complexity overhead, the latter approach needs to be backed by a strong set of consistent use cases. I don't see the multiple-read feature as vital to the TD spec, since

• if a Thing defines views (because that platform supports them, like OCF collections), they can already be represented as compound properties (objects) in a TD, and
• in most other cases (i.e. no direct protocol support) it's a wrapper that could be done by the app or libraries and relieve the TD spec from its complexity (at least for time being).

[egekorkan]

I do not understand the comments of @zolkis exactly. Regarding what @JKRhb wrote, I think it addresses my comment:

One way to do readmultipleproperties would be if a Thing level form has "op":"readmultipleproperties" (so only readmultipleproperties) and somehow describes that the response will contain properties 1, 3 and 5.

which is in case this operation is for a static set of properties. I think this is also a valid option. There should be a way to make this such that the Consumer chooses the properties it wants.