meta operations when multiple content types exist

[egekorkan]

Currently, the meta operations such as readallproperties are not clear about what should happen if there are properties with different content types. For example, there could be multiple JSON properties but one PNG. Given that a meta operation needs a contentType as well, what should happen?

For example, a TD with 4 JSON, 2 PNG properties and a Thing-level form with JSON content type, I can see the following possibilities:

• The non-JSON properties are converted into JSON and put together with the others
• Only JSON properties are reported back. This becomes a readmultipleproperties since the Consumer will not get all the properties in the end.
• It is bad practice to have a readallproperties and should be flagged as a linting/validation error.
• Some kind of HTTP Multi Part can be used since that is meant for multi content type payloads. Sadly it is mostly meant for client side but I do not see a technical problem why it cannot be the other way around. See https://www.w3.org/TR/html401/interact/forms.html#h-17.13.4.2 .

I think WebThings has support for images so maybe @benfrancis has an idea already?

Note: Putting 2.0 label straightaway

[benfrancis]

I think WebThings has support for images so maybe @benfrancis has an idea already?

Currently WebThings Gateway will just return null for the value of an image or video property when included in a response to a readallproperties request with application/json in the Accept header.

This because the type member of the ImageProperty and VideoProperty schemas is specified as "null/undefined". Historically the property affordance would then separately provide a Link with rel: 'alternate' and mediaType: 'image/jpg' so the client knows where to find the image file.

To be honest I'm not sure how it behaves since links were replaced by forms, because I don't have a camera device to hand to test with and the virtual things adapter hasn't been updated yet.

Ideally for readproperty operations you could respond with a binary image file, and for readallproperty operations you could base64 encode the image in a JSON response to combine it with other properties. However, I'm not sure how you'd actually describe that in a Thing Description because contentMediaType and contentEncoding are top level members of an InteractionAffordance, which presumably therefore apply to all forms. As far as I can tell there's therefore no way to describe a property which can be exposed as both a raw binary image and a binary image base64 encoded in JSON.

base64 encoding also wouldn't work for all properties, e.g. a video stream. Ultimately if a Consumer requests a JSON response and there is no JSON representation of a property resource, then the Thing can't provide one.

Some kind of HTTP Multi Part can be used since that is meant for multi content type payloads. Sadly it is mostly meant for client side but I do not see a technical problem why it cannot be the other way around. See https://www.w3.org/TR/html401/interact/forms.html#h-17.13.4.2 .

Maybe, but I think you'd need a payload binding specification to define how to map property names onto property values where each property value is essentially a separate file.

The simplest thing for the Thing Description specification to say would probably be that if a Consumer asks for a JSON response to a readallproperties request and no JSON representation of a given property exists, then that property should be given the value undefined in the JSON response.

[egekorkan]

Thanks a lot for the insight! I think that your approach would be valid for a profile (or default behavior) but a generic solution seems tricky as we would need to describe one of the behaviours:

1. non JSON is converted to JSON
2. non JSON is omitted
3. non JSON has null value

First of all, the simple case should be easy to describe and should probably be the default. Something (ugly) like differentContentTypeValues:[converted,omitted,nulled]

Small note:

As far as I can tell there's therefore no way to describe a property which can be exposed as both a raw binary image and a binary image base64 encoded in JSON.

You can have two forms in this case but we would need a way to indicate that the data schema applies only to one (base64 encoding).