Profile needs a simple self descriptive mechanism without huge implementation demands

[mlagally]

Does a simple thing have to implement a directory service / implement a discovery API?

[benfrancis]

My proposal in w3c/wot-profile#112 for which I have a PR at w3c/wot-profile#121 is that all Things conforming to the Core (HTTP+JSON) Profile must as a minimum have their Thing Description retrievable from an HTTP URL. This would allow for direct discovery (e.g. copy & pasting a URL or using a QR code or Bluetooth beacon).

There was also a suggestion that the Core Profile should make the Well-Known URI introduction mechanism mandatory, but there are issues with that, discussed in w3c/wot-profile#121.

There was also a suggestion that the Core Profile should make support for the Directory Service API mandatory for conformant Consumers, but that would significantly increase implementation complexity.

[mmccool]

• "Peer to Peer" discovery is currently defined here: https://w3c.github.io/wot-discovery/#exploration-self
• This just states that a GET retrieves a TD from a URL directly. It needs to be paired with an "Introduction" mechanism to determine a URL
• The simplest Intro mechanism to pair with this is the "well-known" URI
• So, I would suggest that Profiles require both the well-known URI introduction mechanism and the self-description mechanism described above.
• HOWEVER this does not currently handle the use case of multiple Things on one server. This could happen if there are multiple services running on a hub, for instance. These cases should use a directory, but since queries are not now required for directories, this is not a "huge" implementation burden. The question is if we need something simpler for small devices with a small number of TDs. One use case that comes to mind is a dual-outlet plug with separate Things for each outlet. Doing this makes it easier to set up automations that expect to talk to a single on-off device. I was thinking that we should just extend self-description (section 6.1) to allow it to return a JSON Array of TDs. It would be easiest if it always returned an array, in my opinion... will discuss today. This would only be recommended for a small number of TDs, up to 3 or 4. Beyond that a directory should be used, as it supports pagination, etc.
• A serious problem is security. Right now there is no mandatory security on directories or self-description, only a statement that says that HTTPS SHOULD be used. Unfortunately on LANs this is not possible to the browser requirement for publicly exposed URLs, and so it is usually the case that plain HTTP is used on LANs. Unfortunately this has the net result that devices satisfying an HTTP-based Profile that requires use of .well-known and self-description would essentially require devices to broadcast their TDs without security on LANs. NOT what we wanted. The "right" way to solve this is to have an on-boarding process with a hub that provisions keys and requires some confirmation (entering a PIN, pushing a button) to authenticate the device and establish a cert that TLS can use locally. Unfortunately establishing such a process has not been done for WoT and it is currently out of scope.
• Hubs, however, can generally expose a secure endpoint via a remote access URL/cloud proxy etc. Devices could access this URL and register themselves with a directory service running on the hub over HTTPS. The hub could then provision a cert based on the ID provided in the TD (note: there are other issues with devices providing their own IDs, including the problem of collisions, so we probably also want to add a requirement to Profiles that IDs are cryptographically unique and expressed using uuids). This would basically encode the onboarding process into Profiles, and after this it should be possible to support at least M2M communication over HTTPS on a LAN (for instance, hub to device, device to hub, device to device; browsers may still reject the "self-signed certs", however).

Let me discuss this with the Security and Discovery TF today and see if we can come up with a set of recommendations. At this point I'm thinking that if we want secure self-description, we'll need some kind of on-boarding process described as part of the Profile. Unfortunately (it should be a separate and more general specification).

[mmccool]

Recommendations:

• Disallow self-description under plain HTTP. This basically converts the SHOULD in the Discovery spec for self-description to a MUST.
• In practice, it means an on-boarding process needs to be used to set up keys for Things... but this is not defined in the WoT spec itself. (NOTE: It's not ESSENTIAL to define an on-boarding process in Profiles (I'm not convinced we have the time) but let's take care of the basics before even considering whether we should do so...)
• On a LAN, recommend that pre-shared keys be used in place of certs. See https://datatracker.ietf.org/doc/html/rfc4279 for cipher suites that can be used with TLS 1.2. We will study whether a particular one of these should be supported. In particular, mutual authentication would be good for M2M to browsers may reject it. (To do: test what browsers and https libraries e.g. in node.js works with this configuration...).

In both cases the self-description process itself would be as given above: .well-known + self-description (GET a TD from the given URL). Will look at extending this to multiple TDs in parallel.

Some of the above, e.g. using PSK on LANs, should also go into the Security Considerations section of Discovery.

[benfrancis]

@mmccool wrote:

The simplest Intro mechanism to pair with this is the "well-known" URI

Surely the simplest introduction mechanism is actually Direct, which could literally just be copying & pasting the URL

I would suggest that Profiles require both the well-known URI introduction mechanism and the self-description mechanism described above.

• HOWEVER this does not currently handle the use case of multiple Things on one server.

It does handle the use case of multiple Things on one server in that a well-known URI can return a Directory Description, but this also means that making the well-known URI introduction mechanism mandatory for Core Profile has the side effect of making support for the Directory Service API mandatory for all conformant Consumers. Please see w3c/wot-profile#121 (comment) for a further explanation.

I was thinking that we should just extend self-description (section 6.1) to allow it to return a JSON Array of TDs.

Isn't this exactly what the listing feature of the Directory Service API does? Why create a separate mechanism which does the same thing?

A serious problem is security. Right now there is no mandatory security on directories or self-description, only a statement that says that HTTPS SHOULD be used. Unfortunately on LANs this is not possible to the CA requirement for publicly exposed URLs, and so it is usually the case that plain HTTP is used on LANs. Unfortunately this has the net result that devices satisfying an HTTP-based Profile that requires use of .well-known and self-description would essentially require devices to broadcast their TDs without security on LANs.

I suggest the more salient security problem with discovery is how a Consumer knows how to authenticate to access a protected Thing Description. Please see #135 for a proposed solution to that problem, which hasn't yet been added to the WoT Discovery specification. How to provide transport encryption for Things on a local network is certainly a problem, but it seems largely orthogonal to discovery.

• Hubs, however, can generally expose a secure endpoint via a remote access URL/cloud proxy etc. Devices could access this URL and register themselves with a directory service running on the hub over HTTPS. The hub could then provision a cert based on the ID provided in the TD (note: there are other issues with devices providing their own IDs, including the problem of collisions, so we probably also want to add a requirement to Profiles that IDs are cryptographically unique and expressed using uuids). This would basically encode the onboarding process into Profiles, and after this it should be possible to support at least M2M communication over HTTPS on a LAN (for instance, hub to device, device to hub, device to device; browsers may still reject the "self-signed certs", however).

This sounds like a very complicated mechanism to make a mandatory part of the Core Profile. We explored an approach like this for WebThings Gateway and ultimately it didn't really help because of the self-signed certificates issue. If the hub acts as the router for the local network and controls the DNS configuration of all clients using DHCP then it's technically possible to use a certificate for a public domain on a local network, but in practice that doesn't really work either because web browsers often have their own DNS configuration.

Recommendations:

• Disallow self-description under plain HTTP. This basically converts the SHOULD in the Discovery spec for self-description to a MUST.

Please don't do this. We have discussed many times why making TLS mandatory doesn't help, and just creates more problems.

In practice, it means an on-boarding process needs to be used to set up keys for Things... but this is not defined in the WoT spec itself. (NOTE: It's not ESSENTIAL to define an on-boarding process in Profiles (I'm not convinced we have the time) but let's take care of the basics before even considering whether we should do so...)

I suggest the only on-boarding process that's missing is security bootstrapping, which can be solved with an already standardised 401 Unauthorized response with a WWW-Authenticate header which tells the client what method it should use to authenticate.

On a LAN, recommend that pre-shared keys be used in place of certs. See https://datatracker.ietf.org/doc/html/rfc4279 for cipher suites that can be used with TLS 1.2. We will study whether a particular one of these should be supported. In particular, mutual authentication would be good for M2M to browsers may reject it. (To do: test what browsers and https libraries e.g. in node.js works with this configuration...).

I am very sceptical this will work in web browsers, but as a minimum it requires further incubation and testing.

---

Let's please try to keep this issue to discussion about requirements for the WoT Discovery specification and discuss discovery requirements for the WoT Profile specification in the existing issue w3c/wot-profile#112.

Regarding requirements for the WoT Discovery specification, it would be very useful if the security bootstrapping issue was solved, by explaining what a Consumer should do if it gets the 401 Unauthorized response already defined in the self-description section. Otherwise there's no way for a Consumer to "discover" Things with Thing Descriptions protected by authentication.

Regarding the discovery requirements of the Core Profile, I have added my updated recommendation here w3c/wot-profile#112 (comment)

[mmccool]

OK, let me think about it some more. In the meantime, we did discuss the problem of multiple-TDs-on-one-device and noted that TD Links could be used to solve this. Basically this means the GET at the .well-known URL would return a single TD, but that TD would include links to other TDs at other URLs. This would not require a normative change to the Discovery specification, but perhaps a clarifying statement in section 6.1 would be helpful. We probably should also mandate that consumers following the profile should support this format for fetching multiple TDs upon self-discovery.

[mmccool]

Looking at the discussion under w3c/wot-profile#121 (comment), it seems we should fix the Discovery spec instead to allow TD Links to point to TDs directly rather than just directories. We could also add the constraint that self-discovery could ONLY return links to Things that are NOT directories (e.g. only letting directories link to other directories, which was the INTENDED function).

[farshidtz]

@mmccool Thing Link isn't intended for just directories. Please see the three defined scenarios under https://w3c.github.io/wot-discovery/#example-td-link-type

Apart from adding a clarifying statement to section 6.1, we also need to modify the assertion to allow defining more than just one link in the Thing Link TD. Currently the assertion is as follows:

A Thing Link MUST define the referenced TD as a Link with describedby link relation type, application/td+json media type and href set to the target URL.

Moreover, we may need to recommend a new link relation type, instead of describedby.

We discussed the idea at:

• Read-Only Directories #208 (comment)
• Multiple Responses from Peer-to-Peer Discovery #249 (comment)

[mmccool]

I think we've dealt with this now by (a) making the required features for Directories more lightweight (b) more clearly defining what a Discoverer needs to implement. If this is resolved now, I'd like to close this issue.

[mmccool]

Let's give this one more week for comments. If there aren't any, I'm going to assume this issue has been resolved and will close it.

[benfrancis]

Am I right in thinking that based on the new requirements set out in section 5, a Discoverer MUST understand the difference between a Thing Description, a Thing Link and a Thing Description Directory [Description] by parsing the @type member of a Thing Description, but it doesn't necessarily need to be able to enumerate the list of things from a directory or resolve a link?

If so, it's possible that for the HTTP Baseline Profile we could simply require that:

1. Conformant Things MUST have a Thing Description which can be resolved from an HTTP URL
2. Conformant Consumers MUST be able to fetch a Thing Description from an HTTP URL

In other words they must support the Direct introduction mechanism and Self-description exploration mechanism using HTTP, but don't necessarily have to implement the Directory Service API or Thing Links.

If that's all OK, then yes I think this is resolved.

If not, then what else could be the minimum requirements for a Discoverer using HTTP? What does a Discoverer need to be able to do with a TDD or Thing Link?