-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Propose an outline for a new profile document #152
Comments
Please see #73 (comment) for the specification structure I proposed in March to reflect the agreed reduced scope (focused on interoperability), with a section by section explanation. Below is a summary, including an updated list of protocol binding operations based on the latest specification.
The differences between this and the current draft are that:
You can see a rendered approximation of this structure at https://benfrancis.github.io/wot-profile/, which is basically the current draft with the following PRs merged:
(It's missing the Links section, which is still being discussed in #114). I anticipate that @mlagally will object to removing the whole WoT Core Data Model section, so as a compromise I proposed in #125 (comment) that we could keep a very limited number of data model constraints such as units (#29) and time formats (#94) which would clearly aid interoperability. For anything beyond that I propose (#125 (comment)) re-reviewing sub-section by sub-section to establish consensus. I've also proposed (#115) that the content of the Abstract and Introduction sections should be re-visited to ensure they reflect the agreed reduced scope. |
Thanks @benfrancis for this constructive proposal and for the summary of your PR proposals. Can you please clarify the proposed name/scope of the document (currently it is called "Foo") in the context of the conversation we had in the last architecture call. The slides are available here for reference: https://github.com/w3c/wot-profile/blob/main/contributions/2021-12-02-WoT-Profiles-with-discussion.pdf Depending on the scope (home gateway profile, industrial gateway profile, both?) the document structure needs to be adapted, I'll provide a proposal next week. |
@mlagally wrote:
My proposal for the name of the HTTP+JSON profile we are currently defining is "Common Profile". The scope of the Common Profile would include:
Additional notes given that you're wanting to frame the discussion around the presentation you gave:
|
I think it is important to clarify the following purpose.
In this purpose, I think we should discuss what is Core Profile again. |
Regarding the slides of @mlagally I have a question: In the sensor scenario, where do you use TDs? From my point of view, there can be 3 TDs: Sensor, Gateway, Cloud. In which case do I have to be profile compliant? Does the TD change while the TM stays the same? A feedback: This is a very simple scenario where the devices send data to the cloud and nothing more. Sadly most cloud platforms concentrate on this use case and do not consider how bi-directional (or simply consumer initiated) interactions happen. I think that we should support this use case since it is so common but I would object that this kind of a use case drives a profile that claims to be core, common, basic etc. Sorry that I could not be there in the meeting but I will be this week hopefully. |
TDs describe the sensor and the gateway. Potentially they will also describe the consumer, but this needs more discussion. I think we are getting closer to the actual point, which I hoped to illustrate with the slides: The sensor is just a very simple example of a minimum "HTTP+JSON" thing. Just a message now and then to a server, nothing fancy, perhaps only 5-10 messages per day. A consumer can subscribe to events, gets them sent whenever they occur, unsubscribes from the event when it is no longer interested. If the profile does not support this rather simple scenario, it is not suitable to be called a "common HTTP+JSON" profile.
Are you saying the we cannot even build a trivial sensor with the core profile? Btw.: I demonstrated a cloud platform that triggers actions 2 years ago and know how we could do that in WoT, but that would be a topic for the 1.1 version of the profile. |
We are currently fighting to build a "one size fits all" thing and struggle to put things together because people have different deployment models. I believe there's a constructive way out of the dilemma: We define protocol binding "blocks" for each interaction afordance, which then can be used to define useful profiles. @benfrancis has done quite a lot of work here, which can be extended with the missing pieces. readproperty - done invokeaction - done subscribeevent - pull model - done, push model t.b.d For observable properties and events we should define (at least) two event mechanisms: a "consumer pull" mechanism using JSSE and a "thing push" mechanism using WebHooks. Based on these protocol binding building blocks we can define different profiles: home gateway: properties, actions and events. optional observer / event mechanism: SSE, others? Ultimately I think the name "core" is a big source for the current impasse, since it suggests that there's a SINGLE common profile. Both can be in the scope of the current publication. Alternatively we define a common baseline that combines the home gateway, industrial gateway and sensor. |
In today's editors call, consensus of participants was to first focus and align on common goals before discussing specific document structure proposals. |
@mlagally wrote:
Yes, as I understand it what we are defining in WoT Profile 1.0 is a single common profile to enable out-of-the-box interoperability between greenfield devices and Consumers. Not to cover all possible use cases or existing devices, but to reflect the most common approach to implementing WoT using HTTP+JSON, with the Web Thing as an HTTP server and the Consumer as an HTTP client. It can not possibly fit all deployment models (e.g. Things as HTTP clients) or requirements (e.g. very resource constrained hardware) whilst also guaranteeing that all conforming Consumers can communicate with all conforming Web Things.
I'm happy to write a protocol binding for observing events using Server-Sent Events (to be consistent with the events protocol binding). If that's something people would like to see then please comment in #95, I'm just waiting on feedback.
The current protocol binding only describes a push model (using Server-Sent Events), not a pull model (like polling). Note that the first line of the Server-Sent Events specification says "To enable servers to push data to web pages over HTTP".
In order for a Web Thing and Consumer to support:
both the Web Thing and Consumer must be both an HTTP client and an HTTP server. Do you acknowledge that this would make it very challenging to implement a conforming Consumer as a mobile or desktop application, and impossible to implement in client-side JavaScript in a web application?
There is nothing smart home specific about the current profile, in fact I am currently implementing it for both smart home and enterprise applications spanning device, gateway and cloud deployments. There is no fundamental technical reason why smart home applications should use Server-Sent Events and industrial applications should use WebHooks. If you want to implement a Web Thing which is both an HTTP server and an HTTP client so that it can push events to a cloud server using WebHooks then you can already do that, but I think that's an unreasonable requirement to impose on all WoT Consumers conforming to a profile, even if that profile was specifically for industrial applications. Instead, for that particular use case why not use protocol binding templates for the sensors and just use the Core Profile at the gateway/cloud API level? Or even use a non-WoT implementation for sensors (e.g. using MQTT) and expose them to WoT via a gateway or cloud service? Maybe a future profile could cover the sensor use case and event model you describe, but I would argue that such a profile should be horizontal rather than vertical (i.e. not only for "industrial sensors"). I understand you think that if the Core Profile can't cover your sensor use case then it would harm adoption, but I think having different profiles for different application domains would be far more harmful. It would mean that a Consumer implementing the industrial profile would not be able to communicate with a Web Thing implementing the smart home profile and vice-versa. This would work against the overarching Web of Things goal of interoperability "across application domains", or as Matthias Kovatsch put it at the WoT F2F in Munich in June 2019 where Profiles were first discussed, "that would bring the silos back". For the record, I am concerned that if the WoT Profile goes in the direction you are proposing of making WebHooks support mandatory or having separate smart home and industrial profiles, it would make the specification unimplementable in the WebThings platform because:
If the WoT Profile specification is unimplementable then it may force me to define a separate common profile (e.g. under the Web Thing Protocol Community Group) which can be implemented. That would be far from ideal because having competing profiles would also harm interoperability. Finally, I see that on the mailing list you have directed members to this issue in order to discuss the scope and goals of the WoT Profile specification. To people arriving here from that email, please note that the scope & goals conversation has already been going on in #73 since March and I suggest it would be better to comment there, rather than cover all the same topics again here. There are now two competing proposed resolutions in that issue for people to comment on. Once we have agreed on scope and goals, we can re-visit the document outline most appropriate for that scope. |
Given there are three basic entities defined by the WoT Architecture, Consumer, Intermediary and Thing, I've started to think it would make sense to have three profiles to describe our expectation for those three entities, i.e., Consumer Profile, Intermediary Profile and Thing Profile. |
I think that would not be a good idea. If a Consumer is not conform to the Thing profile, there is simply no interoperability. |
I completely agree with this and don't believe that we have to define silos or different silos in the Profile 1.0. spec, we just have to define a profile that has optional parts, such as 2 different event mechanisms, which seems to be the biggest source of misunderstandings at this point. I think the comment from @ashimura is most helpful to clarify and resolve the problems and when we think about deployment scenarios, the picture will become clearer. We have 3 types of communicating entities in the abstract architecture: Consumers, Intermediaries and Things. The key concept is a servient, which can be a client, a server, or both. The SSE event mechanism is a typical client/server event mechanism, which is fine for a browser, however not scalable for deployment scenarios, where servers and servients interact. If a Thing contains 10 event affordances, in the worst case the client needs 10 open TLS connections. I called it "pull" model to indicate that the client initiates the connection. To avoid further confusion we should call it "client pull" to distinguish it better from the "server push" model. For "Server (thing) to Servient (cloud service)" communication, which is the typical device to cloud deployment scenario, a server push is the natural choice. This also is the case for servient to servient communication, which is the cloud to cloud model. The profile should not enforce a thing to implement ANY event mechanism and leave it to the manufacturer, which mechanism to provide. This depends on the deployment. There may be devices, where both "thing push" and "consumer pull" mechanisms are offered, but that is up to the manufacturer. With these agreements the profile 1.0 enables ALL of these deployment scenarios. |
@ashimura wrote:
My understanding is that a profile is a contract between a Consumer and a Thing (including intermediaries which expose Things), so I don't think a Consumer profile even makes sense. If an intermediary exposes multiple Things (i.e. is a directory or gateway) then I can see that it may have additional requirements since its API surface will be different and that may provide a justification for a separate profile. The Directory Service API in the WoT Discovery specification essentially already provides that, it just doesn't use the profiling mechanism. We could make TDDs and the Directory Service API part of the Core Profile as discussed in #121 (comment), but that would be a big change at this stage. Alternatively a dedicated directory/gateway profile could even go beyond what the Directory Service API provides to define additional operations such as observing properties or subscribing to events from multiple Things over a single connection (which I think would be very useful), but that may require new features in the Thing Description specification and it's probably too late for that in this publication iteration. It's disconcerting to me that the feature complete deadline for the WoT Profile specification is in less than two months and we are still having basic discussions about what a profile is. If we have any hope of meeting that deadline it seems to me that our only realistic option is to:
Anything more than that and it seems inevitable we will need to extend the deadline, since we would really be starting from scratch. Once 1.0 is out the door, we can continue the conversation about whether gateways and cloud services, as well as other deployment scenarios like Things acting as HTTP clients, need their own profiles. We could even work on those profiles within the current charter if we have time. |
@mlagally wrote:
In order to guarantee interoperability a profile can only have optional requirements for Things, not Consumers. Again, if WebHooks is a mandatory event mechanism for Consumers then it makes it means that all Consumers have to be both an HTTP client and an HTTP server. Please answer the question I asked you above:
This is simply not true. See
This is still incorrect.
Actually it requires a new TCP connection for every event, as compared with SSE where all events can share the same connection, including for all event affordances.
I do understand the rationale for wanting Producer initiated connections for events (and property changes) because I have the same use case as you in implementing a cloud service which monitors thousands of gateways and Things. I would like to work with you on solving that problem in the future as an alternative deployment model. I'm just saying that that approach is not compatible with the rest of the currently defined profile for all the reasons explained above, and would be unimplementable for many Consumers.
I agree that a profile can have optional requirements for Things, Things don't have to implement SSE if they don't use events and they should even be able to offer alternative event mechanisms outside of the profile or from other profiles if they want to (#60). However, any event mechanisms which are defined as optional for Things within a profile must be mandatory for Consumers within that same profile, which is what causes the problem explained above.
I really hope you can see from the above that with the best will in the world this is simply not possible. |
@mlagally wrote:
What if we called the current profile something like the "HTTP+JSON Thing Server" profile? That would allow for future profiles for:
|
Based on the proposal by @benfrancis I propose the following document structure for the Profile 1.0 document: HTTP+JSON ProfileIntroduction
Profiling methodologyConformanceTerminology
Deployment scenarios
Common requirements for all HTTP+JSON deployment scenarios
HTTP+JSON Protocol binding
ExamplesValidation (JSON-Schema)Open issues |
@mlagally wrote:
Thank you for your proposal. I'm afraid this structure still does not solve the problems that I have highlighted. Some of the problems I see with this outline are:
Can you please answer my question about WebHooks:
If the specification followed the structure above it would be unimplementable in WebThings for all the reasons I have already explained. I still think we need to agree on goals and scope (#73) before we can agree on a document outline. |
I cannot comment about Javascript, but can confirm that it easy to implement a webhook observer in a mobile or desktop application with a few hundred lines of Java code. |
I would like to thank benfrancis and mlagally for your proposals for document structures. Looking back at the discussions so far, whether the content of the Profile document should be narrowed down to the interoperability or whether the data model should be included depends on readers. Why don't we think concretely about the readers we assume? @mlagally's proposal for the structure seems to cover these contents, so couldn't we redefine the readers of the Profile document and the redefine the scope? I think this is an important point for many readers to use WoT from now on. |
Some overall comments:
|
|
|
|
@mmccool Thanks for sharing theses notes, I'm very much in agreement with you. In particular I agree that although it won't fulfil all possible use cases, a single "HTTP+JSON Profile" where Things are servers and Consumers are clients is a pragmatic way forward for WoT Profile 1.0. I have follow-up points below on scope, and feedback on human readability and length limits based on implementation experience. Scope
I note that none of the other WoT specifications have a "Scope" section either, so I'm not sure whether that text belongs in the specification itself or a separate requirements document? It's regrettable that we don't appear to have an up to date use cases and requirements document specific to the WoT Profile specification which makes the goals and scope of the specification clear (please correct me if I'm wrong), that probably could have prevented the current impasse. I recall that there were use cases and requirements documented inside the repository at some point which had contributions from various members, but they seem to have been deleted before a consensus was reached. Perhaps they were moved somewhere else? I can't find them. I'm not sure to what extent we want to go back to defining requirements at this stage, but we do at least need a common written understanding of the scope and goals of WoT Profile version 1.0, as being discussed in #73. My expectations following today's WoT Architecture meeting are that we:
Human Readability
I think this is a great point. Making titles and descriptions mandatory doesn't necessarily guarantee human readability for all users, only those for whom strings are provided in their language. This is another reason that making titles and descriptions mandatory doesn't necessarily help in the way you might expect. Since we can't mandate providing strings in every language, they can only really be recommended. I also agree it's not clear whether those strings are meant to be for developers or for end users. In many cases it may be both, but in the latter case localisation is arguably even more important. The user interface of WebThings Gateway is translated into 34 languages, but we don't provide metadata for multiple languages inside each Thing Description (my preferred solution for that would be to use HTTP content negotiation to allow users to request a Thing Description in their language, so that Thing Descriptions don't get extremely large).
This is actually what we currently do in WebThings Gateway. Each of the device capabilities, properties, actions and events defined in the WebThings capability schema repository have titles defined for the gateway's UI , with translations of those strings for every supported language. Things can then use those capabilities using semantic annotations in their Thing Descriptions.
We also do this in WebThings Gateway. When a user adds a device, we populate a default title for the device but the title is editable so that it can be customised by users. They can also edit the title at any time.
That's correct, we generate a generic user interface for Things based on the metadata in their Thing Descriptions. We use various metadata fields including If a Thing Description also denotes particular capability schemas using The point of me illustrating all of this is that it's already possible today to generate both generic and more specialised user interfaces from Thing Descriptions. Making particular Thing Description members mandatory doesn't necessarily help in the way you might expect, and in some cases could even cause barriers to interoperability. As with the Web of Pages where rich metadata like OpenGraph is incentivised by richer representations in UIs on Facebook, Twitter, Google, Browsers etc., rich metadata in the Web of Things can be incentivised by richer representations in implementations. Length Restrictions Another topic that came up on the call today was restrictions on the lengths of strings. @mlagally asked what the maximum length of strings is for our existing implementations. I don't have a precise answer to this question for WebThings since it's likely different for the gateway and the 16 library implementations in different programming languages (and impacted by various other implementation-specific or deployment-specific factors), but:
The thing is that at least for our implementation it's not as simple as a single string being stored in a single table column, since we store whole Thing Descriptions so the whole size of the Thing Description is probably more important than the size of individual strings. We also store additional implementation-specific metadata alongside Thing Descriptions, so the size of the source Thing Description is not necessarily the only factor. Since this is so implementation-specific and often configurable, I don't really see how a reasonable limit could be chosen. Implementations can always truncate long strings within their own constraints if they're ridiculously large, but in general this seems like an issue for JSON parser implementations (e.g. preventing buffer overflows) than for WoT implementations which use them. |
Here the interoperability problem becomes very apparent - a profile should avoid these implementation - or deployment specific dependencies and differences.
If Implementations may arbitrarily truncate to their liking, you could have an intermediary with a string length of 32 chars, which then would truncate everything it passes through. This may cause name overlaps and other nasty effects. In my understanding we should prevent these things to happen |
This discussion gets again very technical and I suggest we split the length / limit conversation into a separate issue and suggest to keep this issue focused on thhe proposed outline. |
@mlagally wrote:
Then I guess we need to set the length limit to zero characters, since a JSON parser could be configured to have any number as a maximum...
OK, I've said everything I came here to say on that topic. Happy to focus on the document outline from here as I had hoped to keep the scope discussion in #73. |
@mmccool, I think the people who refer to Profile document are the developers (Thing, Consumer and Intermediary/Gateway developers) and operators. It's not intended for end users who use IoT systems. I don't claim to force the registration of human-readable items. My proposal is these items should be specified in Profile so that they aren't be defined individually as mentioned above. They can be optional, not mandatory. People who think they are necessary can use them, and others can ignore them. |
See also: #214. |
New outline has been adopted, the issue can be closed. |
This issue servers as a single reference point to discuss the scope of a revised profile spec.
Please keep the contributions brief and concise, when discussion points arise, please create a separate issue and continue the discussion there.
The text was updated successfully, but these errors were encountered: