How to avoid duplicates in a directory

[farshidtz]

Registration of TDs using non-idempotent HTTP POST method enables creation of anonymous TDs (TDs without id attribute). The producer can distinguish between the created TDs using the unique-system generated IDs given in the response Location header.

A side-effect of this is that clients will be able to register duplicate TDs accidentally or on purpose.

Need to clarify:

• What are the use cases where a duplicate TD is desired?
• How can we mitigate accidental duplicates (deduplication)?

[farshidtz]

A use case for valid duplicates is for devices which have event interaction affordances to publish/subscribe to an MQTT broker under the same topic or to call an external webhook endpoint whenever data is available.

The TD will be duplicate if it is anonymous (for whatever reason). The system-generated IDs (generated by TDD) may be used to locate the TD or update it. The events produced by these devices may include the ID or have some other identifier.

Example 34 without ID:

{   
    "@context": "https://www.w3.org/2019/wot/td/v1",
    "title": "MyIlluminanceSensor",
    "securityDefinitions": {"nosec_sc": {"scheme": "nosec"}},
    "security": ["nosec_sc"],
    "events": {
        "illuminance": {
            "data":{"type": "integer"},
            "forms": [
                {
                    "href": "mqtt://192.168.1.187:1883/illuminance",
                    "contentType" : "text/plain",
                    "op" : "subscribeevent"
                }
            ]
        }
    } 
}

mqtt://192.168.1.187:1883/illuminance is the the broker URI which can be reused by all luminance sensors.

[farshidtz]

We should note that duplicates can also happen unintentionally when TDs are not anonymous.

In a non-WoT context, we had applications which were generating UUIDs at startup time and using them to register themselves into a directory. One application (service) was crashing and restarting continuously, creating a unique registration every time, until it was fixed. In the meantime, it created thousands of unique looking registrations.

To avoid such scenarios, we introduced mandatory, short TTL for registrations to eventually cleanup accidental registrations and only keep those that are actively maintained (e.g. with a keep alive mechanism). Note: issue #18 relates to this.

[egekorkan]

Regarding the MQTT comment above, in a way, in MQTT case, a TD describes the broker and not really the Thing.

[farshidtz]

I propose closing this issue for the following reasons:

• The accidental duplicate registrations will be automatically removed if the directory enforces TD expiry. See https://w3c.github.io/wot-discovery/#exploration-directory-registration-expiry
• Following the current spec, registered TDs will never be duplicates in enriched form (default) because the identifier (id field) is always unique.

[mmccool]

Discussed July 4, 2022; will close. Resolved (or well, users of the directory are responsible for not using duplicate ids unless they mean to do an update).