Skip to content

Rewrite multiple sources/values documentation #342

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Apr 30, 2017
Merged

Rewrite multiple sources/values documentation #342

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
f2201ba
test: add test case for multiple sources/values
tkurki Mar 4, 2017
91cd2e9
Made a start on multiple values for a single key
rob42 Mar 27, 2017
8f6370c
Adjusted multiple devices
rob42 Mar 27, 2017
48351a1
Correct update format, and add note on grouping
rob42 Apr 15, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
177 changes: 113 additions & 64 deletions gitbook-docs/data_model_multiple_values.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,18 +1,18 @@
# Multiple Values for a Key

There are two use cases for multiple data:

* Multiple instances of a common device - eg two engines or multiple batteries
* Multiple devices providing duplicate data - multiple values for the same Signal K key from different sensors, eg COG from both compass and gps or multiple depth sounders

## Multiple instances of a common device

Some parts of the Signal K schema are **device oriented**.

For example in electrical domain you have the concept of multiple batteries.
Each battery has multiple, common quantities like voltage, current and temperature.
Here we have chosen to organise the Signal K data model hierarchy by instance: multiple batteries are represented as for example `electrical.batteries.starter` and `electrical.batteries.house`. Then underneath that prefix you have the different properties and quantities.
For example in electrical domain you have the concept of multiple batteries. Each battery has multiple, common quantities like voltage, current and temperature. Here we have chosen to organise the Signal K data model hierarchy by instance: multiple batteries are represented as for example `electrical.batteries.starter` and `electrical.batteries.house`. Then underneath that prefix you have the different properties and quantities.

This organisation allows a user interface to organise the individual readings in meaningful groups and you can query all the values related to that piece of equipment via REST API.

This organisation allows a user interface to organise the individual readings in meaningul groups and you can query all the values related to that piece of equipment via REST API.
It maintains the primary requirement that a given data value have a fixed and unique uri, but gives flexibility in the structure and complexities of data.

The same device centric organisation is used in `propulsion` subschema, where the most common use case is having dual engine setup with `propulsion.port` and `propulsion.starboard`.
Expand All @@ -26,99 +26,148 @@ It is quite possible for a key value to come from more than one device. eg posit

All the incoming values may well be valid in their own context, and it is feasible that all of them may be wanted, for instance, displaying depth under each hull on a catamaran.

Hence discarding or averaging is not a solution, and since signalk is unable to derive the best way to handle multiple values it must always fall to a default action, with human over-ride when needed.


***
The solution presented below has flaws. See https://github.com/SignalK/specification/issues/48 for discussion.
***
Hence discarding or averaging values is not a solution, we must provide a way to store multiple values for a single measurement.


In signal K we can leverage the above method and simply store all the devices in the tree under the main item, and have the main items `source` reference the options. Lets consider this for `courseOverGroundTrue`
Lets consider this for `courseOverGroundTrue`

If its the first value for the key, it becomes the default value and looks like this:

```json
{
"vessels": {
"self": {
"urn:mrn:signalk:uuid:c0d79334-4e25-4245-8892-54e8ccc8021d": {
"uuid": "urn:mrn:signalk:uuid:c0d79334-4e25-4245-8892-54e8ccc8021d",
"navigation": {
"courseOverGroundTrue": {
"value": 102.29,
"source": "vessels.self.sources.n2k.actisense-115-129026"
"courseOverGroundMagnetic": {
"value": 3.615624078431453,
"$source": "nmea2.II",
"timestamp": "2017-03-04T14:58:48.000Z",
"sentence": "VTG"
}
},
"sources": {
"n2k": {
"actisense-115-129026": {
"value": 102.29,
"bus": "/dev/actisense",
"timestamp": "2014-08-15-16: 00: 01.083",
"src": "115",
"pgn": "129026"
}
}
}
},
"self": "urn:mrn:signalk:uuid:c0d79334-4e25-4245-8892-54e8ccc8021d",
"version": "0.1.0",
"sources": {
"nmea2": {
"label": "nmea2",
"type": "NMEA0183",
"II": {
"talker": "II",
"sentences": {
"VTG": "2017-03-04T14:58:48.000Z"
}
}
}
}
}
```
It has come from device `vessels.self.sources.n2k.actisense-115-129026`, where further details can be found.
It has come from device `sources.nmea2.II`, where further details can be found.

If another value with different source arrives, we add the source with a unique name, so both values are in there - if its our preferred source (from persistent config) we auto-switch to it, otherwise we just record it. It look like this:
If another value with different source arrives, we add the `values` attribute with and values are in there - if its our preferred source (from persistent config) we auto-switch to it, otherwise we just record it. It look like this:

```json
{
"vessels": {
"self": {
"urn:mrn:signalk:uuid:c0d79334-4e25-4245-8892-54e8ccc8021d": {
"uuid": "urn:mrn:signalk:uuid:c0d79334-4e25-4245-8892-54e8ccc8021d",
"navigation": {
"courseOverGroundTrue": {
"timestamp": "2014-08-15-16: 00: 01.083",
"value": 102.29,
"source": "vessels.self.sources.n2k.actisense-115-129026"
}
},
"sources": {
"n2k": {
"actisense-115-129026": {
"value": 102.29,
"bus": "/dev/actisense",
"timestamp": "2014-08-15-16: 00: 01.083",
"src": "115",
"pgn": "129026"
},
"actisense-201-130577": {
"value": 102.29,
"bus": "/dev/actisense",
"timestamp": "2014-08-15-16: 00: 00.085",
"src": "201",
"pgn": "130577"
"courseOverGroundMagnetic": {
"value": 3.615624078431453,
"$source": "nmea2.II",
"timestamp": "2017-03-04T14:58:48.000Z",
"sentence": "VTG",
"values": {
"nmea1.II": {
"value": 3.6376152270065814,
"sentence": "VTG",
"timestamp": "2017-03-04T14:58:47.000Z"
},
"nmea2.II": {
"value": 3.615624078431453,
"sentence": "VTG",
"timestamp": "2017-03-04T14:58:48.000Z"
}
}
}
}
}
},
"self": "urn:mrn:signalk:uuid:c0d79334-4e25-4245-8892-54e8ccc8021d",
"version": "0.1.0",
"sources": {
"nmea1": {
"label": "nmea1",
"type": "NMEA0183",
"II": {
"talker": "II",
"sentences": {
"VTG": "2017-03-04T14:58:47.000Z"
}
}
},
"nmea2": {
"label": "nmea2",
"type": "NMEA0183",
"II": {
"talker": "II",
"sentences": {
"VTG": "2017-03-04T14:58:48.000Z"
}
}
}
}
}
```
###Update messages

### Rules
When a client subscribes to `navigation.courseOverGroundMagnetic`, they recieve _all_ the values held. The update message does not include the `values` path, the case above looks like:

Now simple rules can apply to obtain the default, or any specific value:

* The implementation must ensure that the `key.value` holds an appropriate value. This will be easy if there is only one, and will probably be user configured if more.
* If the `source` value is `string` then it is a reference key to the source object, and can be a relative or absolute signalk key.
* The `source` (as a reference string) also provides a mechanism to handle deprecated keys.
* If the `source` value is a `json object` then it holds meta data on the source of the value.
* Alternate sources must be discovered manually, or by implementation specific meta-data.
```
{
"context": "vessels.urn:mrn:signalk:uuid:c0d79334-4e25-4245-8892-54e8ccc8021d",
"updates": [
{
"source": {
"label": "nmea2",
"type": "NMEA0183",
"talker": "II"
}
},
"timestamp": "2017-03-04T14:58:48.000Z",
"values": [
{
"path": "navigation.courseOverGroundMagnetic",
"value": 3.615624078431453
}
]
},
{
"source": {
"label": "nmea1",
"type": "NMEA0183",
"talker": "II"
}
},
"timestamp": "2017-03-04T14:58:47.000Z",
"values": [
{
"path": "navigation.courseOverGroundMagnetic",
"value": 3.6376152270065814
}
]
}
]
}

To see all the entries, use the REST api or subscribe to the parent object. A given device may choose to subscribe to a specific entry in the object, allowing multiple displays of the key, or users of the various values. The 'list' verb used in a query message can provide available keys.
```
Individual updates can be distinguished by their source.

###Unique names
If a client wants only the values of a single source it should subscribe to a path that includes the full path under `values` including the source reference key of the source. The source reference should be enclosed in square brackets: `navigation.speedThroughWater.values[n2kFromFile.43]`. The client can retrieve the relevant data via REST API.

The identifier for each device should be unique within the server, and possibly be constructed as follows:
**Note:** The exact format of the update message is affected by the subscription policy. A policy of `instant` will result in changes being sent immediately, so typically one item in `values` per update. A policy of `fixed` will result in periodic updates which may contain many items in `values`.

n2k: producerid-sourceid-pgn (producer id from server configuration, others from n2k data) - NOTE: will change, currently under discussion.
nmea0183: producerid-talkerid-sentence (like n2k)
signalk: any valid string matching regex [a-zA-Z0-9-]. eg alphabet, hyphens, and 0 to 9
The update allows grouping `values` by `source`.

(The nmea0183 talker id is not in the schema as I write this, it will be added shortly)
51 changes: 51 additions & 0 deletions test/data/multiple-values.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
{
"vessels": {
"urn:mrn:signalk:uuid:c0d79334-4e25-4245-8892-54e8ccc8021d": {
"uuid": "urn:mrn:signalk:uuid:c0d79334-4e25-4245-8892-54e8ccc8021d",
"navigation": {
"courseOverGroundMagnetic": {
"value": 3.615624078431453,
"$source": "nmea2.II",
"timestamp": "2017-03-04T14:58:48.000Z",
"sentence": "VTG",
"values": {
"nmea1.II": {
"value": 3.6376152270065814,
"sentence": "VTG",
"timestamp": "2017-03-04T14:58:47.000Z"
},
"nmea2.II": {
"value": 3.615624078431453,
"timestamp": "2017-03-04T14:58:48.000Z",
"sentence": "VTG"
}
}
}
}
}
},
"self": "urn:mrn:signalk:uuid:c0d79334-4e25-4245-8892-54e8ccc8021d",
"version": "0.1.0",
"sources": {
"nmea1": {
"label": "nmea1",
"type": "NMEA0183",
"II": {
"talker": "II",
"sentences": {
"VTG": "2017-03-04T14:58:47.000Z"
}
}
},
"nmea2": {
"label": "nmea2",
"type": "NMEA0183",
"II": {
"talker": "II",
"sentences": {
"VTG": "2017-03-04T14:58:48.000Z"
}
}
}
}
}
6 changes: 6 additions & 0 deletions test/sources.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -141,3 +141,9 @@ describe('Bad sources in delta', function() {
})
});
});

describe('Multiple sources for the same path:', function() {
it("value + values are valid", function() {
require('./data/multiple-values.json').should.be.validSignalK
});
});