Repository navigation
Mir Design Review
The difficulty for companies to monitor their systems
Describe the purpose of the software being developed.
Enable highly secure and reliable communication between your Internet of Things (IoT) application and the devices it manages. Mir IoT Hub provides a cloud or on-premise hosted solution back end to connect virtually any device. Extend your solution from the cloud or your own server to the edge with per-device authentication, built-in device management, device observability, device data, and scaled provisioning.
Mir IoT Hub, act as your command centre:
- processes telemetry and commands with two ways communication
- automatically generate dashboards to observe data
- uses device twin for configuration management
- over the air update
- lightweight and infinitely scalable
Define the scope of the software, including what it will and will not do.
Explain any terms and acronyms used throughout the document.
| Key | Definitions |
|---|---|
| Mir | Russian orbital space station |
List any external resources referenced in the document.
Provide a high-level overview of the document's contents.
Describe the overall architecture of the system, including major components and their interactions.
Outline the software environment, including the development platform, languages, and tools.
Detail the functional requirements the software must meet.
Describe the non-functional requirements such as performance, security, and usability considerations.
Provide a high-level design overview, including major system modules and their responsibilities.
The core is responsible of the management of devices lifetime. It is also the API interface for interacting with surrealdb.
TODO one app in front of the db or many app access - one app which interface the db - each app interact with the db managing their own stuff - see own its done for surreal same document TODO think on how to restrict the api in terms of what to update.
- one endpoint for each type of modification and control?
- just update endpoint with a token or header indicating rights or source
The Twin module gives life to devices with digital twin. It is a digital representation of a physical or virtual device. It stores device metadata, configuration, and state information, allowing the system to manage and monitor devices remotely.
Twin is the reconciliation logic between desired properties and reported properties.
{
"apiVersion": "mir/v1alpha",
"meta": {
"deviceId": "0xf123"
"name": "tempsens_vent_alpha232"
"disabled": false,
"labels": {
"factory": "A"
"airVent": "alpha232"
},
"annotations": {
"mir/device/description": "Sensing environement data in vents",
"deviceType": "temperatureSensor"
}
},
"properties": {
"desired": {
"interval": "60s",
"heisteresist": 0.05 // TODO look for the spelling of that bloody word
},
"reported": {
"temperature": 24,
"humidity": 49
}
},
"status": {
"online": true,
"lastHearthbeat": "24-23-2024T16:34:32Z"
}
}A Twin Template is a JSON patch that defines the desired state of a device twin. It specifies the desired properties and their values that the device should have. It will then use labels to target a specific set of devices or a list of device ids. These are manually or via script triggered.
{
"apiVersion": "mir/v1alpha",
"meta": {
"templateId": "airvents_temp_sensors",
"event": ".create" // Empty mean manually triggered
},
"targets": { // labels and annotations are ANDs for each of their key set. IDs are ORs. All together they are ORs.
"labels": {
"deviceType": "temperatureSensor"
},
"ids": ["asd2", "oxf213"],
"annotations": {
"complex": "A"
}
},
"properties": {
"desired": {
"temperature": 25,
"humidity": 50
}
}
}A Twin AutoTemplate is a template that is associated with a set of labels for targets or list of devices id and an event from the message bus such as:
- Device created
- Device updated
- Device deleted
- Device connected
- Device disconnected
- ... could be events for every module, cpu_high, memory_low, etc
Together, this gives the ability to automatically apply a template to a device when it matches the labels. This is useful for devices that are created dynamically or for devices that need to be updated based on certain events.
{
"apiVersion": "mir/v1alpha",
"meta": {
"templateId": "airvents_temp_sensors",
"event": "device.create" // Empty mean manually triggered
},
"targets": { // labels and annotations are ANDs for each of their key set. IDs are ORs. All together they are ORs.
"labels": {
"deviceType": "temperatureSensor"
},
"ids": ["asd2", "oxf213"],
"annotations": {
"complex": "A"
}
},
"properties": {
"desired": {
"temperature": 25,
"humidity": 50
}
}
}- To make sure order of received properties is maintained, the code can do addtionnal check using timestamps fields, or the framework could make sure to always put latest
// one json object with all the properties
{
"properties": {
"desired": {
"temperature": 25,
"humidity": 50
},
"reported": {
"temperature": 24,
"humidity": 49
}
}
}
// one json with same representation, but with timestamp instead of value
{
"properties": {
"desired": {
"temperature": "2022-01-01T12:00:00Z",
"humidity": "2022-01-01T12:00:00Z"
},
"reported": {
"temperature": "2022-01-01T12:00:00Z",
"humidity": "2022-01-01T12:00:00Z"
}
}
}A JSON alternative could be the timestamp field embeded next to the fields directly.
{
"properties": {
"desired": {
// one way of doing it.
"temperature": 25,
"__temperature": "2022-01-01T12:00:00Z",{
},
"humidity": "50",
// allow more fields that could be leverage, like version
// would version be useful? TODO think about it
"__humidity": {
"timestamp": "2022-01-01T12:00:00Z",
"version": "2"
}
},
"reported": {
// one way of doing it. all nicely lined up under
// feel it would be annoying the value like that
// and the user experience would be bad
"temperature": {
"value": 24,
"timestamp": "2022-01-01T12:00:00Z"
},
"humidity": {
"value": 49,
"timestamp": "2022-01-01T12:00:00Z"
}
}
}
}It could be possible to generate open api validation on the different json templates. That schema would help present options or validations for any clients such as Web or TUI.
The developer could generate a schema from structs or made manually. You upload it to mir. It is like uploading a CRD. When you create a a new twin template, you can attach a schema to it. Like a CR link to a CRD. In the json of the properties, a new field with the linked schema could be added to point to the schema.
{
"properties": {
"openapischema": "<name>",
"desired": {},
"reported": {}
}
}This could be on the json template or on the device twin as well in a more fix manner. A bit more thinking to be done here.
Using the module SDK, developers can subscribe to the different streams of data or requests. This make it easy to extend the system and build new modules with great ergonomics. For example, if a user want to know when a device comes online or goes offline, he/she can register to the heartbeat stream and make the required calculation to know if the device is now online or offline. If a developer want to know when a new device was created, he can register to the device create request stream and validate the fields and then if ok, a new device was created. The pros of this approach is that developer can extend the system by subscribing to different part of the systems and know what's going on. The cons is that many of those stream needs calculation that different modules perfoms so a developer would have too recreate the algorithm. Therefore, subscribing to streams or requests is not enough. This is where Events come in.
Events are sever side computed trigger based on incoming data, telemetry or request. For example, the Core module computes if devices are on or off using the heartbeat stream. It will then publish a device is on/off on the event stream. This way, a developer can just subscribe to this event stream and know when a device is on or off without having to compute it using the heartbeat stream. Additionally, the core module will have the following set of events:
- device has come online
- device has gone offline
- device was created
- device was deleted
- device was updated
The goal of events is to have an easy way to extend the Mir ecosystem by subscribing to them via the module sdk interface.
The telemetry module goal is to create a way for a device to push telemetry in an ergonomic and performant way. Gathering the telemetry to be sent by a device is, for most cases, a short task. On the other hand, creating the ingestion system, a User Interface to display the data and have an alerting system is another story. Mir Telemetry module goal is to provide all of that out of the box, to give an all battery included user experience.
To enable ergonomics and performance, Protobuff is used to define a schema of the telemetry. With a schema solution, we can read it at runtime to generate the ingestion functions as well as different tools such as a Grafana dashboards and more. Using InfluxDB as the timeseries database, we can have access to their alerting system and other toolings.
The difficulty arises in how do you define the schema and where do you store it. There can be many type of devices with different schema as well as many versions of the same schema. We need to ensure that the schema used to deserialize the data matches the actual data. I foresee two different approaches:
- Handshake on device startup
- Manually upload to store
- Device startup
- Device send schema as byte
- Protoflux receives it and creates an entry in its map of map with deviceid as first key, protomessage as second key and ingestion function as value.
- The deviceid comes from the message subject
- The proto message name is either subject or header
- Upon receiving the schema, Protoflux will also save the schema in Nats KeyValue store, where key is the device id an schema is the value. Upon s
sequenceDiagram
participant dev as Device
participant nats as NatsIO
participant pf as Protoflux
participant inf as InfluxDB
pf->>nats: On startup, read for nats kv *1
nats-->>pf: list of schemas per device
pf->>pf: Generate ingestion functions per device
dev->>nats: On startup, send schema *2
nats-->>+pf: send schema
pf->>pf: Compare with stored schema
alt is equivalent
pf->>pf: do nothing
else is different or non existent *3
pf->>pf: create new ingestion functions based on schema
pf->>nats: write new schema to kv store
nats-->>pf:
end
dev->>nats: send telemtry
nats-->>pf: receives telemetry
pf->>pf: find ingestion function using deviceid and message
alt not found
pf->>nats: look for schema in kv
nats-->>pf: return schema or not found
alt not found
pf->>nats: request schema from device
nats->>dev: request fowarded to device
dev-->>nats: respond with schema in bytes
nats-->>pf: receives schema
pf->>pf: create ingesting function
pf->>nats: write schema to kv store
nats-->>pf:
end
end
pf->>pf: deserialize telemetry
alt error deserialing because of wrong proto descriptor
pf->>nats: request schema from device
nats->>dev: request fowarded to device
dev-->>nats: respond with schema in bytes
nats-->>pf: receives schema
pf->>pf: create ingesting function
pf->>nats: write schema to kv store
nats-->>pf:
end
pf->>inf: write to store *4
*1. How to organize the data?
- key is device id and value is schema in byte
- do we organize by schema name instead? that would agglomerate the number of keys per schema instead of per device, but that would make annoying to maintain a proper naming
- or we could have a key be an hash of the schema. would not work, it need to map a device.
*1a. We could also lazy load instead of loading everything. It could be a feature flag. *2. Instead of send schema on startup, we could start straight sending telemetry *3. Use history feature of Nats KV on deviceid schema's *4. For table name, it would be the schema name so its the dev responsability to maintant that. each table would be one of the following:
- <schema_name>_<message_name> (preference)
- <schema_name>, each column as the message_name as prefix
Device Twin, there could be a field in status with the proto schema name so we know which devices write where in influx
The handshake approach has great flexibility and reduce the hassle of schema version management by letting the system handle it for us. That being said, the device has to send the schema via network which can be a bigger payload and create problems for specific cases. The second approach is too manually upload the schema via the MirCLI. You upload the schema in bytes with a device targets (labels, id, name, namespace, etc), and Protoflux will write the schema to each deviceid in the kv store as well as refreshing its map of ingestion functions.
Brainstorm
- Option 1: device listen to its deviceId as subject
- mean we will have to have a header specifying the message type
- Option 2: device.<deviceId>.<module>.<version>.<function>
- Option 3: device.<deviceid>.<version>.<function>
- Option 4: <deviceid>.<version>.<function>
Tech:
- Option 1: do we allow only proto commands?
- users can define their proto command in the proto schema.
- using the schema, we can send commands in UI via JSON to proto or proto struct directly in code
- for the systems command, like request the schema, we can store the proto:
- in mir device sdk, could be cyclic problem ?
- in mir api folder, could be the best
- proto message name is a header like for telemetry
- Option
message EnvironmentTlm {
option (mir.device.v1.message_type) = MESSAGE_TYPE_DESIRED_PROPERTIES;
option (mir.device.v1.meta) = {
};
int32 datarate = 1;
}Each message of desired properties type are stored on the device twin. They are handled like a command receive.
User updated the desired properties on the device twin, server write it to disk and send it to device. The DeviceSDK receives similar to the same pattern of a commands.
The DeviceSDK has a SendReportedProperties method that takes a proto.Message as arguments and it get written to the device twin by the server.
Could be nice to have them stack.
The messages get written to the twin using the proto message name.
- Could be short name?
- Could be a meta field that overwrite the name
- Could be full name if there a collision
Using
Dive into the detailed design of each module, including class diagrams, data flow diagrams, and other relevant details.
Describe the data architecture, including database schemas, data models, and data storage considerations.
Outline security measures, including authentication, authorization, data encryption, and secure data storage.
Describe the user interface design, including mockups, user flow diagrams, and user interaction descriptions.
Detail any application programming interfaces (APIs) or external services the system will interface with.
Outline the development methodology, version control system, branching strategy, and build process.
Describe the deployment process, including deployment environments, continuous integration/continuous deployment (CI/CD) pipelines, and rollback plans.
Outline the overall testing strategy, including types of testing (unit, integration, system, acceptance) and testing tools.
A few approaches to integration testing:
- In unit test
- have files called integration test that will test the integration of the module with the database, the message bus, etc.
- everything is there, but it needs the infra to be on
- bash script that just turn the infra on and off before running the tests
- it could be env vars that get skip if missing, set env vars in pipeline
- could be one IntegrationTest=local, which mean all default vars
func TestIntegration(t *testing.T) {
fooAddr := os.Getenv("FOO_ADDR")
if fooAddr == "" {
t.Skip("set FOO_ADDR to run this test")
}
f, err := foo.Connect(fooAddr)
// ...
}- In a bash script
- have a bash script that will start the infra, run the test, and stop the infra
- gnu parallel could be used to run multiple tests at the same time
- easy to do in pipeline
- could be really good to do with a dedicated cli
- closer to production
- could try to build an integration test framework
Provide examples of test cases for critical functionalities.
Describe quality assurance measures, including code reviews, static code analysis, and performance testing.
Outline the project timeline, including major milestones and estimated delivery dates.
Identify potential risks and outline mitigation strategies.
Detail resource allocation, including team roles and responsibilities.
Provide a glossary of terms used in the document.
Include an index, if the document is lengthy.
Space for signatures from key stakeholders approving the document.
Document the history of revisions to the SDD.