Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(WIP) WAII VISS: Add Apache IoTDB database to supported State Storage backends #20

Closed
wants to merge 2 commits into from

Conversation

slawr
Copy link
Collaborator

@slawr slawr commented Feb 15, 2024

Summary

Currently WAII supports the SQLite and Redis databases as State Storage backends. Extend WAII to support Apache IoTDB database as well.

As we are using a git submodule for the WAII VISS Data Server sources and the changes are not yet upstreamed we must point to a fork that carries the changes. Marking this as a WIP to see if it can be upstreamed before merge which would avoid the upstream CDSP project having to change submodule git reps. Of course anyone wanting to try the code can checkout my PR and build it.

Detail

Development environment:

  • Apache IoTDB v1.2.2
  • Apache IoTDB Go Client v1.1.7

Runtime assumptions:

  • IoTDB server lifetime (e.g. startup and shutdown) is handled externally to WAII
  • Management (e.g. creation/deletion) of the IoTDB timeseries containing VSS nodes is handled externally to WAII.
  • Connection config is currently a build time setting in the Service Manager header. Runtime configuration will be added in a later commit. Runtime configuration is now supported via iotdb-config.json.

Sanity testing

Sanity testing was carried out within a Docker deployment of the COVESA Central Data Service Playground:

  1. Create timeseries within database containing VSS nodes and populate some date.
IoTDB> show timeseries
+-------------------------------------------------------------+-----+---------------+--------+--------+-----------+----+----------+--------+------------------+--------+
|                                                   Timeseries|Alias|       Database|DataType|Encoding|Compression|Tags|Attributes|Deadband|DeadbandParameters|ViewType|
+-------------------------------------------------------------+-----+---------------+--------+--------+-----------+----+----------+--------+------------------+--------+
|          root.test2.dev1.`Vehicle.CurrentLocation.Longitude`| null|root.test2.dev1|   FLOAT|   PLAIN|     SNAPPY|null|      null|    null|              null|    BASE|
|           root.test2.dev1.`Vehicle.CurrentLocation.Latitude`| null|root.test2.dev1|   FLOAT|   PLAIN|     SNAPPY|null|      null|    null|              null|    BASE|
|root.test2.dev1.`Vehicle.Cabin.Infotainment.HMI.DistanceUnit`| null|root.test2.dev1|    TEXT|   PLAIN|     SNAPPY|null|      null|    null|              null|    BASE|
+-------------------------------------------------------------+-----+---------------+--------+--------+-----------+----+----------+--------+------------------+--------+

  1. Using WAII HTML client connect to WAII and 'get' VSS node Vehicle.CurrentLocation.Longitude
  2. Using WAII HTML client connect to WAII and 'set' VSS node Vehicle.Cabin.Infotainment.HMI.DistanceUnit
  3. Check HTML client reports success and values set/returned match the data in the database.
    playground-iotdb-set

Setup

If you wish to try it then using the default configuration WAII makes the following runtime assumptions when accessing the DB:

  1. Your database is root.test2.dev1
  2. That a timeseries exists in the DB with the VSS nodes you want to work with.

Both are easy to create in the DB cli:

  1. With the Playground docker deployment running (see playground docker readme for instructions) connect to the DB CLI from your host:
    $ sudo docker exec -ti iotdb-service /iotdb/sbin/start-cli.sh -h iotdb-service
  2. Create database from CLI command line:
    IoTDB > create database root.test2.dev1
  3. Create timeseries from CLI command line:
IoTDB > CREATE ALIGNED TIMESERIES root.test2.dev1(`Vehicle.CurrentLocation.Longitude` FLOAT, `Vehicle.CurrentLocation.Latitude` FLOAT, `Vehicle.Cabin.Infotainment.HMI.DistanceUnit` TEXT)

4 Add some data into the timeseries:

IoTDB> insert into root.test2.dev1(`Vehicle.CurrentLocation.Longitude`, `Vehicle.CurrentLocation.Latitude`, `Vehicle.Cabin.Infotainment.HMI.DistanceUnit`) values(-42.4567, 22.1234, "MILES")

SUMMARY
Currently WAII supports the SQLite and Redis databases as State Storage
backends. Extend WAII to support Apache IoTDB database as well.

As we are using a git submodule for the WAII VISS Data Server sources
and the changes are not yet upstreamed we must point to a fork that
carries the changes.

DETAIL
Both 'get' and 'set' are supported.

For development Apache IoTDB v1.2.2 and the Apache IoTDB Go client v1.1.7
were used.

Integration was sanity tested using the COVESA Central Data Service
Playground and the WAII HTML Client.

Signed-off-by: Stephen Lawrence <stephen.lawrence@renesas.com>
Update the WAII VISS Data Server submodule to add support for runtime
configuration for the Apache IoTDB connector via a config file.

The following notes are reproduced from the commit message:
The config file is named iotdb-config.json and placed in the vissv2server
directory. The file is in JSON format to follow the pattern set by
existing config files within the VISS server.

The contents of the file are not validated and are assumed to be both
complete and correct.

Example file:
{
		"host": "localhost",
		"port": "6667",
		"username": "root",
		"password": "root",
		"queryPrefixPath": "root.test2.dev1",
		"queryTimeout(ms)": 5000
}

host, port, username and password are the client configuration to be
used to create a session (connection) with the IoTDB Server.

queryPrefixPath is the IoTDB database timeseries prefix to be used to
access VSS nodes/keys in get/set operations.

queryTimeout(ms) is the timeout to be used in milliseconds when querying
the database.

If no config file is found on server startup the default (build-time)
configuration is used.

Signed-off-by: Stephen Lawrence <stephen.lawrence@renesas.com>
@slawr
Copy link
Collaborator Author

slawr commented Feb 28, 2024

Updated PR to include support for configuring the Apache IoTDB connector via a config file. See commit 1ef2fef for details

@slawr slawr linked an issue Mar 15, 2024 that may be closed by this pull request
2 tasks
@UlfBj
Copy link

UlfBj commented Mar 20, 2024

WAII is now VISSR
https://github.com/COVESA/vissr

@slawr
Copy link
Collaborator Author

slawr commented Apr 9, 2024

Closing pull request as the changes contained here have now been merged upstream in the VISSR project and the playground was rebased to use those changes in PR #30 which has now been merged.

@slawr slawr closed this Apr 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Rebase Playground on upstream WAII to get latest IoTDB support
2 participants