Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add removal of nodes / branches v2 #348

Merged
merged 5 commits into from
Apr 12, 2024
Merged

add removal of nodes / branches v2 #348

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
c86821b
add 'delete' element for node/branch removal
nwesem Mar 5, 2024
c357d32
use matrix testing to prevent code duplication
nwesem Apr 9, 2024
db92b7e
update documentation and tests according to PR https://github.com/COV…
nwesem Apr 9, 2024
be1b085
extend tests for instances and deletion of wrong instance
nwesem Apr 9, 2024
55effdb
fix pre-commit
nwesem Apr 10, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
100 changes: 86 additions & 14 deletions docs/vspec2x_arch.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
# vspec2x Design Decisions, Architecture and Limitations



## Overlays

Each overlay is treated as a separate tree and merged (after expansion) on the existing tree.
Expand All @@ -11,7 +9,6 @@ This means that descriptions, comments, datatype, unit and similar attributes ge

Expansion is the process where a branch with instances is replaced with multiple branches.


This will result in that e.g. `Vehicle.Cabin.Door` is expanded to the following branches

* `Vehicle.Cabin.Door.Row1.Left`
Expand All @@ -22,42 +19,118 @@ This will result in that e.g. `Vehicle.Cabin.Door` is expanded to the following
For some exporters expansion can be suppressed by using the `--no_expand` option.
Then instance information will be represented by other means in the resulting output.

## Deletion / Node removal

Nodes can be removed from the tree by using the `delete` element in the overlay.
This is useful when a signal is not used in a next version of the specification or if you
simply want to delete a node or branch from the specification, e.g. you are not using a signal of the base
specification.
Please note, that the deletion for branches will delete all nodes that are connected to that branch (which is
desired behavior). Also, if a branch node is deleted, all nodes that are connected to that branch will be deleted
irrespective of what their delete element value is.

We chose three examples to show what you can do with the delete element. Let's say we have the following excerpt from
the base vehicle signal specification:

```yaml
Vehicle.Service:
description: Service data.
type: branch

Vehicle.Service.DistanceToService:
datatype: float
description: Remaining distance to service (of any kind). Negative values indicate service overdue.
type: sensor
unit: km

Vehicle.Service.IsServiceDue:
datatype: boolean
description: Indicates if vehicle needs service (of any kind). True = Service needed now or in the near future. False = No known need for service.
type: sensor

Vehicle.Service.TimeToService:
datatype: int32
description: Remaining time to service (of any kind). Negative values indicate service overdue.
type: sensor
unit: s
```

Now if you want to delete the `Vehicle.Service.TimeToService` node from the specification, you can do this by adding the
delete element to your overlay like this:

```yaml
Vehicle.Service.TimeToService:
type: sensor
datatype: int32
delete: true
```

Let's say you now want to delete the whole branch `Vehicle.Service` from the specification. You can do this by adding:

```yaml
Vehicle.Service:
type: branch
delete: true
```

Also, the delement element can be used on instances after they have been expanded. If you want to delete a node or
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's neat that this works, BTW. Cool!

As a side question, what kind of error is generated if one tries to delete a non-existing instance? Is it differentiated from an error message generated by trying to delete a node that has no instances?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done even added a test for this case, it fails at this position in the code:

...
        # Type presence should have been tested earlier, but is tested here again for completeness    
        if "type" not in self.source_dict.keys():                                                     
            logging.error("Invalid VSS element %s, must have type", self.name)                        
            sys.exit(-1)

branch that has been expanded using instances you can add the `delete` element to the overlay, let's say the vehicle
only has two doors in the front. In this case we would like to delete the signals for the rear doors:

```yaml
Vehicle.Cabin.Door.Row2:
type: branch
delete: true
```

By adding the `delete: true` to a node or branch all nodes and branches connected to it are deleted by vss-tools
when converting to a different format.

Please note that for branches you need to provide at least the `type` element to
the overlay. For nodes you at least have to provide the `type` and `datatype` elements. Currently, the elements provided
do not have to match the previously given elements in the base specification.

## Expansion and Overlays

Sometimes an overlay only refers to a signal in a specific branch, like:

```
```yaml
Vehicle.Cabin.Door.Row2.Right.NewSignal:
datatype: int8
type: sensor
unit: km
description: A new signal for just one door.
```

We do not want this signal expanded, and the tooling prevents expansion by taking the instance `Row2` and comparing with instances declared for `Door`.
We do not want this signal expanded, and the tooling prevents expansion by taking the instance `Row2` and comparing with
instances declared for `Door`.

```
```yaml
Door:
type: branch
instances:
- Row[1,2]
- ["Left","Right"]
- [ "Left","Right" ]
description: d1
comment: c1
```

It will do this by taking the instances on first level (`Row1` and `Row2`) and if comparing with current branch (`Row2`).
If they match it will repeat the check for `Right`and finally merge `NewSignal` with other signals in `Vehicle.Cabin.Door`.
It will do this by taking the instances on first level (`Row1` and `Row2`) and if comparing with current
branch (`Row2`).
If they match it will repeat the check for `Right`and finally merge `NewSignal` with other signals
in `Vehicle.Cabin.Door`.

Description, comments and other data defined for a specific instance (like `Vehicle.Cabin.Door.Row2.Right.NewSignal` above) have precedence
Description, comments and other data defined for a specific instance (like `Vehicle.Cabin.Door.Row2.Right.NewSignal`
above) have precedence
over data defined for the unexpanded signal `Vehicle.Cabin.Door.NewSignal`.

The merge algorithm tries to be smart, so that if you use `Row*` it assume it is an instantiated branch if branch has an instance declaration of type `Row[m,n]`,
even if the the value of `Row*` is outside the range. It will however in that case not inherit values from the base branch.
The merge algorithm tries to be smart, so that if you use `Row*` it assume it is an instantiated branch if branch has an
nwesem marked this conversation as resolved.
Show resolved Hide resolved
instance declaration of type `Row[m,n]`,
even if the the value of `Row*` is outside the range. It will however in that case not inherit values from the base
branch.

## Linters and Static Code Checkers


### MyPy

[Mypy](https://mypy-lang.org/) is used for static type checking of code.
Expand All @@ -80,7 +153,6 @@ Suppressed error categories include:
and can thus not be analyzed.
* Mypy does not like "method variables" like `load_flat_model.include_index`


### Flake8

Flake8 is used as linter. It is also configured to be used as pre-commit hook.
Expand Down
1 change: 1 addition & 0 deletions tests/vspec/.gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,4 @@
**/out.json
**/out.txt
**/out.vspec
**/out.yaml
71 changes: 71 additions & 0 deletions tests/vspec/test_node_removal/test_files/test.vspec
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# Copyright (c) 2024 Contributors to COVESA
#
# This program and the accompanying materials are made available under the
# terms of the Mozilla Public License 2.0, which is available at
# https://www.mozilla.org/en-US/MPL/2.0/
#
# SPDX-License-Identifier: MPL-2.0

A:
type: branch
description: A is a test node
A.Float:
datatype: float
type: actuator
unit: mm
description: A.Float is a leaf of A of datatype float.
A.Int16:
datatype: int16
type: sensor
unit: rpm
description: A.Int16 is a leaf of A of datatype int16.
A.String:
datatype: string
type: sensor
description: A.String is a leaf of A of datatype string.
deprecation: This is test deprecation, let's say it used to be called Str instead String.
A.StringArray:
datatype: string[]
type: sensor
description: A.StringArray is a leaf of A of datatype string array.
A.B:
type: branch
description: B is a branch of A.
A.B.Int32:
datatype: int32
type: sensor
unit: rpm
description: A.B.Int32 is a leaf of A.B of datatype int32.
A.B.NewName:
datatype: uint32
type: sensor
unit: mm
description: A.B.NewName's old name is 'OldName'. And its even older name is 'OlderName'.
fka: ['A.B.OlderName', 'A.B.OldName']
A.B.IsLeaf:
datatype: string
type: actuator
allowed: ["YES", "NO"]
description: This node is a leaf of the tree and it has allowed values (aka an enum).
A.B.Min:
datatype: uint8
type: sensor
min: 10
unit: percent
description: A leaf that uses a minimum value.
A.B.Max:
datatype: uint8
type: sensor
unit: percent
min: 0
max: 100
description: A leaf that uses a maximum value.
A.C:
type: branch
description: C is a branch of A.
instances: Instance[1,2]
A.C.Test:
datatype: uint32
type: sensor
unit: mm
description: A.C.Test is a leaf of A.C of datatype uint32.
11 changes: 11 additions & 0 deletions tests/vspec/test_node_removal/test_files/test_del_branch_overlay.vspec
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Copyright (c) 2024 Contributors to COVESA
#
# This program and the accompanying materials are made available under the
# terms of the Mozilla Public License 2.0, which is available at
# https://www.mozilla.org/en-US/MPL/2.0/
#
# SPDX-License-Identifier: MPL-2.0

A.B:
type: branch
delete: true
11 changes: 11 additions & 0 deletions tests/vspec/test_node_removal/test_files/test_del_instance_overlay.vspec
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Copyright (c) 2024 Contributors to COVESA
#
# This program and the accompanying materials are made available under the
# terms of the Mozilla Public License 2.0, which is available at
# https://www.mozilla.org/en-US/MPL/2.0/
#
# SPDX-License-Identifier: MPL-2.0

A.C.Instance2:
type: branch
delete: true
12 changes: 12 additions & 0 deletions tests/vspec/test_node_removal/test_files/test_del_node_overlay.vspec
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Copyright (c) 2024 Contributors to COVESA
#
# This program and the accompanying materials are made available under the
# terms of the Mozilla Public License 2.0, which is available at
# https://www.mozilla.org/en-US/MPL/2.0/
#
# SPDX-License-Identifier: MPL-2.0

A.B.Int32:
type: sensor
datatype: int32
delete: true
11 changes: 11 additions & 0 deletions tests/vspec/test_node_removal/test_files/test_del_wrong_instance_overlay.vspec
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Copyright (c) 2024 Contributors to COVESA
#
# This program and the accompanying materials are made available under the
# terms of the Mozilla Public License 2.0, which is available at
# https://www.mozilla.org/en-US/MPL/2.0/
#
# SPDX-License-Identifier: MPL-2.0

A.C.Instance3:
type: branch
delete: true
72 changes: 72 additions & 0 deletions tests/vspec/test_node_removal/test_files/test_deleted_branch.vspec
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
# Copyright (c) 2024 Contributors to COVESA
#
# This program and the accompanying materials are made available under the
# terms of the Mozilla Public License 2.0, which is available at
# https://www.mozilla.org/en-US/MPL/2.0/
#
# SPDX-License-Identifier: MPL-2.0

A:
type: branch
description: A is a test node
A.Float:
datatype: float
type: actuator
unit: mm
description: A.Float is a leaf of A of datatype float.
A.Int16:
datatype: int16
type: sensor
unit: rpm
description: A.Int16 is a leaf of A of datatype int16.
A.String:
datatype: string
type: sensor
description: A.String is a leaf of A of datatype string.
deprecation: This is test deprecation, let's say it used to be called Str instead String.
A.StringArray:
datatype: string[]
type: sensor
description: A.StringArray is a leaf of A of datatype string array.
A.B:
type: branch
description: B is a branch of A.
delete: true
A.B.Int32:
datatype: int32
type: sensor
unit: rpm
description: A.B.Int32 is a leaf of A.B of datatype int32.
A.B.NewName:
datatype: uint32
type: sensor
unit: mm
description: A.B.NewName's old name is 'OldName'. And its even older name is 'OlderName'.
fka: ['A.B.OlderName', 'A.B.OldName']
A.B.IsLeaf:
datatype: string
type: actuator
allowed: ["YES", "NO"]
description: This node is a leaf of the tree and it has allowed values (aka an enum).
A.B.Min:
datatype: uint8
type: sensor
min: 10
unit: percent
description: A leaf that uses a minimum value.
A.B.Max:
datatype: uint8
type: sensor
unit: percent
min: 0
max: 100
description: A leaf that uses a maximum value.
A.C:
type: branch
description: C is a branch of A.
instances: Instance[1,2]
A.C.Test:
datatype: uint32
type: sensor
unit: mm
description: A.C.Test is a leaf of A.C of datatype uint32.
Loading