Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
89 commits
Select commit Hold shift + click to select a range
40d07ad
Groud Truth validator for osistrings
Nov 5, 2018
ba933de
Added KPI definitions for number of messages
Nov 5, 2018
638b860
Updated .gitignore
Nov 5, 2018
7854b7e
Collection of utilities to handle osi data in python
Nov 16, 2018
7378038
Removed Data due to size limitations
Mar 8, 2019
d9780d4
Modified .gitignore
Mar 8, 2019
a461a86
Creation of the KPI validation for messages of osi_sensorview.proto
Mar 15, 2019
93129ca
Add some improvement for Doxygen: renaming md to dox to prevent consi…
Mar 15, 2019
b4e8677
Add details on timestamp and "\c" indicator
Mar 18, 2019
fbbd942
Merge branch 'doxygen-improvements' into development
Mar 18, 2019
a8afa93
Deploy KPIs documentation on github.io
Mar 18, 2019
2ed0649
Correction of a field name
Mar 18, 2019
b1e7701
Merge branch 'development' into gh-pages
Mar 18, 2019
2582b7f
Edit the title of the documentation
Mar 18, 2019
6c16d05
Merge branch 'development' into gh-pages
Mar 18, 2019
6167e7b
Deploy the modified the title of the documentation
Mar 18, 2019
a381784
Add a brief for the AntennaDiagramEntry
Mar 18, 2019
e8ef12b
Merge branch 'development' into gh-pages
Mar 18, 2019
9b06592
deployement add brief to AntennaDiagramEntry
Mar 18, 2019
ae5d566
Add Sensor Data KPIs definition and update the deployment
Mar 18, 2019
98a22b0
Correct some requirements on sensor_id
Mar 19, 2019
939b276
Updated deployment
Mar 19, 2019
3faa5ef
Merge pull request #2 from ainar/development
CezarySzyszkaAltran Mar 19, 2019
10d111e
Update to reader and examples
CezarySzyszkaAltran Mar 20, 2019
d754b2b
Remove obsolete example
CezarySzyszkaAltran Mar 20, 2019
303a3d4
Merge branch 'development' of https://github.com/OpenSimulationInterf…
Mar 20, 2019
940f3eb
Updated test examples
CezarySzyszkaAltran Mar 20, 2019
f7d20e6
Create some documentation for Doxygen KPI documentation and moving it…
Mar 20, 2019
1efdc44
correction of a spelling mistake
Mar 20, 2019
a020bca
Merge remote-tracking branch 'official/development' into development
Mar 21, 2019
97bfe1b
Add YAML and spelling mistakes
Mar 22, 2019
eff8b4f
cleaning
Mar 22, 2019
f6abeb1
Add osi_data_container and rule_checker modules
Mar 22, 2019
3c39dac
First version of a generic validator, working but not complete
Mar 22, 2019
9c26824
fix in is_global_unique
Mar 25, 2019
14e4707
Add the rules in the folder
Mar 25, 2019
1901198
Add support for repeated fields
Mar 25, 2019
dae24f3
progress on KPIs
Mar 25, 2019
a44636d
Add new rules and progress on the validator
Mar 25, 2019
96cee4f
Add rules and rules methods
Apr 2, 2019
dfc9f94
Add all the rules in yml
Apr 3, 2019
537ffaf
Validator is now working, still need some precisions
Apr 4, 2019
ade6e9d
Base validator now working
Apr 5, 2019
aa4daa4
Corrected spelling mistakes and bug fixes of validator
Apr 15, 2019
95b81f5
Spelling and code style fixes
Apr 16, 2019
8a66c7f
Removing old validator
Apr 16, 2019
9d8996d
Add some documentation
Apr 17, 2019
b322f08
Rename to comply with github pages folder "docs"
Apr 17, 2019
519d417
Add KPIs to documentation (architecture change of GH pages)
Apr 17, 2019
9cd3b08
move doxyfiles and update the KPIs doc generator
Apr 17, 2019
ec1ba55
fix generation routine and deploy github pages
Apr 17, 2019
963d13e
Code cleaning and docstring commenting
Apr 17, 2019
0765474
deleting extra now useless stuff
Apr 17, 2019
4a1b4b7
creating documentation of the osi validator
Apr 17, 2019
a2a8adc
add nojekyll file
Apr 17, 2019
8d67da1
fix routine with nojekyll file and some update
Apr 17, 2019
d166916
Readme: documentation about installation
Apr 17, 2019
e532892
add introduction in documentation of OSI Validator
Apr 17, 2019
582a472
fix title in README
Apr 17, 2019
9a128da
spelling mistake corrected
Apr 17, 2019
6e7c90a
wrote usage, writing rulesn completed the home page of the documentat…
Apr 18, 2019
af320ee
Documentation update in writing rules
Apr 18, 2019
a2b2a5e
pylint compliance
Apr 18, 2019
0ce149f
OSIRuleChecker now fully comply with PEP
Apr 19, 2019
7aae784
clearing outputs
Apr 19, 2019
d8cc2cd
update documentation
Apr 19, 2019
6fdeab8
clean code, modify architecture and started autodocumentation generator
Apr 26, 2019
54ec46f
Encapsulation of rules into objects, optimized++
May 2, 2019
8301a0c
Cleaning code and add support of SQLite for log
May 3, 2019
495fe45
cleaning code
May 3, 2019
63214e5
cleaning code
May 3, 2019
18a0bea
add KPIs doc generator (begin)
May 3, 2019
94e9343
Setting of the severity of version increased
May 10, 2019
3676385
update on the generation of documentation from yml
May 10, 2019
786c295
Cleaning and optimisation
May 10, 2019
3790cd9
Packaging
May 16, 2019
99b541b
Cleaning
May 16, 2019
27bb489
Packaging, optimization, cleaning
May 16, 2019
d808fd1
Documentation update according last commits
May 16, 2019
f942985
Move documentation to cleaner tree
May 17, 2019
8bce768
corrected redirection page title
May 17, 2019
b7325e3
Consistency modification and doc update
May 17, 2019
6b012d6
move .nojekyll
May 17, 2019
eab2e28
Update link of documentation
May 17, 2019
5f502dd
add .nojekyll to doc
May 17, 2019
b485aea
edit link and add nojekyll to doc
May 17, 2019
9484e14
Clean documentation
May 17, 2019
4c0e99e
Remove useless doc, update version in conf, clean
May 17, 2019
6be55c3
empty html dir in master
May 17, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -102,3 +102,11 @@ venv.bak/

# mypy
.mypy_cache/

# Visual Studio Code
.vscode

# Madeup extension for OSI byteencoded string
*.osibytes
*.sosibytes
.DS_Store
2,494 changes: 2,494 additions & 0 deletions KPIs/Doxyfile

Large diffs are not rendered by default.

10 changes: 10 additions & 0 deletions KPIs/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
KPISDOCDIR=${KPIDOCDIR}

.PHONY: clean

doxy: clean
doxygen >> /dev/null > /dev/null 2>&1
mv html $${KPIDOCDIR:-here}

clean:
rm -rf $${KPIDOCDIR:-here}/*
22 changes: 22 additions & 0 deletions KPIs/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# KPIs of OSI

## Presentation

As a scrum team, we would like to create a data validity KPI (requirements) for each message of OSI.

The KPI should be followed to create a OSI 3 message to be considered as *valid*.

These KPI requirements are available here on a Doxygen documentation: http://opensimulationinterface.github.io/osi-validation/KPIs/

## How to deploy the Doxygen documentation

### Requirements (Linux only)

- Doxygen (on Debian/Ubuntu: `sudo apt install doxygen`)
- make

### Deploy

Run in the KPIs folder: `make`

To clean the folder: `make clean`
3 changes: 3 additions & 0 deletions KPIs/all_other.dox
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
\namespace osi3

The OSI classes live in namespace osi3 to access this namespace in CPP use
35 changes: 35 additions & 0 deletions KPIs/osi_BaseMoving.dox
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
/*!
\class BaseMoving
\brief The base attributes of an object that is moving.

This includes the \c MovingObject , \c DetectedMovingObject , \c HostVehicleData ,
\c SensorData .

\par Requirements

General requirements:
- All fields must be set and initialized
- Meaningful data must be set for all the data fields



Attribute | Type | Repeated | Unit | Requirements
:----------------|---------------|----------|---------------------|------------------------------------------------------
dimension | Dimension3d | No | [m] | Must be set, all components must be positive
position | Vector3d | No | [m] | Must be set
orientation | Orientation3d | No | [rad] | Must be set, all values must me inside range (-pi,pi]
velocity | Vector3d | No | [m/s] | Must be set
acceleration | Vector3d | No | [m/s<sup>2</sup>] | Must be set
orientation_rate | Orientation3d | No | [rad/s<sup>2</sup>] | Must be set
base_polygon | Vector2d | Yes | [m] | See the notes in \c BaseStationary


\par Difference between GroundTruth and SensorData

All coordinates and orientations from \c GroundTruth objects are relative to the
global ground truth frame. All coordinates and orientations from detected
objects (\c SensorData) are relative to the \c HostVehicle frame.

\par base_polygon data field
See the dicsussion in \c BaseStationay/
*/
47 changes: 47 additions & 0 deletions KPIs/osi_BaseStationary.dox
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
/*!
\class BaseStationary
\brief The base attributes of a stationary object or entity.

This includes the \c StationaryObject , \c TrafficSign , \c TrafficLight , \c RoadMarking messages.

\par Requirements

General requirements:
- All fields must be set and initialized
- Meaningful data must be set for all the data fields


Field | Type | Repeated | Unit | Requirements
:------------|---------------|----------|-------|---------------------------------------------
dimension | Dimension3d | No | [m] | Must be set, all components must be positive
position | Vector3d | No | [m] | Must be set
orientation | Orientation3d | No | [rad] | All values must me inside range (-pi,pi]
base_polygon | Vector2d | Yes | [m] | See the note below


\par Difference between GroundTruth and SensorData
All coordinates and orientations from \c GroundTruth objects are relative to the
global ground truth frame. All coordinates and orientations from detected
objects (\c SensorData) are relative to the \c HostVehicle frame.

\par base_polygon data field

Usage as ground truth:

The two dimensional (flat) contour of the object. This is an extension of the concept of a bounding box as defined by Dimension3d. The contour is the projection of the object's outline onto the z-plane in the object frame (independent of its current position and orientation). The height is the same as the height of the bounding box.

Usage as sensor data:
The polygon describes the visible part of the object's contour.

General definitions:
- The polygon is defined in the local object frame: x pointing forward and y to the left.
- The origin is the center of the bounding box.
- As ground truth, the polygon is closed by connecting the last with the first point. Therefore these two points must be different. The polygon must consist of at least three points.
- As sensor data, however, the polygon is open.
- The polygon is defined counter-clockwise.
- The validity of base_polygon depend on context it is used

If the base_polygon field is used in GroundTruth context it is always a function of object's bounding box dimension,orientation and position. And is always four distinct elements.
If the base_polygon is used in SensorData context it can be polygon of more then four elements but also less the four elements 0 element array if the object is not visible to the sensor at all.
The situation when visible part of the objects consists of two separate contours is not handled
*/
18 changes: 18 additions & 0 deletions KPIs/osi_Dimension3d.dox
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
/*!
\class Dimension3d
\brief The dimension of a 3D box, e.g. the size of a 3D bounding box or its uncertainties.

Dimension is defined in the specified reference coordinate frame along the x-axis (=length), y-axis (=width) and z-axis (=height).

\par Requirements

All fields must be initialized.

All fields must be positive.

Field | Type | Unit | Requirements
:---------|--------|------|----------------:
length | double | [m] | Must be positive
width | double | [m] | Must be positive
height | double | [m] | Must be positive
*/
145 changes: 145 additions & 0 deletions KPIs/osi_EnvironmentalConditions.dox
Original file line number Diff line number Diff line change
@@ -0,0 +1,145 @@
/*!
\class EnvironmentalConditions
\brief The conditions of the environment.

Definition of light, weather conditions and other environmental conditions.


\par Requirements

field | type | repeated | Requirements
-------------------- | --------------------- | -------- | -----------
ambient_illumination | EnvironmentalConditions::AmbientIllumination | no | Must be set and valid
time_of_day | EnvironmentalConditions::TimeOfDay | no | Must be set and valid
atmospheric_pressure | double | no | Must be in Pascal [Pa]; must be set
temperature | double | no | Mun be in Kelvin [K]; must be set
relative_humidity | double | no | Must be in range [0 - 100] [%]; must be set
precipitation | EnvironmentalConditions::Precipitation | no | Must be set and valid
fog | EnvironmentalConditions::Fog | no | Must be set and valid

\note temperature, atmospheric_pressure and relative_humidity refer to these values at z = 0.0 height 0m see level

\note atmospheric_pressure is essensialy [QNH](https://en.wikipedia.org/wiki/QNH). Accepted extreme values are 800 hPa and 1200 hPa.

\note temperature has accepted extreme values of 170\ K and 340\ K.

\note Should we use more common units like. Celcius, hPa to avoid errors.

###############################################################################

\class EnvironmentalConditions::Precipitation
\brief Definition of discretized precipitation states

Definition of discretized precipitation states according to [1].
(I = Intensity of precipitation in mm per hour [mm/h])

\par Requirements

Number | Type name | Requirements
:----- | :------------------ | :---------
0 | PRECIPITATION_UNKNOWN | Intensity of precipitation is unknown (must not be used in ground truth).
1 | PRECIPITATION_OTHER | Other (unspecified but known) intensity of precipitation.
2 | PRECIPITATION_NONE | No precipitation, when I in [0,0.1[ [mm/h]
3 | PRECIPITATION_VERY_LIGHT | Very light intensity of precipitation, when I in [0.1,0.5[ [mm/h]
4 | PRECIPITATION_LIGHT | Light intensity of precipitation, when I in [0.5,1.9[ [mm/h]
5 | PRECIPITATION_MODERATE | Moderate intensity of precipitation, when I in [1.9,8.1[ [mm/h]
6 | PRECIPITATION_HEAVY | Heavy intensity of precipitation, when I in [8.1,34[ [mm/h]
7 | PRECIPITATION_VERY_HEAVY | Very heavy intensity of precipitation, when I in [34,149[ [mm/h]

\par References:

[1] PAULAT, Marcus, et al. A gridded dataset of hourly precipitation
in Germany: Its construction, climatology and application.
Meteorologische Zeitschrift, 2008, 17. Jg. Nr. 6, S. 719-732.

###############################################################################

\class EnvironmentalConditions::Fog
\brief Definition of discretized fog states

Definition of discretized fog states according to [2]. The bandwidth of thick and dense fog was adjusted to fit the German StVO
regarding rear fog lights [3]. (V = Visibility in meters [m])

Visibility is defined as the length of the atmosphere over which a beam of light travels before its luminous flux is reduced to 5% of its original value (definition used by the Meteorological Office [4]).
This is approximately equivalent to visibility measured in terms of the contrast of a distant object against its background.

\par Requirements

Number | Type name | Requirements
:----- | :------------------ | :---------
0 | FOG_UNKOWN | Visibility is unknown (must not be used in ground truth).
1 | FOG_OTHER | Other (unspecified but known) fog intensity.
2 | FOG_EXCELLENT_VISIBILITY | Excellent visibility, when V in [40000,infinity[ [m]
3 | FOG_GOOD_VISIBILITY | Good visibility, when V in [10000,40000[ [m]
4 | FOG_MODERATE_VISIBILITY | Moderate visibility, when V in [4000,10000[ [m]
5 | FOG_POOR_VISIBILITY | Poor visibility, when V in [2000,4000[ [m]
6 | FOG_MIST | Mist, when V in [1000,2000[ [m]
7 | FOG_LIGHT | Fog, when V in [200,1000[ [m]
8 | FOG_THICK | Thick fog, when V in [50,200[ [m]

\par References:
- [2] SHEPARD, Frank D. Reduced visibility due to fog on the highway. Transportation Research Board, 1996.
- [3] [StVO 17 chapter 3](https://www.stvo.de/strassenverkehrsordnung/101-17-beleuchtung)
- [4] [Homepage of the Meteorological Office](http://www.metoffice.gov.uk/guide/weather/observations-guide/how-we-measure-visibility)

###############################################################################

\class EnvironmentalConditions::TimeOfDay
\brief The time of day at the location of the vehicle.

\par Requirements

field | type | repeated | Requirements
-------------------- | ------- | -------- | -----------
seconds_since_midnight |uint32 | no | Must be in range [0 , 86400[

The number of seconds [s] that have passed since midnight local time.
Used for determining the current state of the circadian rhythm of a driver.

\note Rename "local time" to local time zone.

\note No changes of daylight saving time or time zones are considered. So we should specify which time are we using.

\note This field intent is to describe fitness of the driver to operate a vehicle. Maybe we should move this field to Occupant. Or have another field hours awake.

###############################################################################

\class EnvironmentalConditions::AmbientIllumination
\brief Definition of discretized ambient illumination states

Definition of discretized ambient illumination states:
Ambient light is any light in the vehicle's environment that is not
emitted by the vehicle itself. It can include sun/moon light, light from
other cars, street lights etc.

Categorization is done based on natural day/night time illuminance levels
[6] and standards for required lighting levels on roads [6, 7, 8, 9].

\par Requirements

Number | Type name | Requirements
:----- | :------------------ | :---------
0 | AMBIENT_ILLUMINATION_UNKNOWN | Ambient illumination is unknown (must not be used in ground truth).
1 | AMBIENT_ILLUMINATION_OTHER | Other (unspecified but known) ambient illumination.
2 | AMBIENT_ILLUMINATION_LEVEL1 | Level 1 illumination in ]0.001, 0.01[ [lx] E.g. Night with no artificial light. Note Use \c #AMBIENT_ILLUMINATION_LEVEL1 if illumination is less than 0.001 [lx]
3 | AMBIENT_ILLUMINATION_LEVEL2 | Level 2 illumination in [0.01, 1[ [lx] E.g. Night full moon / Deep twilight.
4 | AMBIENT_ILLUMINATION_LEVEL3 | Level 3 illumination in [1, 3[ [lx] E.g. Deep to average twilight / Minimum lighting on local low pedestrian conflict roads.
5 | AMBIENT_ILLUMINATION_LEVEL4 | Level 4 illumination in [3, 10[ [lx] E.g. Average to full twilight / Minimum lighting on collector roads / Minimum lighting on major and expressway roads with low to average pedestrian conflict.
6 | AMBIENT_ILLUMINATION_LEVEL5 | Level 5 illumination in [10, 20[ [lx] E.g. Minimum lighting on major and expressway roads with high pedestrian conflict.
7 | AMBIENT_ILLUMINATION_LEVEL6 | Level 6 illumination in [20, 400[ [lx] E.g. Roads with more lighting / Very dark overcast day to sunrise or sunset on a clear day.
8 | AMBIENT_ILLUMINATION_LEVEL7 | Level 7 illumination in [400, 1000[ [lx] E.g. Sunrise or sunset on a clear day / Overcast day.
9 | AMBIENT_ILLUMINATION_LEVEL8 | Level 8 illumination in [1000, 10000[ [lx] E.g. Average to full daylight.

\note Flickering is not considered. It might be very important for camera sensors. Flickering in tunels.

\note Ambient ilumination color temperature is not considered. It might be important for determination of colors e.g. Road markings white or yellow.

\note Position of the Sun relative to ego might influence the sensor output quality, as well as driver situational awarness.

\par References:
- [5] [The NIST Reference on Constants, Units, and Uncertainty](https://physics.nist.gov/cuu/Units/units.html)
- [6] [National Optical Astronomy Observatory](https://www.noao.edu/education/QLTkit/ACTIVITY_Documents/Safety/LightLevels_outdoor+indoor.pdf)
- [7] [Standards for required street lighting in the USA](http://www.wsdot.wa.gov/research/reports/fullreports/847.1.pdf)
- [8] [Canadian IES-RP-8 standards for road lighting - municipality of Saint-Gedeon-de-. Beauce](http://sslnet.ca/wp-content/uploads/2011/10/LTE-RT-2011-0076-Anglais.pdf)
- [9] [European standards for road lighting](http://courtneystrong.com/wp-content/uploads/2017/07/css-sl1-class-and-quality-of-street-lighting.pdf)
*/
74 changes: 74 additions & 0 deletions KPIs/osi_GroundTruth.dox
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
/*!
\class SensorView

\brief The ground truth information from the simulation environment.

This ground truth information is supposed to describe the whole simulated
environment around any simulated vehicle. For each simulated host vehicle
(there may be one or multiple), define an area around the vehicle which
is greater than the combined field of views (FOV) of all obstructed sensors
in the vehicle. The ground truth data is supposed to describe the convex
hull of all such areas w.r.t. a global simulation coordinate system.

The simulation coordinate system may change during the simulation if and
only if all coordinates w.r.t. this coordinate system are also changed.

The data has to be sent at a rate defined by the receiving partner. When
sending, values with default values might be left default in order to improve
performance.

To provide a complete interface, all fields of all contained messages must be
properly set unless specifically stated in the field's definition that the
field may remain unset.

In enums (e.g. types) the unknown (first / default) value is not allowed to
be used in the ground truth interface.

Field | Type | Repeated | Requirements
-------------------------|----------------------------|----------|---------------------------------------
version | \c InterfaceVersion | No | Must be set and valid
timestamp | \c Timestamp | No | Must be set and valid
host_vehicle_id | \c Identifier | No | Must correspond to the host_vehicle_id
stationary_object | \c StationaryObject | Yes | If set must be valid
moving_object | \c MovingObject | Yes | If set must be valid
traffic_sign | \c TrafficSign | Yes | If set must be valid
traffic_light | \c TrafficLight | Yes | If set must be valid
road_marking | \c RoadMarking | Yes | If set must be valid
lane_boundary | \c LaneBoundary | Yes | If set must be valid
lane | \c Lane | Yes | If set must be valid
occupant | \c Occupant | Yes | If set must be valid
environmental_conditions | \c EnvironmentalConditions | No | Must be set and valid
country_code | \c uint32 | No | Must be set and correspond to a ISO-3166 country code
proj_string | \c string | No | If set must be valid
map_reference | \c string | No |


\par Details on timestamp

The data timestamp of the simulation environment. The zero time point is
arbitrary but must be identical for all messages.
Recommendation: Zero time point for start point of the simulation.

\note Zero time point does not need to coincide with the UNIX epoch.

\note For ground truth data this timestamp coincides both with the
notional simulation time the data applies to and the time it was sent
(there is no inherent latency for ground truth data, as opposed to
sensor data).

\par proj_string
The string follows the PROJ.4 project rules for projections [1].

\par References:
- [1] [Proj.4 Projections] (https://proj4.org/usage/projections.html)

\par Details on map_reference
Opaque reference of a map.

\note Origin and orientation of the map have to coincide with the
inertial coordinate frame of the ground truth.

\note It is implementation-specific how map_reference is resolved.

\note OSI uses singular instead of plural for repeated field names.
*/
Loading