Skip to content

Plotting Tool

Jump to bottom Edit New page
sedrakGR edited this page Feb 8, 2019 · 10 revisions

Plotting Tool

The MADARA and GAMS middleware provide many tools for data introspection like karl and stk_inspect, which allow command-line text-based printouts of knowledge within a knowledge base. However, many types of data are harder to understand in only episodic text printouts. To help developers with visuals of how data and knowledge change over time in a robotic system or a group of robotic systems, we have also created a python-based visualization system.

The plotter requires that MADARA and GAMS be installed with at least python support (e.g., base_build.sh gams madara python). Once python support is compiled into MADARA and GAMS, this plotter then becomes available with either real-time plotting of knowledge over the network or offline plotting of knowledge that has been saved to disk in the serialized temporal knowledge (STK) format.

Table of Contents

Yaml config file

The plotting tool is configured from a yaml file. Yaml is an open specification that is meant to be human-readable and extensible.

Specifying data source

The plotter works off a data source, which can be an offline or online knowledge base. An offline knowledge base is a serialized temporal knowledge (STK) file that has been saved by tools like karl or by simply saving checkpoints to a file. An online knowledge base is a MADARA knowledge base that is currently being modified by logics or external data sources.

Each option for plotting requires the specification of a schemas directory if custom data types are used.

Plotting Offline from an stk file

To plot from a saved knowledge base, the yaml file needs to be configured with the location of the stk file with stk_file.

The following example shows a yaml seciton for an offline STK file.

data_source:
  stk_file: '/path/to/mystk.stk'
  schemas: '/path/to/schemas/folder'

Plotting Live From a Network Transport

To plot an active knowledge base, the yaml file needs to be configured with a transport entry that includes keys like transport_type. transport_type: specifies MADARA transport type and can be ZMQ, UDP, MULTICAST or BROADCAST. hosts: specifies KB hosts with ports, queue_length: indicates the transport queue length in bytes, read_threads: is the number of read threads to launch for the transport, read_thread_hertz: is the maximum read thread poll frequency, and kb_name: is the kb host name for knowledge base (this is the unique identifier for the kb, which shouldn't conflict with anything else).

The following shows an example configuration in YAML.

data_source:
  transport:
    transport_type: ZMQ
    hosts: ['tcp://127.0.0.1:52502', 'tcp://127.0.0.1:52500']
    queue_length: 100000000
    read_threads: 1
    read_thread_hertz: 1000
    kb_name: nothing
  schemas: '/path/to/schemas/folder'

Specifying Variables to Plot

The plotter supports most common MADARA types in one, two, and three dimensional plots. However, it also supports complex type aggregatation from multiple variables.

There are 3 useful types of knowledge variables and key aggregations you can specify.

  1. The most powerful is the aggregation of variables into a new reference frame. If you want a composite graph to be plotted, you can specify it under the frame_types key. Specify name of the key, e.g. p1_base_link: and under it reference_frame: as an array, e.g. ['p1_odom', 'p1_base_link']. See the example below for how this can be used.

  2. If you want an 'Any' type to be plotted, use the any_types key. Any types are typically capnp types. The plotter is powerful enough to look into a capnp type to retrieve subkeys of custom types.

  3. Native MADARA types on an x or y axis. These can be specified in a direct manner with an entry for the name of the graph and then how the types map to

Shaping Plots into Useful Graphs

  1. plot_<number> to specify certain subplots for the given key. For each axes you need to specify also the index and the name, e.g. 0: 'x' and 1: 'y' or 0: 'x' -1: 'time' (-1 index means plotting the key against time), here first plots x against y.

  2. points_per_plot defines the number of points to keep in the plot, and is defined as a number

  3. 3D defines if the key needs to plotted as 3d or as 2d. If the value has more than 1 subvalues they all will be plotted against time. If 3D is set to True then the key will be attemped to plot in 3D graph.

  4. If the type is a map (it is appearing for capnp types defined in any_types), subplots can be specified with subkeys, e.g. position.x: 'x' and position.y: 'y'. In this case full path to the exact subkey in the map can be passed using ., e.g. position.x subkey for pose_in_map key points to pose_in_map[position][x] in the map. If any of the values in the map is an array, it can be accessed by putting the index, e.g. 'key1.key2.[3].key3', where [3] shows that key2 is an array where we take index 3.

  5. You can also leave the content empty and it will be plotted. If you don't mentioned the plot axes, all the keys are being plotted against time.

  6. You can have several values to be plotted in one graph by specifying the range of indexes for similar KRs, and all the indexes in the range will be plotted. E.g.

  agent.0-2.location:
    plot_1: 
        0: 'x'
        1: 'y'
    points_per_plot: 100

this will plot 0 and 1 indexed values of array for agent.0.location, agent.1.location and agent.2.location

Example Config File

# config.yaml

data_source:

  stk_file: '/media/sedrak/OS/SET/projects/test.stk'
  schemas: '/media/sedrak/OS/SET/projects/capnp_schemas'

knowledge_record_subkeys: 

  # will plot 'sensors.imu.index' key against toi, as it is 
  sensors.imu.index:

  # frames
  frame_types:     
  
    # Default: Use if reference frame is geo and would like to plot full 
    # pose (position and orientation)  
    'p1_base_link':

    
    # First entry should be upstream reference frame
    'p1_base_link': 
      reference_frame: ['geo', 'p1_base_link']
      points_per_plot: 200
    
    #'p1_base_footprint':
    #  points_per_plot: 50
      #3D: True

    # Use if you would like to map specific position/orientation indices 
    # to x, y, or z axes. Value on lhs of colon indicates index of pose array
    # and rhs indicates label for axis. Plot_1, etc. indicate subplots in figure
    'p1_base_link': 
      reference_frame: ['p1_odom', 'p1_base_link']
      plot_1: 
        0: 'x'
        1: 'y'
      plot_2: 
        -1: 'time'
        0: 'x'
      plot_3:
        -1: 'time'
        1: 'y'
      plot_4: 
        2: 'z'
        -1: 'time'
      points_per_plot: 25

    'p1_base_footprint':
      3D: True

  #Any data types
  any_types:

    'state_estimation.pose_graph.pose_in_map.1':
       plot_1:
         pose.position.x: 'x'
         pose.position.y: 'y'
       plot_2:
         pose.position.x: 'x'
         pose.position.z: 'z'

    'state_estimation.pose_graph.pose_in_map.2':
       3D: True
       plot_1:
         pose.position

Running the Plotting Tool

After configuring the yaml file, you can run the plotting tool by calling: python $GAMS_ROOT/port/python/tools/plotting_sample.py <path_to_config.yaml>

Add a custom footer
Add a custom sidebar

Clone this wiki locally