Skip to content


Switch branches/tags

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time

Build Status


A collection of tools for offline processing of data from a robotic black box (as implemented in

The toolbox primarily uses Jupyter notebooks for data plotting and analysis with the hope that this simplifies the sharing, presentation, and above all, processing of data.


  • Data processing relies on the black_box_utils package that is included here as a standalone Python library
  • pymongo
  • numpy
  • matplotlib
  • scipy
  • rospy
  • rospy_message_converter
  • yaml
  • termcolor


The following notebooks are provided with this toolbox:

  • sw_data_plot: Plots data from the smart wheels of a platform
  • cmd_vel_plot: Plots velocity commands
  • cmd_vel_plot-multi_db:
  • pivot_encoder_plot-multi_db:


Plots various smart wheel measurements, in particular:

  • wheel currents
  • wheel voltage measurements
  • wheel velocities
  • IMU acceleration measurements per wheel
  • IMU gyroscope measurements per wheel


Plots planar base velocity commands, namely:

  • linear velocity along x
  • linear velocity along y
  • angular velocity


A Python package implementing various utilities for working with black box data. There are three scripts that are part of this package: data_utils, db_utils, and plot_utils.


Defines a DBUtils class with the following static methods:

  • restore_db: Restores a MongoDB database dumped in a given directory
  • get_all_docs: Returns all documents contained in a specified collection of a given database
  • restore_subdbs: Restores multiple dumps into a single database.
  • dump_db: Dumps a MongoDB database in the specified directory.
  • get_data_collection_names: Returns the names of all black box data collections in the specified database.
  • clear_db: Drop all collections in the given database.
  • drop_db: Drops the given database.
  • get_subdb_metadata: Returns a list of dictionaries containing metadata about any sub-databases stored in the given database.
  • get_subdb_docs: Returns a list of dictionaries in which the keys are sub-database names and the values are lists of documents in the given collection corresponding to the respective sub-databases.
  • get_docs: Returns all documents contained in the specific collection of the given database within given time duration.
  • get_docs_of_last_n_secs: Return documents from given collection name of last n seconds from given db name.
  • get_doc_cursor: Returns a cursor for all documents in the specified collection of the given database which have the given 'timestamp' value in the given range.
  • get_collection_metadata: Returns the entry of the 'black_box_metadata' collection for the specified collection.
  • get_oldest_doc: Returns the oldest document in the given collection name.
  • get_newest_doc: Returns the newest document in the given collection name.
  • get_last_doc_before: Returns the last document in the collection with the given name that is before given timestamp.
  • get_db_oldest_timestamp: Gets the oldest record in the mongo db and returns the corresponding timestamp.
  • get_db_newest_timestamp: Gets the newest record in the mongo db and returns the corresponding timestamp.
  • get_db_client: Returns a MongoDB client at :.
  • get_db_host_and_port: Returns a (host, port) tuple which is ("localhost", 27017) by default, but the values can be overridden by setting the environment variables "DB_HOST" and "DB_PORT" respectively.


Defines a DataUtils class with the following static methods:

  • get_all_measurements: Returns all measurements of a single variables (that potentially has multiple instances - e.g. one instance per wheel)
  • filter_data: Filters data using a given data filter and returns the filtered array.
  • find_correlated_variables: Returns a list of variable name pairs where each pair denotes that two measurements in a given measurement matrix are correlated (based on the Pearson correlation coefficient).
  • get_windowed_correlations: Returns a list of variable name pairs and a 2D list of windowed pairwise correlations, given a list of variable names and a data array.
  • get_var_value: Returns the value of a given variable in the given dictionary;
  • get_variable_list: Returns a list of the names of all variables stored in a given collection.
  • get_flattened_variable_names: Recursive method that returns a flattened list of the variable names in a given dictionary
  • expand_var_names: Generates a new list of variable names from a given list of such that the * character in each entry of the original list is replaced by a zero-based index
  • get_bb_query_msg_template: 'Returns a dictionary which represents a template for a black box query message.
  • get_bb_query_msg: Returns a black box data query message.
  • get_bb_latest_data_query_msg: Returns a black box latest data query message.
  • parse_bb_variable_msg: Returns a nested dictionary that reconstructs the structure of the data represented by the variables in the list of a given black box variable message dictionary.
  • parse_bb_data_msg: Returns a tuple (variables, data), where variables is a list of variables that were queried and data a list of variable values.
  • parse_bb_latest_data_msg: Returns a tuple (variables, data), where variables is a list of variables that were queried and data a list of the latest variable values.
  • split_into_windows: Given a one-dimensional list of elements "data", returns a 2D numpy array of sliding windows of size "window_size".
  • safe_literal_eval: Uses ast.literal_eval to parse a given string.


Defines a PlotUtils class with the following static methods:

  • subplot_data: Plots a single time series on a subplot (and potentially annotates any events of interest)
  • subplot_data_lists: Plots multiple time series on a single subplot (and potentially annotates any events of interest)
  • plot_position_velocity: Plots the position of robot with dots and its velocity at each position with a line. Colors represent the intensity of velocity.


Python library for interacting with a robotic black box







No releases published


No packages published