Design Automated Testing / Eval Framework

[andrewjong]

See #333

Need a way to ensure performance persists across versions.

[andrewjong]

We should start with Systems Testing, because if we go any lower we risk getting lost in the weeds and exponentially increasing testing run time. Systems Testing seems most relevant to what we’d want to observe to maintain system performance across versions

• Unit Test: make sure that every function produces the expected output given different arguments. This belongs in the ROS2 package itself and written in the same language as the functions being tested. For example, that the make_plan() function in random walk consistently only produces plans within the given bounds
• Integration test: make sure that specific modules work together, so i.e. specific small set of ROS2 nodes work with other ROS2 nodes, e.g. that disparity_expansion works with droan_local_planner. These would also live in robot/ros_ws itself
• System test: make sure that the system works as a whole, e.g. simulation with autonomy, or e.g. that sending a set of tasks all produces expected behavior (takeoff, Navigate to waypoint, land). I think this is the correct level for testing containers together. This one would orchestrate and probe containers
• Acceptance test: put it in front of end-users and get their green light that they’re happy with it. I guess this is more like user studies and observing user behavior with the software