Code that commands Romo to identify clutter and move it to a goal.
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Type Name Latest commit message Commit time
Failed to load latest commit information.

Decluttering Robot

What it does

Identify "clutter" (an orange ping pong ball) and command Romo to push the clutter to a predetermined goal. The environment is very constrained and impractical--an upside down cardboard box with a webcam looking down from above.

Video Demos

Current Planning Algorithm

  1. Find the line that passes through the ping pong ball and goal
  2. Get Romo's center on that line found in step 1
  3. Move Romo towards the goal
  4. If the ball or Romo are no longer on the same line, go back to step 1


Works with python 2.7

Stuff you need to pip install:


  1. Place the clutter in environment.

  2. Dock the iDevice and start up the iOS app, RomoComm, and place Romo in the environment.

  3. Start the client-side script.

     $ python client/


  • client
    • Main script. Finds the clutter and commands Romo to push it toward the goal.
    • Means of communication with Romo. Use websockets to communicate with Romo.
    • Interfaces with the webcam and provides OpenCV functionality. I did not write this, it is an opencv sample.
    • depends on this. I did not write this, it is an opencv sample.
  • ios: App for Romo to receive commands from client.


Box: The round hole is for the light source. The square hole is for reaching into and manipulating the environment.

![image]( =480x)

Light source: By closing the box off from all other light sources and providing the same light source every time, the HSV's are consistent at all times of day.

![image]( =480x)


![image]( =480x)

Romo: The bright yellow index card on the back provides a unique color whose HSV is easy to find. The cardboard wedge attached to the front makes pushing the clutter easier.

![image]( =480x)