Skip to content
Code that commands Romo to identify clutter and move it to a goal.
Branch: master
Clone or download
Latest commit 2e44c54 Jan 26, 2015
Type Name Latest commit message Commit time
Failed to load latest commit information.
client Initial commit Jan 26, 2015
ios Initial commit Jan 26, 2015

Decluttering Robot

What it does

Identify "clutter" (an orange ping pong ball) and command Romo to push the clutter to a predetermined goal. The environment is very constrained and impractical--an upside down cardboard box with a webcam looking down from above.

Video Demos

Current Planning Algorithm

  1. Find the line that passes through the ping pong ball and goal
  2. Get Romo's center on that line found in step 1
  3. Move Romo towards the goal
  4. If the ball or Romo are no longer on the same line, go back to step 1


Works with python 2.7

Stuff you need to pip install:


  1. Place the clutter in environment.

  2. Dock the iDevice and start up the iOS app, RomoComm, and place Romo in the environment.

  3. Start the client-side script.

     $ python client/


  • client
    • Main script. Finds the clutter and commands Romo to push it toward the goal.
    • Means of communication with Romo. Use websockets to communicate with Romo.
    • Interfaces with the webcam and provides OpenCV functionality. I did not write this, it is an opencv sample.
    • depends on this. I did not write this, it is an opencv sample.
  • ios: App for Romo to receive commands from client.


Box: The round hole is for the light source. The square hole is for reaching into and manipulating the environment.

![image]( =480x)

Light source: By closing the box off from all other light sources and providing the same light source every time, the HSV's are consistent at all times of day.

![image]( =480x)


![image]( =480x)

Romo: The bright yellow index card on the back provides a unique color whose HSV is easy to find. The cardboard wedge attached to the front makes pushing the clutter easier.

![image]( =480x)

You can’t perform that action at this time.