Lab 5: Object recognition

In this lab you will practice using supervised learning to train an object recognizer for images obtained from the Cozmo robot's camera.

---

Step 1: The goal of this lab is to enable Cozmo to distinguish the following seven symbols (see picture below) from each other, and from other random scenes the robot might see. To that end you will train a classifier that can recognize the object in the image. As input, your algorithm will be given images previously taken by the robot (see example picture below). You must determine whether one of the seven symbols is in the image, and if so, which one.

We provide an image dataset containing grayscale images taken by the robot. The dataset contains 8 types of images, corresponding to either one of the seven symbols or the "none" category for pictures not containing any symbol (just a picture of an empty arena, cube or wall). Each of the symbols is flanked by vertical bars on the left and right, as can be seen in Figure 2. These can be used to locate the symbol in an image if needed.

We recommend using scikit-image and scikit-learn libraries. If you do not have prior experience with these tools, we recommend reading through:

• "Getting Started" and "A crash course on Numpy for images" section of the scikit-image user guide
• "An introduction to machine learning with scikit-learn" section of the scikit-learn user guide

For both scikit-image and scikit-learn, you are not restricted to only using methods covered in class; you can use the entire functionality of both libraries.

---

Step 2: Under the lab5/ directory, you are provided with the following files:

• train/ - directory containing images to be used for training your model. The correct label of each image is listed in its file name.
• test/ - directory containing images to be used for testing your model. The correct label of each image is listed in its file name.
• imgclassification.py – this is the main file where you will enter your solution. The code already contains functions for reading in the images from the train/ and test/ directories and for formatting the data into an array. Add code to convert pixel values of the images to features that can be used for training a classifier. Then train a classifier of your choice and test its predictive performance on the test set. You will need to fill in three functions in the ImageClassifier class, namely, extract_image_features, train_classifier, and predict_labels. You may add helper functions as needed, but make sure your code is self-contained.

Although we will not evaluate the performance of your implementation, we recommend working on this lab to obtain good recognition rates (>85%) before moving onto Lab 6 where you will integrate the object recognizer into the Cozmo robot's behavior.