Lab 2

Link to source code

Team Members:

Name: Avni Mehta, Class Id: 11

Name: (Trinadha) Raji Muppala, Class Id: 14

Task 1 Objective:

Build an application to summarize a video by using Clarifai API. Use OpenImg Library to detect key frame images from the Clarify API.

Input:

The code is run for three different input videos (<15 second) of McDonald’s commercial, cars and kids

Code

Explaination:

1. In KeyFrameDetection.java, import required libraries for Spark, OpenImj and Clarifai API
2. In public class KeyFrameDetection, for the given .mkv video, get all the frames from the video.
3. For frames extraction, iterate over video frames and select the main frames in the video.
4. Compare SIFT features with neighbouring images. When common features < certain threshold, shot the transition.
5. Find all the main key points, collect them and output the results as mainframes.
6. Connect to the Clarifai API server by using API key and access code for the token.
7. Using this connection, access the mainframes file and scan the image in detail and predict information present in the image.
8. Update the image with all the possible contents in each image. Output the image.

Screenshots:

Output:

Image Annotation for McDonald’s commercial Key Frame:

Image Annotation for car video Key Frame:

Image Annotation for kids video Key Frame:

Output on the console:

Task 2 Objective:

Classify images related to project and using Random Forest, Decision Tree and Naïve Bayes models and compare their accuracy

Input:

Images are from MS COCO dataset. We have Identified 4 categories – animals, people, food, interiors. Divided data into train and test4 (for test data).

Code:

a. Random Forest Model: We have modified the already given code for Random Forest according to the categories based for our VQA project.

After importing the required libraries, a cluster vector is created and the image categories is stored as a List. Then we start extracting information from the dataset and save the features from the images.

Clusters are processed using Kmeans. The feature data is loaded and parsed. Here, we are clustering the data into 4 classes. ‘Within Set Sum of Squared Errors’ is calculated. Then histogram for all images and reduced features is created

Build the random forest model from the histogram. Data is split into 70% (training) and 30% (testing). Random forest is trained for 4-10 trees. Best error and parameters is printed. The model is saved.

Test classification using the test images and histogram size. Determine the prediction for the image. Next, generate confusion matrix and print the model accuracy.

b. Naïve Bayes Model: Build the Naïve Bayes model from the histogram. Data is split into 70% for training and 30% for testing. Compute test error by evaluating the model on test instances.

Set spark configuration and settings. Run the function that trains the naïve bayes model. Test the model.

Test image classification by using test images. Use histogram size to predict the model. Generate confusion matrix and print accuracy.

c. Decision Tree Model: Similar to the Naïve Bayes model, build Decision Tree model from the histogram. Data is split into 70% for training and 30% for testing.

Compute test error by evaluating the model on test instances. Set spark configuration and settings.

Run the function that trains the decision tree model. Test the model.

Test image classification by using test images. Use histogram size to predict the model. Generate confusion matrix and print accuracy.

Output:

Observation: Random Forest model has highest accuracy as compared to Decision Tree model and Naïve Bayes Model. As expected, the Naïve Bayes model accuracy is lowest amongst the three. Overall the accuracy is in the lower range because a subset of the dataset is used.

Contritbution of Team Members:

100% both team members completed the assignment together.