An awesome leftover estimator!
Task
•
Final Paper
The goal of this project is to develop a computer vision system capable of scanning a canteen consumer's food tray at the end of a meal to estimate the amount of leftovers for each type of food. Such a system must be able to analyze pairs of images: one of the tray before the meal and one of the tray at the end of the meal. From the first image, the system will recognize the various types of food ordered, keeping track of the initial quantity of each food; at the end of the meal, the system must analyze a new image of the tray, recognizing which types of food are still present and in what quantity.
NOTE: As indicated by the professor, the trays used represent the entire available world and no limitations have been imposed on temporal and spatial complexity
The code structure for identifying food and estimating leftovers is quite simple and it is mainly divided into 4 macro sections that are:
The fundamental idea is to load the images of each tray one at a time and commence with Food Localization, which will be responsible for cleaning the image from objects or parts that are irrelevant or do not contain food, in order to more precisely identify the boxes for each course and/or food.
Once the bounding boxes in the image have been detected, they are passed on to the next section, which will take care of refining and perfecting them through food segmentation, preparing them for classification. This operation also has a "side effect" that further improves the bounding boxes found in the previous section.
At this point, a vector containing each dish present in the image will be returned. After completing this step, the vector containing the outlines of the foods found in the image is passed to the classification section. Herare based on relies on the colors, shapes, and brightness of the image to identify the types of food.
At the same time, at each iteration in each image, the bounding boxes will be drawn for each food with the relative label. Which will be saved and merged with the other tray images at the end to show the final result.
Here you can find the result returned from some of the trays used for testing the project.
 |
 |
 |
 |
The project was developed in C++ using the OpenCV library and employing image processing techniques learned during the Computer Vision course.
It's important to note that the code could have been written using Deep Learning
approaches (Not Machine Learning), which would have significantly simplified and
improved food detection. However, I chose to develop the project exclusively with
the knowledge acquired during the course to apply what I've learned and directly
tackle the challenges and problems.
C++ • OpenCV • Computer Vision
Below there is an example of how you can run the project.
- Clone the repo
git clone https://github.com/Piero24/Food-recognition-and-leftover-estimation.git-
Install OpenCV. details can be found Here
-
From terminal set up the project with CMake
cmake - Compile the project
make - Start the project
./Food-Recognition ./Food-recognition-and-leftover-estimation/datasetWhere ./Food-recognition-and-leftover-estimation/dataset is the path of the dataset
- Food Localization
- Food Segmentation
- Split the dish with different foods
- Food Classification
- Implement the Leftover Estimation
- Performance measurement
- Save the output for comparison
See the open issues for a full list of proposed features (and known issues).
We assume no responsibility for an improper use of this code and everything related to it. We do not assume any responsibility for damage caused to people and / or objects in the use of the code.
By using this code even in a small part, the developers are declined from any responsibility.It is possible to have more information by viewing the following links: Code of conduct • License
MIT LICENSE
Copyright (c) 2023 Andrea Pietrobon
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction.
Preamble
The GNU General Public License is a free, copyleft license for software and other kinds of works.
License Documentation »
👨🏽💻: Pietrobon Andrea
🌐: pietrobonandrea.com
🗄: Food Recognition and Leftover Estimation
My name is Pietrobon Andrea, a computer engineering student at the
My passion turns towards AI and ML. I have learned and worked in different sectors that have allowed me to gain skills in different fields, such as IT and industrial design. To find out more, visit my website »
Copyright (C) by Pietrobon Andrea
Released date: Jul-21-23