-
Notifications
You must be signed in to change notification settings - Fork 109
trainHOG Tutorial
This short example program demonstrates how to train a custom HOG detecting descriptor vector to use with OpenCV on unixoid operating systems.
For my diploma thesis, I implemented a framework with different training algorithms (SVMlight, LIBSVM, boosting, bag of features and some combinations hereof) using different features (Histograms of Oriented Gradients [HOG], Normal Aligned Radial Features [NARF], Fast Point Feature Histograms [FPFH] among others and combinations) on different Kinect inputs (color/grayscale image, depth image, point cloud and combinations) using OpenCV and ROS among other libraries. The objective was to investigate and evaluate which are the "best" choices of combinations (features, machine learning algorithms) to perform object detection using Kinect input.
One part I implemented was the HOG training algorithm on grayscale images compatible with OpenCV, to which I got several e-Mails from people trying to do exactly that, so I decided to write a small example program to show how I did it. My diploma consultants of the University of Bonn (Germany) kindly granted me explicit permission to publicize and share parts of the implementation, although the source code provided here is nearly completely written anew. This program thus should not be understood as a complete instruction, but rather as a guideline to refer to. Use at your own risk.
For the paper regarding Histograms of Oriented Gradients (HOG), see http://lear.inrialpes.fr/pubs/2005/DT05/.
This program uses, but is not restricted to SVMlight as machine learning algorithm (see http://svmlight.joachims.org/), it can be easily extended for other machine learning algorithms by replacing or extending the wrapper implementation (svmlight/svmlight.h) included with my source code.
Tested in Ubuntu Linux 64bit 12.04 "Precise Pangolin" with OpenCV 2.3.1, SVMlight 6.02, g++ 4.6.3 and standard HOG settings with training images of size 64x128px.
What this program basically does:
- Read positive and negative training sample image files from specified directories
- Calculate their HOG features and keep track of their classes (pos, neg)
- Save the feature map (vector of vectors/matrix) to file system
- Read in and pass the features and their classes to a machine learning algorithm, e.g. SVMlight
- Train the machine learning algorithm using the specified parameters
- Use the calculated support vectors and SVM model to calculate a single detecting descriptor vector
- Set the detecting descriptor vector using and use the camera input to test the newly generated HOG detector.
- Download my example program source code from http://github.com/DaHoC/trainHOG, especially main.cpp and svmlight/svmlight.h or grab the source code from: https://github.com/DaHoC/trainHOG/archive/master.zip
- Download recent SVMlight version from http://svmlight.joachims.org/ (I used version 6.02).
- Get positive and negative sample images, e.g. you can obtain the INRIA positive person training dataset from http://pascal.inrialpes.fr/data/human/. Put them into the example program folders pos/ and neg/ for positive and negative training images respectively.
- Extract the SVMlight files into folder svmlight/ alongside svmlight.h wrapper.
- Now you should be able to compile the code, either using a IDE or issue following commands in a Linux shell in the example program base dir:
g++ `pkg-config --cflags opencv` -c -g -MMD -MP -MF main.o.d -o main.o main.cpp gcc -c -g `pkg-config --cflags opencv` -MMD -MP -MF svmlight/svm_learn.o.d -o svmlight/svm_learn.o svmlight/svm_learn.c gcc -c -g `pkg-config --cflags opencv` -MMD -MP -MF svmlight/svm_hideo.o.d -o svmlight/svm_hideo.o svmlight/svm_hideo.c gcc -c -g `pkg-config --cflags opencv` -MMD -MP -MF svmlight/svm_common.o.d -o svmlight/svm_common.o svmlight/svm_common.c g++ `pkg-config --cflags opencv` -o trainhog main.o svmlight/svm_learn.o svmlight/svm_hideo.o svmlight/svm_common.o `pkg-config --libs opencv`
After successful compilation, the program can be called with .
Supported operating systems
At least one of the functions (opendir()) doing file system operations is Unix/Linux-only, using the program in other operating systems requires calling alternative filesystem API functions. All other source code except the file system access should be cross-platform.
I will not provide source code for closed source operating systems.
Memory considerations
Be aware that the program may consume a considerable amount of main memory, hard disk memory and time, depending on the amount of training samples.
Also be aware that (esp. for 32bit systems), there are limitations for the maximum file size which may take effect when writing the features file.
The code is annotated and variables and functions have descriptive names, so the code functionality should be self-explanatory. If it is not, please don't hesitate to send me an e-Mail.
You can find the source code at my git repository http://github.com/DaHoC/trainHOG. The code is a bit too lengthy to completely paste it on this page, but here goes the important part:
/**
* Main program entry point
* @param argc
* @param argv
* @return EXIT_SUCCESS (0) or EXIT_FAILURE (1)
*/
int main(int argc, char** argv) {
HOGDescriptor hog; // Use standard parameters here
// Get the files to train from somewhere
static vector<string> positiveTrainingImages;
static vector<string> negativeTrainingImages;
static vector<string> validExtensions;
validExtensions.push_back("jpg");
validExtensions.push_back("png");
validExtensions.push_back("ppm");
getFilesInDirectory(posSamplesDir, positiveTrainingImages, validExtensions);
getFilesInDirectory(negSamplesDir, negativeTrainingImages, validExtensions);
/// Retrieve the descriptor vectors from the samples
unsigned long overallSamples = positiveTrainingImages.size() + negativeTrainingImages.size();
// Make sure there are actually samples to train
if (overallSamples == 0) {
printf("No training sample files found, nothing to do!\n");
return EXIT_SUCCESS;
}
/// @WARNING: This is really important, some libraries (e.g. ROS) seems to set the system locale which takes decimal commata instead of points which causes the file input parsing to fail
setlocale(LC_ALL, "C"); // Do not use the system locale
setlocale(LC_NUMERIC,"C");
setlocale(LC_ALL, "POSIX");
printf("Reading files, generating HOG features and save them to file '%s':\n", featuresFile.c_str());
float percent;
/**
* Save the calculated descriptor vectors to a file in a format that can be used by SVMlight for training
* @NOTE: If you split these steps into separate steps:
* 1. calculating features into memory (e.g. into a cv::Mat or vector< vector<float> >),
* 2. saving features to file / directly inject from memory to machine learning algorithm,
* the program may consume a considerable amount of main memory
*/
fstream File;
File.open(featuresFile.c_str(), ios::out);
if (File.good() && File.is_open()) {
File << "# Use this file to train, e.g. SVMlight by issuing $ svm_learn -i 1 -a weights.txt " << featuresFile.c_str() << endl; // Remove this line for libsvm which does not support comments
// Iterate over sample images
for (unsigned long currentFile = 0; currentFile < overallSamples; ++currentFile) {
storeCursor();
vector<float> featureVector;
// Get positive or negative sample image file path
const string currentImageFile = (currentFile < positiveTrainingImages.size() ? positiveTrainingImages.at(currentFile) : negativeTrainingImages.at(currentFile - positiveTrainingImages.size()));
// Output progress
if ( (currentFile+1) % 10 == 0 || (currentFile+1) == overallSamples ) {
percent = ((currentFile+1) * 100 / overallSamples);
printf("%5lu (%3.0f%%):\tFile '%s'", (currentFile+1), percent, currentImageFile.c_str());
fflush(stdout);
resetCursor();
}
// Calculate feature vector from current image file
calculateFeaturesFromInput(currentImageFile, featureVector, hog);
if (!featureVector.empty()) {
/* Put positive or negative sample class to file,
* true=positive, false=negative,
* and convert positive class to +1 and negative class to -1 for SVMlight
*/
File << ((currentFile < positiveTrainingImages.size()) ? "+1" : "-1");
// Save feature vector components
for (unsigned int feature = 0; feature < featureVector.size(); ++feature) {
File << " " << (feature + 1) << ":" << featureVector.at(feature);
}
File << endl;
}
}
printf("\n");
File.flush();
File.close();
} else {
printf("Error opening file '%s'!\n", featuresFile.c_str());
return EXIT_FAILURE;
}
/// Read in and train the calculated feature vectors
printf("Calling SVMlight\n");
SVMlight::getInstance()->read_problem(const_cast<char*> (featuresFile.c_str()));
SVMlight::getInstance()->train(); // Call the core SVMlight training procedure
printf("Training done, saving model file!\n");
SVMlight::getInstance()->saveModelToFile(svmModelFile);
printf("Generating representative single HOG feature vector using svmlight!\n");
vector<float> descriptorVector;
vector<unsigned int> descriptorVectorIndices;
// Generate a single detecting feature vector (v1 | b) from the trained support vectors, for use e.g. with the HOG algorithm
SVMlight::getInstance()->getSingleDetectingVector(descriptorVector, descriptorVectorIndices);
// And save the precious to file system
saveDescriptorVectorToFile(descriptorVector, descriptorVectorIndices, descriptorVectorFile);
printf("Testing custom detection using camera\n");
hog.setSVMDetector(descriptorVector); // Set our custom detecting vector
VideoCapture cap(0); // open the default camera
if(!cap.isOpened()) { // check if we succeeded
printf("Error opening camera!\n");
return EXIT_FAILURE;
}
Mat testImage;
while ((cvWaitKey(10) & 255) != 27) {
cap >> testImage; // get a new frame from camera
// cvtColor(testImage, testImage, CV_BGR2GRAY); // If you prefer working with grayscale images
detectTest(hog, testImage);
imshow("HOG custom detection", testImage);
}
return EXIT_SUCCESS;
}Jan Hendriks (dahoc3150 [at] gmail.com)
Keep up the free and Open Source spirit, this is my way to give back something to the great OpenCV community.
Open Source lives from people sharing their knowledge and experience, so if you have something to share (especially regarding this program), share it!
This program is to be used as an example and is provided on an "as-is" basis without any warranties of any kind, either express or implied.
Use at your own risk.
Feel free to use it as draft, I would be glad it you would send me a note that you are using it.
For used third-party software, refer to their respective terms of use and licensing.
I decided to put this piece of software under the (rather open) Apache 2.0 license.