traincascade in opencv 2.4.11 and opencv 3.0: "Required leaf false alarm rate achieved.."

[zhihongzeng]

Please state the information for your system

• OpenCV version: 2.4.11 / 3.0
• Host OS: Linux (Ubuntu 14.04)

In which part of the OpenCV library you got the issue?

• apps/traincascade

Actual behaviour

opencv_traincascade -data cascade_output -vec positive.vec -bg negative.txt -numPos 1000 -numNeg 1000 -precalcValBufSize 2000 -precalcIdxBufSize 2000 -featureType LBP -w 50 -h 100

output:
===== TRAINING 0-stage =====
<BEGIN
POS count : consumed 1000 : 1000
Train dataset for temp stage can not be filled. Branch training terminated.
Cascade classifier can't be trained. Check the used training parameters.

the bug

The problem was traced:
in opencv/apps/traincascade/trainscade.cpp:

bool CvCascadeClassifier::train( ..) 
{
...
if ( !updateTrainingSet( requiredLeafFARate, tempLeafFARate ) )
        {
            cout << "Train dataset for temp stage can not be filled. "
                "Branch training terminated." << endl;
            break;
        }
...
}

bool CvCascadeClassifier::updateTrainingSet( ...)

int CvCascadeClassifier::fillPassedSamples(..)

then
CvCascadeImageReader::getNeg(..)
negReader.get(_img) which return false because imread fail to read negative file from negative.txt. That happens because the image file name string have \n\r\t when it is read from negative.txt

The details:
in opecv/apps/traincascade/imagestorage.cpp:

bool CvCascadeImageReader::NegReader::create( const string _filename, Size _winSize )
{
....
    while( !file.eof() )
    {
        std::getline(file, str);
        if (str.empty()) break;
        if (str.at(0) == '#' ) continue; /* comment */
        imgFilenames.push_back(str);
    }
...
} 

str need trim "\n\r\t" before further process.

Here is my solution which was tested in my computer.

bool CvCascadeImageReader::NegReader::create( const string _filename, Size _winSize )
{
....
    while( !file.eof() )
    {
        std::getline(file, str);
        str.erase(str.find_last_not_of(" \n\r\t")+1); // ******* a fix **********
        if (str.empty()) break;
        if (str.at(0) == '#' ) continue; /* comment */
        imgFilenames.push_back(str);
    }
...
} 

[StevenPuttemans]

Simply said, if you run into this problem at stage 0, it means that the files inside your negatives.txt are read incorrectly. Can you make sure that you have used absolute paths inside the txt file instead of relative file paths?

[zhihongzeng]

I tried to use both absolute paths and relative paths and traced to the path before imread. The problem is same (imread failed to read the images due to no trimmed path).

[alalek]

You should provide right "negative.txt" file without garbage like '\t' or '\r'. Try to use dos2unix utility to fix it.

[zhihongzeng]

That is what my proposed fix does. Is there anything wrong about that fix?

[StevenPuttemans]

@mshabunin actually this is not a real bug ... it is a matter of not providing corrupted files with \t or \r at the end.... its a simple unix <-> interchangebility problem ...

[mshabunin]

Personally, I think that applications should either support both popular variants (LF, CRLF) either document supported line ending format and encoding for text input files. In Windows notepad mentioned file conforms to each line contains an image filename rule but application does not accept it, thus I considered it a bug.

[StevenPuttemans]

Well if you look at it like that, then indeed, some fixing should be required, and maybe the proposed fix could do the trick.

[souch55]

Hi mshabunin fixed the issue as stated here in this issue ,please review and merge my changes the PR for this change ##7205

[zhihongzeng]

what is the following line for?
if (str.at(0) == '#' ) continue;