The vivalib framework

###Introduction

If you have developed an OpenCV application to capture video files or cameras, you most likely have rewritten the same while(true) loop using the OpenCV’s VideoCapture API. You basically define an input (i.e. video file or a installed webcam feed) and apply a set of OpenCV’s functions to each captured frame from the sequence to create your application.

These lines of code are probably the most repeated set of instructions when developing image processing or computer vision algorithms. Every programmer creates his or her own variant of the while loop adding the desired algorithm. Although simple, this makes many available algorithms not modular and not easily portable. Probably more important programmers need to recode the same functionalities to their algorithms (e.g. save your results to disk, read a list of images from a folder, execute tasks in different threads).

We saw all these as a disadvantage and wanted to unify all the efforts in one framework. Our goal was to identify the most used functionalities and provide a framework to facilitate this work. This way, if programmers use the same structure you could easily reuse your friend/coworker code; compare outputs of similar algorithms or different variants of the same algorithm. As a member of a computer vision and image processing research lab, you can see the benefits of having such a tool.

The vivalib API is a concurrent C++11 framework to process video or image sequences independently of the source. It sends mouse or keyboard inputs to your algorithm allowing selection of regions of interest or change between algorithm functionalities. Optionally, you could save your results in a video file using a particular codec or a set of images. We make use of the C++11 built-in support for threads, to execute tasks in parallel. Our algorithm will process frames as soon there are available from the source and generate a video output, all in separated threads.

OpenCV's VideoCapture API example:

#include "opencv2/opencv.hpp"

using namespace cv;

int main(int argc, const char * argv[])
{
    VideoCapture cap(0); // open the default camera
    if(!cap.isOpened())  // check if we succeeded
        return -1;

    Mat edges;
    namedWindow("edges",1);
    for(;;)
    {
        cap >> edges; // get a new frame from camera
        GaussianBlur(edges, edges, Size(7,7), 1.5, 1.5);
        Canny(edges, edges, 0, 30, 3);
        imshow("edges", edges);
        if(waitKey(30) >= 0) 
             break;
    }
    
    return 0;
}

Equivalent vivalib program (the camera feed is extracted in parallel to the edge detection routine):

#include "viva.h"

using namespace viva;

int main(int argc, const char * argv[])
{
    
    auto processFrame = [](size_t frameN, Mat &input, Mat &output){
        GaussianBlur(input, output, Size(7,7), 1.5, 1.5);
        Canny(output, output, 0, 30, 3);
    };
    
    Processor processor;
    processor.setProcess(processFrame);
    processor.run();
    
    return 0;
}