How can I use other OpenCV functions ?

[orgicus]

Hi,

I've successfully compiled a swc using the instructions provided.
At the moment I'm playing with other OpenCV functions.
I've swapped

#include "opencv/cv.h"

with

#include <opencv2/core/core.hpp>
#include <opencv2/imgproc/imgproc.hpp>

recompiled, all went fine.

Then added this:

void process(cv::Mat &image, int div=64) {
    //colorReduce
    int nl= image.rows; // number of lines
    int nc= image.cols * image.channels();
    for (int j=0; j<nl; j++) {
        uchar* data= image.ptr<uchar>(j);
        for (int i=0; i<nc; i++) {
            data[i]= data[i]/div*div + div/2;
        }
    }
}

above setFramePtr and in I'm calling it after the RGBA2GRAY
conversion passing in gs_frame

If I try to do f lip:

cv::flip(gs_frame,gs_frame,1);

The code compiles, but the rendering in flash is a bit scrambled (row/column order is a bit strange).

My main problem is when I try to call Canny or threshold.
The swc doesn't compile and I get this in Terminal:

g++ -swc -Wall -O3 -DOSX -I ../include/ -o camlib.swc ./o/*.o camlib.cpp 
WARNING: While resolving call to function 'main' arguments were dropped!
WARNING: While resolving call to function 'stub_null' arguments were dropped!
WARNING: While resolving call to function 'stub_zero' arguments were dropped!
WARNING: While resolving call to function 'stub_zero' arguments were dropped!
WARNING: While resolving call to function 'stub_zero' arguments were dropped!
WARNING: While resolving call to function 'stub_zero' arguments were dropped!
WARNING: While resolving call to function 'stub_zero' arguments were dropped!
Assertion failed: (TLI.isTypeLegal(VT) && "Intrinsic uses a non-legal type?"), function visitTargetIntrinsic, file /Volumes/data/dev/FlaCC/llvm-2.1/lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp, line 2465.
0   llc                                 0x00636dfe _ZNSt8_Rb_treeIN4llvm3sys4PathES2_St9_IdentityIS2_ESt4lessIS2_ESaIS2_EE13insert_uniqueERKS2_ + 6078
1   llc                                 0x006373a2 _ZNSt8_Rb_treeIN4llvm3sys4PathES2_St9_IdentityIS2_ESt4lessIS2_ESaIS2_EE13insert_uniqueERKS2_ + 7522
2   libSystem.B.dylib                   0x92ff705b _sigtramp + 43
3   ???                                 0xffffffff 0x0 + 4294967295
4   libSystem.B.dylib                   0x930845a5 raise + 26
5   libSystem.B.dylib                   0x9309a6e4 abort + 93
6   libSystem.B.dylib                   0x9308720f __assert_rtn + 252
7   llc                                 0x003f3e2a _ZN4llvm11StoreSDNodeD1Ev + 90026
8   llc                                 0x003f5256 _ZN4llvm11StoreSDNodeD1Ev + 95190
9   llc                                 0x003f817c _ZN4llvm11StoreSDNodeD1Ev + 107260
10  llc                                 0x0040bc68 _ZN4llvm11StoreSDNodeD1Ev + 187880
11  llc                                 0x0040d3f2 _ZN4llvm11StoreSDNodeD1Ev + 193906
12  llc                                 0x0040f92e _ZN4llvm11StoreSDNodeD1Ev + 203438
13  llc                                 0x005d1926 _ZN4llvm12FunctionPassD1Ev + 20998
14  llc                                 0x005d1f3a _ZN4llvm12FunctionPassD1Ev + 22554
15  llc                                 0x005d20c5 _ZN4llvm12FunctionPassD1Ev + 22949
16  llc                                 0x00002e44 0x0 + 11844
17  llc                                 0x00001f36 0x0 + 7990
18  ???                                 0x00000006 0x0 + 6

I have experience with actionscript, but am learning c++/opencv, so the above errors do
no help me understand what's wrong whatsoever :(

Also, on my machine, I have OpenCV 2.3.1 installed, but the .o files are generated from the code you supplied in the project.
Would that make any difference ?

Any hints/tips would help.

Thanks,
George

[orgicus]

I've found your stackoverflow post (http://stackoverflow.com/questions/7257620/weird-adobe-alchemy-compiler-fail) and followed the adobe forums thread as well. When I've tried comment out the #if CV_SSE2 part in arithm.cpp, I couldn't find the text in the source file. Still, I've commented out all #if CV_SSE2 blocks and left the else parts in there, but I haven't been able to recompile the swc.

This is the output I've got:

WARNING: While resolving call to function 'main' arguments were dropped!
WARNING: While resolving call to function 'stub_null' arguments were dropped!
WARNING: While resolving call to function 'stub_zero' arguments were dropped!
WARNING: While resolving call to function 'stub_zero' arguments were dropped!
WARNING: While resolving call to function 'stub_zero' arguments were dropped!
WARNING: While resolving call to function 'stub_zero' arguments were dropped!
WARNING: While resolving call to function 'stub_zero' arguments were dropped!
Couldn't allocate output reg for contraint '{ax}'!

How do you make sense of the error messages ?
Have you managed to do a haarcascade load and detection ?

[bonext]

Well, one by one:

1. Scrambled output in flash is (most probably) a result of a different byte ordering in flash vs. opencv, I did some trickery with mixChannels in camlib.cpp, but it seems to me now you can solve it via regular opencv cvtColor functions
2. Canny and threshold are most probably a part of opencv I never touched and stripped, so even though it compiles in general, you get issues when you actually reference stuff. I looked canny and thresh through and my guess is that alchemy doesn't like cv::AutoBuffer calls in canny. Thresh.cpp is totally unstripped, so it should be necessary to delete all SSE_2 optimization from there.
3. you are doing all right by using sources and .o generated from them, that go with this repo. I cut quite a lot from there :)
4. No, I did not try to do a haarcascade detection with my code, but if you need any kind of realtime detection, you should look to Marilena for flash: http://www.francois-tarlier.com/blog/marilena-opencv-port-to-actionscript-3-as3-flash/
   The reason is that anything you can do with alchemy and opencv is some 40 times slower than it is under opencv (some of the forkers did tests), so all this is primarily a proof of concept stuff. Good luck :)

[orgicus]

Thank you taking your time to through this step by step.

1. I'll play with cvtColor and see what happens
2. I've tried Canny and threshold just as basic test to see if I can use other function from OpenCV. How do you do your guesses to what's wrong in the cpp code / What do I need to look for to figure out these things by myself ?
3. I'll have a look at Marilena. I'm not particularly interested in running a face detection sample per se. I was excited by the possibility of doing all the stuff CV stuff that can be done in C++ with OpenCV in flash as well. One project in particular I looked at is this: http://web.mac.com/jsaragih/FaceTracker/FaceTracker.html . I've looked at the source code and it's using a haar cascade briefly and doings lots of maths I don't fully understand yet. There are also a few trained sets which I presume I could [Embed]/load in flash and pass the bytearray to C++, so I thought I could run that in Flash. Since this is primarily a proof of concept, it's seems I can't get very far with this: I will need to dig deep and chance quite a few things in the OpenCV source, disable SSE2 support (which I imagine will add to the overall slowdown) and eventually, if I somehow manage to compile everything, the speed will probably not be good enough for a realtime application ...sigh. Does it make sense to pursue this (getting FaceTracker to run in Flash Player) ?

[bonext]

How do I do any guesses about what's wrong? I simply guess :) Everything else in the code seems to be pretty usual except for AutoBuffer and cv::Sobel calls (any of which therefore could be a problem).

Okay, now about the whole thing. I don't think that it makes real sense to port FaceTracker to flash using alchemy. I'm familiar with the tracker and it uses quite a lot of math for face recognition and then rigid face model fitting. So, again, if you manage to run it somehow, most certainly it will be sloooow.
I guess it is even a better idea to port it to flash in a straightforward way by rewriting it in AS (you could even use parts from Marilena to do initial haar cascade face recognition), and I even think it will take you roughly the same (if not less) time as alchemy hacking.

In general, OpenCV-alchemy could be used in flash to do some non-realtime image analysis such as segmentation and filtering, maybe even alignment and 3d-reconstruction, but in general it is way too slow and insanely hard to run because you actually have to strip all the stuff that possibly would not translate into flash bytecode and even then you can't be sure it will run at all.