Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

T-API python support implemented #6078

Merged
merged 1 commit into from Oct 5, 2016

Conversation

9 participants
@PolarNick239
Copy link
Contributor

commented Feb 7, 2016

cv2.UMat implemented - python thin UMat wrapper
No implicit copy from GPU to Host done, resulting UMat can be passed to next function without overhead cv2.UMat.get() - to fetch data to Host

New tests covers: ORB, BFMatcher, goodFeaturesToTrack, calcOpticalFlowPyrLK

NB: effective support for vector<UMat> was not implemented (vector<Mat> used).

Example of usage:

ps, descs_umat = orb.detectAndCompute(cv2.UMat(img), None)
descs = descs_umat.get()
matcher = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck=True)
res = matcher.match(descs_umat, descs_umat)

For more you can see tests.

PolarNick239 added a commit to PolarNick239/opencv that referenced this pull request Feb 8, 2016

T-API python support implemented
PullRequest to master branch: opencv#6078

@PolarNick239 PolarNick239 force-pushed the PolarNick239:master branch 3 times, most recently from 2e69c4f to 6136796 Feb 8, 2016

@PolarNick239

This comment has been minimized.

Copy link
Contributor Author

commented Feb 9, 2016

Hm, It seems that failed tests are non deterministic - can you re-run OpenCL build please?

@PolarNick239 PolarNick239 force-pushed the PolarNick239:master branch from 6136796 to eda4a09 Feb 9, 2016

T-API python support implemented:
 - cv2.UMat implemented - python thin wrapper for UMat
 - no implicit copy from GPU to Host done, resulting UMat can be passed to next function without overhead
 - cv2.UMat.get() - to fetch data to Host
 - new tests covers: ORB, BFMatcher, goodFeaturesToTrack, calcOpticalFlowPyrLK

@PolarNick239 PolarNick239 force-pushed the PolarNick239:master branch from eda4a09 to 46e08d3 Feb 9, 2016

PolarNick239 added a commit to PolarNick239/opencv that referenced this pull request Feb 9, 2016

T-API python support implemented
PullRequest to master branch: opencv#6078
@ghost

This comment has been minimized.

Copy link

commented Feb 12, 2016

This is fantastic news for python and opencv :D

@alalek alalek referenced this pull request Feb 18, 2016

Closed

T-Api in Python bindings #5043

@alalek

This comment has been minimized.

Copy link
Contributor

commented Mar 2, 2016

@vpisarev There is an example of generated wrappers:

static PyObject* pyopencv_cv_Canny(PyObject* , PyObject* args, PyObject* kw)
{
    using namespace cv;

    {
    PyObject* pyobj_image = NULL;
    Mat image;
    PyObject* pyobj_edges = NULL;
    Mat edges;
    double threshold1=0;
    double threshold2=0;
    int apertureSize=3;
    bool L2gradient=false;

    const char* keywords[] = { "image", "threshold1", "threshold2", "edges", "apertureSize", "L2gradient", NULL };
    if( PyArg_ParseTupleAndKeywords(args, kw, "Odd|Oib:Canny", (char**)keywords, &pyobj_image, &threshold1, &threshold2, &pyobj_edges, &apertureSize, &L2gradient) &&
        pyopencv_to(pyobj_image, image, ArgInfo("image", 0)) &&
        pyopencv_to(pyobj_edges, edges, ArgInfo("edges", 1)) )
    {
        ERRWRAP2(cv::Canny(image, edges, threshold1, threshold2, apertureSize, L2gradient));
        return pyopencv_from(edges);
    }
    }
    PyErr_Clear();

    {
    PyObject* pyobj_image = NULL;
    UMat image;
    PyObject* pyobj_edges = NULL;
    UMat edges;
    double threshold1=0;
    double threshold2=0;
    int apertureSize=3;
    bool L2gradient=false;

    const char* keywords[] = { "image", "threshold1", "threshold2", "edges", "apertureSize", "L2gradient", NULL };
    if( PyArg_ParseTupleAndKeywords(args, kw, "Odd|Oib:Canny", (char**)keywords, &pyobj_image, &threshold1, &threshold2, &pyobj_edges, &apertureSize, &L2gradient) &&
        pyopencv_to(pyobj_image, image, ArgInfo("image", 0)) &&
        pyopencv_to(pyobj_edges, edges, ArgInfo("edges", 1)) )
    {
        ERRWRAP2(cv::Canny(image, edges, threshold1, threshold2, apertureSize, L2gradient));
        return pyopencv_from(edges);
    }
    }

    return NULL;
}

UMat code path is used as a fallback if some parameters can't be used as Mat objects or invalid. It is not very effective, but it should not introduce significant performance impact to the original "Mat"-like code. Also there is no mixing of UMat/Mat object types (mat = fn(umat, umat, mat) uses all UMat for C++ function, return type is also forced to UMat).

@nzjrs

This comment has been minimized.

Copy link
Contributor

commented Mar 2, 2016

but it should not introduce significant performance impact to the original "Mat"-like code.

I feel like this should be tested...

@PolarNick239

This comment has been minimized.

Copy link
Contributor Author

commented Mar 3, 2016

UMat code path is used as a fallback if some parameters can't be used as Mat objects or invalid. It is not very effective, but it should not introduce significant performance impact to the original "Mat"-like code.

It has no performance impact on usages without UMat (in case of single-signature functions). And in case of multiple-signature functions this pattern already used:
Try next signature, if types conversion failed - fallback to next signature

So if user does not now about T-API it will not see any difference on usage or performance.

If user wants to use T-API - he just passes one of params wrapped with cv2.UMat(param), and unwrap results when he needs them.

The alternative is to write accurate if with check that one of passed InputArrays is Umat, and then branching. But IMHO this is very error prone - because of auto-generating nature.

return type is also forced to UMat

Python style is to return results via returned tuple, not via arguments, so user have no possibility to specify result type. I don't see problems here.

Tests implementing for Python is complicated - you can not just call function in old way and in new way with different params wrapped with UMat, and then compare results for equality - because:

  1. Results of CPU and OpenCL implementations can vary a lot
  2. Some old way usages are broken (seems that the reason in absence of python bindings) (for example when I was implementing tests for this PR I faced broken ORB - #6081)

Also there is no mixing of UMat/Mat object types (mat = fn(umat, umat, mat) uses all UMat for C++ function

Is it bad in some way? User still can mix them on his python side.

@alalek

This comment has been minimized.

Copy link
Contributor

commented Mar 3, 2016

I have no more comments.
Looks good to me! 👍

@vpisarev Please take a look

@PolarNick239

This comment has been minimized.

Copy link
Contributor Author

commented Mar 23, 2016

Up! :)

@vpisarev

This comment has been minimized.

Copy link
Contributor

commented Oct 5, 2016

@PolarNick239, sorry for a huge delay. And big thanks for the contribution! Let's finally merge it in! I'm not completely happy with some of the decisions; in particular, I'd rather convert numpy array to cv::Mat and cv2.UMat to cv::UMat, then wrap it into InputArray/OutputArray and then let the function decide what to do with mixed-type arguments. But the proposed solution looks good enough to be merged. We can refine it further. Once again, thank you for the contribution! 👍

@opencv-pushbot opencv-pushbot merged commit 46e08d3 into opencv:master Oct 5, 2016

1 check passed

default Required builds passed
Details
@Isha8

This comment has been minimized.

Copy link

commented Mar 3, 2017

I am new to using github and openCV.
@PolarNick239 , when I add cv2.UMat, I get error module cv2 has no attribute UMat, could you please tell if I need to download or install anything for this?

PS: The original error that I was getting was for :

flann = cv2.FlannBasedMatcher(index_params,search_params)
matches = flann.knnMatch(des1,des2,k=2)

the des1 and des2 are SIFT descriptors.

Also , I tried setting openCL(False) but the error "data should be normally NULL" didn't go for me.

Thank you.

@PolarNick239

This comment has been minimized.

Copy link
Contributor Author

commented Mar 3, 2017

@Isha8, what version of OpenCV are you using?

@Isha8

This comment has been minimized.

Copy link

commented Mar 3, 2017

I am using OpenCV 3.1
It worked without error when I switched to BF matcher from Flann
And with BF matcher, I dont get "data should be NULL" error even without UMat or setting ocl to False.

@PolarNick239

This comment has been minimized.

Copy link
Contributor Author

commented Mar 3, 2017

This feature was merged and released with 3.2 version.

@Isha8

This comment has been minimized.

Copy link

commented Mar 3, 2017

Okay, thank you.
Would upgrade to the newer version.
This is really fascinating how the open source things works.
Cheers :)

@crackwitz

This comment has been minimized.

Copy link
Contributor

commented Jun 18, 2017

What's the suggested way to use cv::abs() for UMats in python? you seem to have omitted cv::abs()/cv2.abs() because numpy can do that, but numpy can't do that for UMats, and I haven't seen anything in cv2.ocl with that functionality.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.