New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Include non-free algorithms #126

Open
ceccoemi opened this Issue Sep 12, 2018 · 7 comments

Comments

Projects
None yet
6 participants
@ceccoemi
Copy link

ceccoemi commented Sep 12, 2018

Hello,

with the recent update, in Opencv 3.4.3, the non-free algorithms such as SIFT and SURF are no more available.

>>> import cv2
>>> cv2.__version__
'3.4.3'
>>> cv2.xfeatures2d.SIFT_create()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
cv2.error: OpenCV(3.4.3) /io/opencv_contrib/modules/xfeatures2d/src/sift.cpp:1207: error: (-213:The function/feature is not implemented) This algorithm is patented and is excluded in this configuration; Set OPENCV_ENABLE_NONFREE CMake option and rebuild the library in function 'create'

>>> cv2.xfeatures2d.SURF_create()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
cv2.error: OpenCV(3.4.3) /io/opencv_contrib/modules/xfeatures2d/src/surf.cpp:1016: error: (-213:The function/feature is not implemented) This algorithm is patented and is excluded in this configuration; Set OPENCV_ENABLE_NONFREE CMake option and rebuild the library in function 'create'

As the error messages say, the library should be recompiled with -D OPENCV_ENABLE_NONFREE=1.
Would it be possible to reintroduce these algorithms in the pip package?

@skvark

This comment has been minimized.

Copy link
Owner

skvark commented Sep 12, 2018

Those algorithms have been included erroneously before because they were not properly protected in the upstream by the OPENCV_ENABLE_NONFREE flag.

I am not a lawyer. I'm not sure if I can redistribute those algorithms.

Edit: See: opencv/opencv_contrib#1668

@Jindil

This comment has been minimized.

Copy link

Jindil commented Oct 16, 2018

I'm not sure if I can redistribute those algorithms.

I understand your concern. The problem is that SIFT is the gold standard algorithm, it is highly used in research and other non-commercial applications.
It does not mean that it should be freely available and redistributed, but to my mind it is the responsibility of the programmer that make use of it, to make sure he is respecting the patent (which is apparently only covering the USA by the way).
Which implementation he is using and how he had access to it probably does not matter.

The SIFT implementation is anyway available from the C++ repository of opencv, and I am sure that if the patent owner wanted to ban SIFT from opencv they would have done it before.
The thing is that they would have no interest doing that because no one would be able to use SIFT because no handy implementation would be available.

Removing SIFT from this repo is first probably breaking a lot of scripts that were relying on the previous version, and prevent people to test this solution for their problem.
Of course there are others free alternatives but if SIFT is the gold standard, it's for a good reason !

What about adding a warning when a SIFT detector/descriptor object is created ? Like for deprecated stuff but here mentioning the patent.

@skvark

This comment has been minimized.

Copy link
Owner

skvark commented Oct 16, 2018

I know that both SIFT and SURF are used widely. I agree that the programmer or a company is responsible for checking that the patents and licenses of the software are respected.

However, "probably does not matter" is not enough. These pre-built packages are freely available around the world so everything must be considered even if it concerns only USA. There have been other interesting discussions about this very same matter: https://answers.ros.org/question/34557/opencv-patent/

Snippet from https://github.com/opencv/opencv_contrib/blob/master/modules/xfeatures2d/src/sift.cpp:

//    Note that restrictions imposed by this patent (and possibly others)
//    exist independently of and may be in conflict with the freedoms granted
//    in this license, which refers to copyright of the program, not patents
//    for any methods that it implements.  Both copyright and patent law must
//    be obeyed to legally use and redistribute this program and it is not the
//    purpose of this license to induce you to infringe any patents or other
//    property right claims or to contest validity of any such claims.  If you
//    redistribute or use the program, then this license merely protects you
//    from committing copyright infringement.  It does not protect you from
//    committing patent infringement.  So, before you do anything with this
//    program, make sure that you have permission to do so not merely in terms
//    of copyright, but also in terms of patent law.
//    Please note that this license is not to be understood as a guarantee
//    either.  If you use the program according to this license, but in
//    conflict with patent law, it does not mean that the licensor will refund
//    you for any losses that you incur if you are sued for your patent
//    infringement.

I didn't remove anything from this repo. OpenCV developers moved SIFT and SURF behind the non-free build flag in the most recent release which is the way they should have been always but, for some reason, weren't. If I turn the non-free build flag on, it's not only SIFT and SURF that get included in the wheels. There are other non-free modules in OpenCV.

I can't add any warnings in this repository to the OpenCV code (without very cumbersome patches) since this repository just pulls a certain vanilla OpenCV version from the upstream and builds it.

If you need the non-free modules you can build the Python bindings yourself and flip the non-free flag on. After all, that's the official way. These packages are unofficial and not endorsed by OpenCV.

Best solution would be probably to create proper support for sdist for these packages and allow users to force the rebuild of the packages themselves. This requires of course compilers and other tools to be installed on the target system and the installation would take about ~10-15 minutes.

@waveform80

This comment has been minimized.

Copy link

waveform80 commented Oct 16, 2018

but to my mind it is the responsibility of the programmer that make use of it, to make sure he is respecting the patent (which is apparently only covering the USA by the way). Which implementation he is using and how he had access to it probably does not matter.

I'm afraid it does matter. Or to be more precise, it's not in the "definitely doesn't matter" category, or even in the "probably does not matter" category. More like in the "might well matter" category! Here's some background on the interesting question of why distributing source code might be in the "probably does not matter" category, but distributing built binaries isn't:

https://patents.stackexchange.com/questions/13992/in-the-usa-is-it-legal-to-distribute-source-code-that-when-run-would-violate

(which in turn answers why the code is still in the upstream OpenCV repository, just guarded with a compiler flag)

Best solution would be probably to create proper support for sdist for these packages and allow users to force the rebuild of the packages themselves. This requires of course compilers and other tools to be installed on the target system and the installation would take about ~10-15 minutes.

More like 2 hours on a Pi :)

Still, as the person who builds the wheels for opencv-python on the piwheels platform (with the relevant bits from skvark's repo here), I'm entirely in agreement with skvark here and will be following upstream's lead. IANAL (too), and while it's interesting (to some?) to read patent-related SE posts and the odd legal paper I wouldn't trust my conclusions to defend me from a pack of rabid patent attorneys.

@ubaierbhat

This comment has been minimized.

Copy link

ubaierbhat commented Oct 27, 2018

Is it possible to install opencv-python to include these non-free algorithms with pip ?

@skvark

This comment has been minimized.

Copy link
Owner

skvark commented Oct 27, 2018

Old versions have SIFT and SURF included due to the reasons explained above. The latest release does not include them. Any upcoming release will not include them. It's better not to rely on these packages if you wish to use non-free algorithms.

@native-api

This comment has been minimized.

Copy link
Contributor

native-api commented Oct 29, 2018

@Jindil http://answers.opencv.org/question/18259/is-surf-algorithm-used-in-opencv-patented/ <- it says there are better, free alternatives. Judging by https://www.quora.com/Computer-Vision-They-dont-use-SIFT-at-Microsoft-they-use-MOPS-Meaning, they seem to only differ by some reference dataset or something.

http://answers.opencv.org/question/73877/opencv-free-for-commercial-use/ <- this says they are free for non-commercial use only, so if they are included, opencv-python cannot be distributed under the MIT license as it is now, or a commercial user or @skvark could be held liable for patent infringement (depending on jurisdiction).

@ubaierbhat you can tweak CMake command line (cmake_args) in setup.py and make a custom build.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment