Segmentation fault using ORB

[FiReTiTi]

Expected behaviour

I'm using ORB FE = cv2.ORB_create(self.MAX_FEATURES) as features extractor to do registration.

Actual behaviour

But when I call TargetKP, TargetDesc = FE.detectAndCompute(ImTarget, None), I get a segmentation fault error: Fatal Python error: Segmentation fault.

I have to use the option -X faulthandler to be able to find the line that generates this error, and it the one I mentioned above TargetKP, TargetDesc = FE.detectAndCompute(ImTarget, None), but I just get this line, nothing more, not trace.

Steps to reproduce

• example code
  FE = cv2.ORB_create(2013)
TargetKP, TargetDesc = FE.detectAndCompute(ImTarget, None)

ImTarget being a 23K x 29K pixels image encoded on 8 bits. For the record, it is dense with information as it is a tissues section, so full of overlapping cells.

• operating system Linux Centos 7

• opencv-python version 4.5.1

[asmorkalov]

OpenCV uses int type for cols, rows, size in cv::Mat and some other objects. It could be the issue reason. Could you down-scale the image twice and repeat the proceduce again. Also it'll be great if you can provide repro that generates test image and runs ORB on it. Please fix seed value, if random is used.

[FiReTiTi]

Thank you for your answer. 23K x 29K pixels image encoding is still into int range, so likely not the issue.
I didn't mentioned it, but I downsized the images to 20Kx25K and it was still the same issue.

[asmorkalov]

@FiReTiTi Small reproducer with automatically generated image should definitely help.

[FiReTiTi]

I cannot share the image that is way too big, or is there an easy solution to share such big image?

[asmorkalov]

You do not need to share image. It's to large and most probably not meaningful for the bug. The reproducer should auto-generate it:

• generate random noise with RNG in OpenCV with fixed seed and check if the issue reproducible;
• save small tile/very scaled image and fill large image with this tile;
• draw a lot of random lines, circles, etc with OpenCV functions;

What I need to understand first if it's just image size issue or the image contains too many ORG features and it's something ORB specific. Depending on it it'll be easier to create focused reproducer without very large images.

[asmorkalov]

@FiReTiTi do you have any update on the issue? Are you able to reproduce the issue with random generated images?

[FiReTiTi]

@FiReTiTi do you have any update on the issue? Are you able to reproduce the issue with random generated images?

Not yet, but I'll try to reproduce it. Donc close it yet.

[FiReTiTi]

So, here is some code to reproduce it:

import cv2
import numpy as np
from skimage.draw import line_aa

width = 25000 #32760
height = 25000 #32760
image = np.uint8(np.random.randint(0, 50, (height, width)))

border = 23

for i in range(23):
	x1 = np.random.randint(border, width - border)
	x2 = np.random.randint(border, width - border)
	y1 = np.random.randint(border, height - border)
	y2 = np.random.randint(border, height - border)
	rr, cc, val = line_aa(x1, y1, x2, y2)
	image[cc, rr] = val * 255
	image[np.random.randint(border, height - border), np.random.randint(border, width - border)] = 255

target = np.roll(image, 17, axis=0)
target = np.roll(target, 17, axis=1)
print("Images created")
sys.stdout.flush()

fd = cv2.ORB_create(31)
#fd = cv2.SIFT_create(31)

ImageKP, ImageDesc = fd.detectAndCompute(image, None)
print("Features extracted from original image")
sys.stdout.flush()
TargetKP, TargetDesc = fd.detectAndCompute(target, None)
print("Features extracted from target image")
sys.stdout.flush()

It works with SIFT, but not ORB.
For ORB, I have to use the option -X faulthandler to get the trace, else I get a simple old segfault. The trace indicates that the faulty command is ImageKP, ImageDesc = fd.detectAndCompute(image, None), so the features extraction with ORB.

Btw, it works with 20Kx20K images, but no longer with 25Kx25. I haven't narrow it down more than this.

[asmorkalov]

Thanks a lot for the reproducer. I'll take a look.

[asmorkalov]

The issue reproduced with 3.4 and master. Stack trace for 3.4 (in debug):

Thread 1 "python3" received signal SIGSEGV, Segmentation fault.
0x00007ffff3763da1 in cv::HarrisResponses (img=..., layerinfo=std::vector of length 8, capacity 8 = {...}, pts=std::vector of length 67, capacity 128 = {...}, blockSize=7, harris_k=0.0399999991)
    at /home/alexander/Projects/opencv-3.4/modules/features2d/src/orb.cpp:163
163                 int Ix = (ptr[1] - ptr[-1])*2 + (ptr[-step+1] - ptr[-step-1]) + (ptr[step+1] - ptr[step-1]);
(gdb) bt
#0  0x00007ffff3763da1 in cv::HarrisResponses(cv::Mat const&, std::vector<cv::Rect_<int>, std::allocator<cv::Rect_<int> > > const&, std::vector<cv::KeyPoint, std::allocator<cv::KeyPoint> >&, int, float) (img=..., layerinfo=std::vector of length 8, capacity 8 = {...}, pts=std::vector of length 67, capacity 128 = {...}, blockSize=7, harris_k=0.0399999991) at /home/alexander/Projects/opencv-3.4/modules/features2d/src/orb.cpp:163
#1  0x00007ffff3768553 in cv::computeKeyPoints(cv::Mat const&, cv::UMat const&, cv::Mat const&, std::vector<cv::Rect_<int>, std::allocator<cv::Rect_<int> > > const&, cv::UMat const&, std::vector<float, std::allocator<float> > const&, std::vector<cv::KeyPoint, std::allocator<cv::KeyPoint> >&, int, double, int, int, int, bool, int) (imagePyramid=..., uimagePyramid=..., maskPyramid=..., layerInfo=std::vector of length 8, capacity 8 = {...}, ulayerInfo=..., layerScale=std::vector of length 8, capacity 8 = {...}, allKeypoints=std::vector of length 67, capacity 128 = {...}, nfeatures=31, scaleFactor=1.2000000476837158, edgeThreshold=31, patchSize=31, scoreType=0, useOCL=false, fastThreshold=20)
    at /home/alexander/Projects/opencv-3.4/modules/features2d/src/orb.cpp:897
#2  0x00007ffff3769e53 in cv::ORB_Impl::detectAndCompute(cv::_InputArray const&, cv::_InputArray const&, std::vector<cv::KeyPoint, std::allocator<cv::KeyPoint> >&, cv::_OutputArray const&, bool) (this=
    0x100d160, _image=..., _mask=..., keypoints=std::vector of length 67, capacity 128 = {...}, _descriptors=..., useProvidedKeypoints=false) at /home/alexander/Projects/opencv-3.4/modules/features2d/src/orb.cpp:1119
#3  0x00007ffff61d32de in pyopencv_cv_Feature2D_detectAndCompute(PyObject*, PyObject*, PyObject*) (self=0x7ffff6756870, args=0x7fff52e7ec08, kw=0x0)
    at /home/alexander/Projects/opencv-3.4-build-debug/modules/python_bindings_generator/pyopencv_generated_types_content.h:8432
#4  0x000000000050a5a5 in  ()
#5  0x000000000050bf44 in _PyEval_EvalFrameDefault ()
#6  0x0000000000507cd4 in  ()
#7  0x000000000050ae13 in PyEval_EvalCode ()
#8  0x0000000000635262 in  ()
#9  0x0000000000635317 in PyRun_FileExFlags ()
#10 0x0000000000638acf in PyRun_SimpleFileExFlags ()
#11 0x0000000000639671 in Py_Main ()
#12 0x00000000004b0e40 in main ()

[AleksandrPanov]

Problem in HarrisResponses(const Mat& img, const std::vector<Rect>& layerinfo, std::vector<KeyPoint>& pts, int blockSize, float harris_k) from orb.cpp

An integer overflow has occurred in this line:
const uchar* ptr0 = ptr00 + (y0 - r + layerinfo[z].y)*step + x0 - r + layerinfo[z].x;

A fix like this fixes the problem:

const uchar* ptr0 = ptr00 + (size_t)(y0 - r + layerinfo[z].y)*(size_t)step + (size_t)x0 - (size_t)r +
                    (size_t)layerinfo[z].x;

I will add a test and open a pull request.

[asmorkalov]

Please do not use cast for the patch, but set proper local variables types.

[AleksandrPanov]

Here's a simpler reproducer without rand:

reproducer

import cv2
import sys
import time
import numpy as np
from skimage.draw import line_aa

width = 25000
height = 25000
image = np.zeros((height, width), dtype=np.uint8)

border = 23
num_lines = 23

for i in range(num_lines):
	x1 = border + i*100
	x2 = width - border - i*100
	y1 = border + i*100
	y2 = height - border - i*100
	rr, cc, val = line_aa(x1, y1, x2, y2)
	image[cc, rr] = val * 255

target = np.roll(image, 17, axis=0)
target = np.roll(target, 17, axis=1)
print("Images created")
sys.stdout.flush()

fd = cv2.ORB_create(31)
#fd = cv2.SIFT_create(31)
print("ORB_create")

start_time = time.time()
ImageKP, ImageDesc = fd.detectAndCompute(image, None)
print("--- %s seconds ---" % (time.time() - start_time))
print("Features extracted from original image")

sys.stdout.flush()
TargetKP, TargetDesc = fd.detectAndCompute(target, None)
print("Features extracted from target image")
sys.stdout.flush()