-
-
Notifications
You must be signed in to change notification settings - Fork 55.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Very large image not supported (area > 2^32) #10211
Comments
I'm planning to "fix" this issue for my job (as it is an urgent matter). |
It will be impossible to fix across the entire OpenCV library. OpenCV can be seen as having several layers of "business value". Among them, one of the layer is the definition of Another of the layer is the vast collection of image processing functions. All of them operate on The issue of assuming that memory address operations don't involve offsets (sizes, lengths, differences) more than 32-bit affects all layers. So, even if you fix the issue in Thus, if it is a job necessity, then the first priority is to identify a minimum subset of OpenCV API that needs to be fixed, and then reimplement those functionalities by taking the code from OpenCV and modify it, or to completely reimplement the functionalities. A typical approach is to use tiling. This involve defining your own large matrix data type, say Years ago I had implemented a tiling approach to workaround a limitation in OpenCV function called OpenCV has not yet adopted a library-wide framework for very large matrices or tiled matrices yet. Usually, it is up to the users of OpenCV to implement such framework. |
I faced the same problem long time ago (my images were 10+ GB). @K-Shinotsuka is correct about tiling idea if you face such limitation in size. Moreover you can enhance his idea by just assigning pointers from your big image/class into cv::mat objects without copy pasting your large memory area. But then you have to write a wrapper/framework to perform the same operation on multiple tiles. |
I think tiling may be a good idea, but will this idea still work on "whole-image" algorithm (like the ones who needs the full histogram of the image, ie. automatic thresholding) ? Even if I implement this idea for myself, I honestly think others will fall in the same "trap" as me. I don't remember seeing any clue that OpenCV could not handle such images. |
As far as I know any version above (or equal to) 3.3.1 doesn't load very large image. I downgraded to 3.3.0.10 and it works. |
@ntomita Thanks for the info. |
@FlorentTomi I see, that's why newer version has assertion statements to void the unintentional crash. Definitely the data type need to be updated so it can handle large images, which are "large" at this time but for next 5 years I believe people, including scientists, wouldn't think those are large as technology updates. |
Is there a plan to address this in OpenCV 4.0, @vpisarev ? IHMO this could go as evolution proposal too. Starting with converting cv::Mat to use size_t everywhere and then we could fix bug-by-bug all the other places using ints. |
done! see #11505 |
Is it done? I tried to open a 49800 x 39401 image and these restrictions are still here opencv/modules/imgcodecs/src/loadsave.cpp Line 66 in ea64e86
|
To work with large files (at your own risk) you can modify environment variables CV_IO_MAX_IMAGE_WIDTH, CV_IO_MAX_IMAGE_HEIGHT, OPENCV_IO_MAX_IMAGE_PIXELS by doing for example
Thaks to @alalek for pointing this out to me. |
What about an ability to change this from inside the program? |
i'm also trying to modify these variables from within a jupyter notebook running on a remote server, i've tried %env os.environ["OPENCV_IO_MAX_IMAGE_PIXELS"] = pow(2,90).str() without success |
@jtoa2 Didn't work for me either. Neither did setting the environment variable directly on Linux. |
Related forum link
http://answers.opencv.org/question/178840/very-very-large-image-causes-issues-with-function-int-type-related/
Detailed description
Large images with an area greater than 2^32 cause crashes or do not work in some functions.
A simple example is the cv::threshold method with Triangle method flag:
The operation
size.width *= size.height;
is done (in file imgproc/src/thresh.cpp, line 1099), but in the case of large images, this overflows the "small" integerint
.Moreover, images with area greater than 2^30 cannot be opened (see imgcodecs/src/loadsave.cpp, line 65). As a workaround, I just modified the mentioned line with
#define CV_IO_MAX_IMAGE_PIXELS((uint64)1 << 40
, but this can't be a long term fix.Suggestions
Maybe using other types (like int64 or size_t) for size-related (or maybe index-related) code may be a good idea.
Maybe this can be activated via a preprocessor directive.
The text was updated successfully, but these errors were encountered: