-
-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CPU pegged at 100% and sched_yield() called in infinite loop when OpenMP enabled #7190
Comments
We don't maintain the imagick extension so you will need to ask your question somewhere else. |
Thanks for the response! This same issue also happens on standard convert commands, outside of the Imagick extension. But I would be happy to ask the question in a more appropriate location; do you happen to know where that might be? |
The stacktrace that you included contains php and imagick methods so I assumed this was only an issue when using it in php. How can we reproduce this with |
Thanks again for the response, dlemstra. I was able to recreate the issue without Imagick in roughly the same manner. I created two processes, with each process repeatedly adding a watermark to another image, and then I just waited for something to lock up. I am adding watermarks to the top-left corner of another image using the following: Here are the traces for the two processes when the lock happened:
And then the second one that locked up at the same time...
|
What do you mean by I created two processes? We still don't have enough information to reproduce this. And this really feels like an internal OpenMP issue. I suspect this is happening because you are calling our static omp_lock_t your_lock;
omp_init_lock(&your_lock); |
Thank you for your response. Allow me to provide further clarification on the issue I'm experiencing, as it seems there might be a misunderstanding. The issue I'm encountering involves a deadlock scenario during the execution of multiple concurrent ImageMagick processes. While the deadlock does involve the OpenMP library, the root cause appears to be related to the way ImageMagick interacts with OpenMP within a multi-threaded environment. To illustrate the issue, I simplified the scenario by creating two separate processes, each performing a similar task of adding a watermark to an image (different images in each process) using the Here's a breakdown of the observed behavior:
It's important to note that this issue wasn't specific to ImageMagick's Imagick extension, as I was able to reproduce it by directly invoking the I understand your suspicion regarding the shared lock contention. However, the deadlock occurs even when each process operates on separate images, indicating that the contention is not solely due to shared resources within ImageMagick. Given these observations, I believe there might be an underlying issue with how ImageMagick interacts with OpenMP in a multi-threaded environment, potentially leading to race conditions or synchronization problems. |
I should probably also add some more clarification the method This all points to something inside the OpenMP library. We are using We cannot really help you further, it is up to you now to send us a patch if there is an issue inside ImageMagick. But I really think you are looking in the wrong direction. If this would be a general issue inside ImageMagick we would have heard about this from more of our users. |
Thank you for your help and suggestions on this. A fix for the issue was recently added to the OpenMP/LLVM project, here: llvm/llvm-project#88539 |
Thanks for coming back and leaving a message for other users. |
ImageMagick version
7.1.1-26
Operating system
Other (enter below)
Operating system, version and so on
FreeBSD 13.2-RELEASE-p9
Description
We have been using OpenMP with Imagick for as long as I can remember (many years), but after our system was recently updated to PHP 8.1+ and ImageMagick 7.1.1-26, we started experiencing random instances of processes getting locked at 100% CPU usage with
truss
revealing that sched_yield(); is being called in an infinite loop on the locked processes. This appears to happen, on seemingly random occasions, when the Imagick and OpenMP modules initially load. Backtrace details for an example process that was stuck in a sched_yield() infinite loop can be seen below:We have tried updating from PHP 8.1 to 8.3, and we have tried rebuilding Image Magick with a few different options disabled, but the problem persists. Unfortunately, we cannot simply disable OpenMP support, since doing so results in a significant slowdown of the convert processes.
Steps to Reproduce
Repeatedly run a
convert
operation (any operation will do) on an image in a loop, using PHP, until the process gets stuck. Then, view what's happening through the use oftruss
, which will reveal that it is stuck in a sched_yield() infinite loop.Images
No response
The text was updated successfully, but these errors were encountered: