OpenEXR seems to have issues with forking

[Amir-Arsalan]

I am using Blender to do some renderings. As far as I know, Blender uses your code to store the renderings in OpenEXR format. In my case, I have to do millions of renderings and want to store the results using OpenEXR. To reduce the time complexity I decided to use Python's multiprocessing.Process to do my renderings faster. However, I noticed that Blender freezes when I create processes and run them. I do not encounter this issue when storing my results in any other format (PNG, HDR, BMP etc) but OpenEXR. I posted a bug report for this in developer.blender.org. Here's the link to the bug report. From the comment that the Blender developer has posted and given that I can store my rendering results in all other formats, it looks like OpenEXR has some issues work forking. I wonder, if OpenEXR developers are aware of such issue or if someone can make sure whether or not OpenEXR library experiences issues with forking. I am running Ubuntu 16.04.

@ehanway-ilm @nickrasmussen @meshula @pstanczyk

[meshula]

This sounds difficult to solve without a way to reproduce the problem that does not involve blender. I asked around a little about batching Blender jobs and the people I asked run blender in batch mode from sh scripts, or they use a job manager such as SLURM. Sorry I haven't got a better answer for you.

[Amir-Arsalan]

@meshula Thank you so much for trying to resolve this issue. Did you click on the bug report link that I posted in my first post? There, I am sharing a piece of code that should reproduce the problem. Let me know if you get a chance to play around with it.

[Amir-Arsalan]

@meshula Hi, I wonder if you got a chance to look into this?

[meshula]

I'm sorry I haven't got anything to suggest, especially since things work unless you import bpy. As I mentioned, others have had problems with mulitprocessing with Blender as well, and work around it, as you have, by running many Blender processes.