Temporary files not deleted

[48design]

Hi,

I'm having trouble with Magick.NET not deleting cache/temp files. Is there something I can do to trigger this? I tried using Dispose() after having no need for an image anymore but this doesn't help against the temp files. I also tried to delete the files afterwards but most of the time don't get access to them.

What's good practice here?

[dlemstra]

You should always call Dispose when you are done with the image. Are you using the latest version of Magick.NET? We have had problems with temp files in the past but we haven't had any reports in the last year. Is it possible to create a small program that demonstrates this issue?

[48design]

Thank you for the fast reply! I've read about this problem years ago while researching.

About the small program: I use a Parallel.For loop to load 40 images (CR2/RAW files) inside a task and continue this task with converting the images in another Parallel.For loop to matrices (Math.NET) calling Dispose() at the end. Nothing very special. After conversion I don't need the images anymore, I do re-convert the matrices to MagickImages later but the temporary files are created in the first stage.

I will try to do it in one go but it was better in two steps for getting feedback about the different stages.

I also checked if it has something to do with debug compilation or exiting the program the hard way but that's not the case, it also happens in release mode.

Edit: I use the latest NuGet package.

[dlemstra]

Would it be possible to create a small program/github repo that I can use to reproduce the issue?

[48design]

I've attached a small program. Please note that the nuget package for MagickNet was removed for a smaller file size.
MagickNetTempfileTest.zip

Thanks!

Btw.: the EXIF reading possibility seems to be broken at least in the latest builds. My images have EXIF information (tested with another library) but I do always get "null" for the EXIF data...

[dlemstra]

Your example does not contain any images and I do really need some images to reproduce this. I can contact you by email if you cannot share the images public and you can email me a download link then.

The program should show demonstrate the issue and create temporary files on your machine each time you run it. My guess is that this has to do with temporary files. You might need to tweak the memory limits (https://www.imagemagick.org/discourse-server/viewtopic.php?t=26801) to force swapping to disk a bit quicker. You can use MagickNET.Initializee(ConfigurationFiles) for that.

And can you open a new issue for the EXIF problem so we can check that separately? Please include an image and some sample code in that issue.

[48design]

Thanks again for your reply and by the way thank you VERY much for MagickNet!! 👍
The images can be downloaded here: https://www.48design.de/downloads/raw_images/raw_images.7z

[dlemstra]

When I run your program it just crashed terrible due to out of memory issues inside the constructor of MagickImage. And because there is no try/catch for reading the image the program exists and leaves the temporary files on disk. With some refactoring I managed to get into the same situation as what you have and I managed to find a solution for your issue.

You are not properly handling errors inside your tasks and that is why you will not always dispose the images properly. I would advise you to rewrite your code and not try to load all the images at once but only process 2-4 images at the same time. Not sure what would be the best limit but you should probably do some testing yourself. You should reduce the number of images in memory because ImageMagick will swap to disk when you are running out of memory. This code might need some tweaking but this is what I have done now:

string[] rawFiles = openFileDlg.FileNames;
int maxConcurrent = 4;

for (int i = 0; i < rawFiles.Length; i += maxConcurrent)
{
    // Local list
    MagickImage[] images = new MagickImage[maxConcurrent];
    int length = Math.Min(maxConcurrent, rawFiles.Length - i);

    Task loadImagesTask = Task.Factory.StartNew(() =>
    {
        // load (and convert from raw)
        Parallel.For(0, length, f =>
        {
            images[f] = new MagickImage(rawFiles[i + f], magickReadSettings);
        });
    });

    Task correctGammaTask = loadImagesTask.ContinueWith(_ =>
    {
        // some image manipulations
        Parallel.For(0, length, f =>
        {
            images[f].GammaCorrect(1.0);
        });
    }, TaskContinuationOptions.OnlyOnRanToCompletion);

    Task lastTask = correctGammaTask.ContinueWith(_ =>
    {
        Parallel.For(0, length, f =>
        {
            // do some conversions here...
        });
    }, TaskContinuationOptions.OnlyOnRanToCompletion);

    try
    {
        // just for debugging to show "finished" message...
        lastTask.Wait();
    }
    // Without this we don't go the finally?
    catch (AggregateException)
    {
    }
    finally
    {
        for (int j = 0; j < maxConcurrent; j++)
        {
            images[j]?.Dispose();
        }
    }
}

p.s. Since this no longer is a Magick.NET issue I hope you will figure out the rest by yourself 👍