Different results from same code

[jzadra]

I'm having an issue where running the same code will sometimes produce the desired result, and other times just produce a blank white image of the correct dimensions.

I have a suspicion that it is a memory issue for the following reasons:

1. The more operations I chain, the more likely I am to get a blank result.
2. The exact same code works almost every time on another computer with a better processor and 128 GB of ram (the problem computer has 16 GB). The method of install (just using magick package binary, no brew install) on each computer is the same, as is the version of R (3.5.2), magick (2.0), and OSX (10.14.2).

I am also experiencing problems where the R temp dir will fill up with magick temp files to the point of using all the drive's free space in case that is related.

So, if it is a memory issue, is there something I can adjust to eliminate this problem? If someone suspects a different issue, I can test. Please let me know what more info I can provide.

Here is file and the code:

require(tidyverse)
require(magick)
blank_fills <- image_read("blankfills-01.png")

blank_fills
# A tibble: 1 x 7
#  format width height colorspace matte filesize density
#  <chr>  <int>  <int> <chr>      <lgl>    <int> <chr>  
# 1 PNG    10626  13750 sRGB       TRUE    577969 118x118


background <- blank_fills %>% 
  image_background("white")

background
# A tibble: 1 x 7
#  format width height colorspace matte filesize density
#  <chr>  <int>  <int> <chr>      <lgl>    <int> <chr>  
# 1 PNG    10626  13750 sRGB       TRUE         0 118x118

#(Result is white image most of the time on one computer, but sometimes works.  result is correct most the time on computer with more memory.  The info output does not differ depending on whether it worked correctly or not (ie filesize is always showing 0 in both cases).

[jeroen]

It would be a bug in the version of imagemagick that we ship with the CRAN package. To narrow it down would it be possible to test this with another version of imagemagick? E.g. on macos you can do:

brew install imagemagick@6

And then

install.packages("magick", type = 'source')

Or alternatively try on Windows or Linux. If the problem does not appear here, then It was likely a bug in imagemagick and I can easily fix it by upgrading the version of imagemagick that we ship on CRAN for the macos package.

[jzadra]

I tried the code on an older PC, and it worked.

I also removed magick, then installed imagemagick@6, and then installed magick from source on the low-performance mac, using:

brew install imagemagick@6 --with-fontconfig --with-librsvg 
brew link --force imagemagick@6

So far, it appears that imagemagick@6 solved the issue. However, I do need to be able to do annotations, and brew didn't accept the formula to include fontconfig:

✘  ~  brew install imagemagick@6 --with-fontconfig --with-librsvg
Warning: imagemagick@6: this formula has no --with-fontconfig option so it will be ignored!
Warning: imagemagick@6: this formula has no --with-librsvg option so it will be ignored!

[jzadra]

Also, it is still generating lots of 1.1 gb files and isn't cleaning them up:

[jeroen]

Do they get cleaned when you run gc() ?

[jzadra]

gc() removed only one file.

[jzadra]

Can you recommend any way to get imagemagick@6 working with annotations, or alternatively another version of image magick that might both solve the problem and have font support?

[jzadra]

It looks like homebrew is no longer supporting formula for imagemagick:

"Usage of options has been discontinued in homebrew-core as it created a higher support burden than it provided benefits."

https://discourse.brew.sh/t/imagemagick-formula-not-working/4034/3

[jzadra]

I've removed magick and imagemagick, and reinstalled imagemagick using macports It put 6.9.9.40 on my machine. I then reinstalled magick from source and it links correctly to imagemagick and has the fontconfig and freetype features enabled (necessary for my project).

But, I stilll have the same problem. Generating tons of 1.1 GB files, sometimes produces a result, most of the time does not.

gc() does seem to collect all the recent files now.

I've attempted putting gc() in a loop where I'm compositing the graphics, but it seems to delete files that R/magick isn't actually done with and I get errors:
Error in magick_image_composite(image, composite_image, offset, operator, : Magick: unable to read pixel cache /var/folders/5_/l71sk6kn29z17n011g8kld5m0000gp/T//RtmpjJwyIA/magick-944373bVqoVNdIk1C': Undefined error: 0 @ error/cache.c/ReadPixelCachePixels/4890`

I've also attempted manually deleting the oldest temp files as new ones are being generated, and that seems to allow the process to work (still takes several minutes to do 12 composites).

I"m nearing panic mode here as I am dependent on r magick to be able to complete this project which was due to the client a few days ago.

My thoughts - is there a better way to delete the older temp files (without deleting the ones actually needed)?

Is there are reason that I"m getting 1.1 GB temp files generated for a PNG that ends up only being < 1 MB?

[jeroen]

@jzadra did you try it with imagemagick7? Just use

brew remove imagemagick@6
brew install imagemagick

And then install.packages("magick", type = "source")

@dlemstra do you have any guess why these huge temporary files are created?

[jzadra]

Yes, I did try with imagemagick 7. Thanks, Jon

…

-- Jonathan Zadra, PhD Director of Data Science Sorenson Impact Center David Eccles School of Business, University of Utah (801) 581-4815 jonathan.zadra@sorensonimpact.com www.sorensonimpact.com

On Feb 15, 2019, 5:46 AM -0700, Jeroen Ooms ***@***.***>, wrote: @jzadra did you try it with imagemagick7? Just use brew install imagemagick And then install.packages("magick", type = "source") @dlemstra do you have any guess why these huge temporary files are created? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

--

[dlemstra]

@jeroen Those temporary files are created when there is not enough memory for the pixels. It will swap to disk instead then.

[jzadra]

@dlemstra the problem is that sometimes they don't seem to get cleaned up. I will have finished an image magick operation and the files are still there.

[jeroen]

Tracking the tempfile / gc issue here: #194

If other problems persist, please open a new issue (specific for each problem).