Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

A mistake when i try to process a big number of images,about resource destory #361

Closed
cdEthan opened this issue Jan 29, 2018 · 5 comments
Closed
Milestone

Comments

@cdEthan
Copy link

@cdEthan cdEthan commented Jan 29, 2018

I try to process PDF to JPG,i find many file which name like'magick-xxxxxxxx' in '/local/tmp'
i think is destory() didn't work
my code:
with(Image(filename=file, resolution=300)) as img:
images = img.sequence
pages = len(images)
for i in range(pages):
images[i].type = 'truecolor'
save_name = save_dir + file_name_prefix +'_'+ str(i+1) + '.jpg'
img.alpha_channel = False
Image(images[i]).save(filename=save_name)

in resourse.py,said that the resourse will destory automatic, when project start, file 'magick-xxxxxx' desory correctly,but after process some image,it didnt destory,and it is coincidental which 'magick-xxxxxx' file is 33984KB

4 f74g08a 947yot3d 60dw

@SylvainBert
Copy link

@SylvainBert SylvainBert commented Mar 30, 2018

Hi,
I had the same problem but I don't find a proper solution for that.
So, i add in my code:

cmd = 'sudo rm /../../tmp/magick-*'
arg = shlex.split(cmd)
arg = arg[:-1] + glob.glob(arg[-1])
p = subprocess.Popen(arg)

Like this, after each file i delete magick-xxxx to avoid problem of memory!

@cdEthan
Copy link
Author

@cdEthan cdEthan commented May 23, 2018

but when code running,can't delete this temp files,system remind that there another process are use them.
i was work on win 10 ,python3.6

@emcconville
Copy link
Owner

@emcconville emcconville commented Nov 19, 2018

@cdEthan Are you able to test this again with the latest master branch? The temporary resources should be cleared out on MagckTerminus event at the end of the execution routine.

@emcconville emcconville added this to the Wand 0.6.0 milestone Dec 19, 2019
@jseuribe
Copy link

@jseuribe jseuribe commented Dec 23, 2019

Just to compound on this, it looks like this is still an issue.
Using:

  • python 3.6.8,
  • Wand 0.5.8.
  • ImageMagick 6.9.7-4
  • Ubuntu 18.04.2

Here's a code snippet and the Traceback. I didn't see any of the other folk here post tracebacks, so maybe this will help?

Unfortunately the only alternative for me, for now, is to use subprocess to call the convert command on the target file.

Also worth noting that if I use with image.clone() as img the cache resource issue still causes the program to terminate early.

from wand.image import Image

pdf="1003rev.pdf"

fmt = 'png'

for x in range(1,500):
  print("iter:",x)
  with open(pdf, 'rb') as f:
    images = Image(blob=f.read(), resolution=300)

    for i,image in enumerate(images.sequence):
      img = image.clone()
      img.format = fmt
      img.alpha_channel = 'remove'
      img.compression_quality = 100
      img.save(filename=f"{pdf}-{i}.{fmt}")
      img.destroy()
      image.destroy()
  print("FINISHED ITER:", x)

Exception thrown:

  File "wand-test.py", line 10, in <module>
    images = Image(blob=f.read(), resolution=300)
  File "/home/jose/.local/lib/python3.6/site-packages/wand/image.py", line 8227, in __init__
    self.read(blob=blob, resolution=resolution, units=units)
  File "/home/jose/.local/lib/python3.6/site-packages/wand/image.py", line 8710, in read
    self.raise_exception()
  File "/home/jose/.local/lib/python3.6/site-packages/wand/resource.py", line 243, in raise_exception
    raise e
wand.exceptions.CacheError: cache resources exhausted `/tmp/magick-20150-k-iR69JUVXN1' @ error/cache.c/OpenPixelCache/3984```

@emcconville emcconville mentioned this issue Mar 1, 2020
@emcconville
Copy link
Owner

@emcconville emcconville commented May 14, 2020

Closing this out. The 0.6.0 release should help with this as the clean--up routines will be called during Python's shutdown/unload pipeline. However, magick-xxxxx can still be left on the system if ...

  1. ImageMagick is unable to unlink cache files due to the file being open ( common on windows )
  2. Script doesn't shutdown cleanly ( uncaught error, or worse, an error caught but ignored )

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants