Massive memory usage when using MSS with queues

[edelmanjm]

General informations:

• OS name: macOS
• OS version: 10.12.5 "Sierra"
• OS architecture: 64 bits
• Result of the command python --version: Python 2.7.10, however python3 --version yields Python 3.6.2 (which is what was used.)

For GNU/Linux users:

• Desktop Environment: Quartz

Description of the warning/error

Putting and removing images captured using MSS in a queue results in a massive memory usage. This example code resulted in about 6GB used in less that 15 seconds.

from mss import mss
import time
import queue

myqueue = queue.Queue()

while True:
    with mss() as sct:
        myqueue.put(sct.shot())
    time.sleep(0.25)
    print(myqueue.get())

Full message

N/A

Other details

Willing to provide more info as needed!

[BoboTiG]

Wow, it seems huge! Could you try with the dev branch to see if recent changes affect this case?

git checkout dev
python -m pip install -e .

Thanks for the report.

[BoboTiG]

I fixed the memory leak in the dev branch. Could you confirm it solves the issue? From my side, I went from +700 Mo to constant 60 Mo. The minimum memory needed seems to be between 50 and 65 Mo. I tried with 1000 screenshots and it works fine now.

Another advice, you can setup MSS outside the while loop:

with mss() as sct:
    while True:
        myqueue.put(sct.shot())
        time.sleep(0.25)
        print(myqueue.get())

[BoboTiG]

Some results after the fix.

GNU/Linux:

Line #    Mem usage    Increment   Line Contents
================================================
11     14.2 MiB      0.0 MiB   @profile
12                             def test():
13     14.2 MiB      0.0 MiB       myqueue = queue.Queue()
14     18.9 MiB      4.7 MiB       for _ in range(100):
15     18.9 MiB      0.0 MiB           with mss() as sct:
16     18.9 MiB      0.0 MiB               myqueue.put(sct.shot())
17     18.9 MiB      0.0 MiB           time.sleep(0.25)
18     18.9 MiB      0.0 MiB           print(myqueue.get())

macOS:

Line #    Mem usage    Increment   Line Contents
================================================
11     27.5 MiB      0.0 MiB   @profile
12                             def test():
13     27.5 MiB      0.0 MiB       myqueue = queue.Queue()
14     56.4 MiB     28.9 MiB       for _ in range(100):
15     56.4 MiB      0.0 MiB           with mss() as sct:
16     56.4 MiB      0.0 MiB               myqueue.put(sct.shot())
17     56.4 MiB      0.0 MiB           time.sleep(0.25)
18     56.4 MiB      0.0 MiB           print(myqueue.get())

Windows:

Line #    Mem usage    Increment   Line Contents
================================================
11     10.5 MiB      0.0 MiB   @profile
12                             def test():
13     10.5 MiB      0.0 MiB       myqueue = queue.Queue()
14     13.9 MiB      3.4 MiB       for _ in range(100):
15     13.9 MiB      0.0 MiB           with mss() as sct:
16     13.9 MiB      0.0 MiB               myqueue.put(sct.shot())
17     13.9 MiB      0.0 MiB           time.sleep(0.25)
18     13.9 MiB      0.0 MiB           print(myqueue.get())

[edelmanjm]

I used sudo pip3 install git+git://github.com/BoboTiG/python-mss.git@dev to install MSS, hopefully that's fine.

I'm getting the following error:

Traceback (most recent call last):
  File "/Users/edelmanjm/Desktop/msstest.py", line 9, in <module>
    myqueue.put(sct.shot())
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/mss/base.py", line 124, in shot
    return next(self.save(**kwargs))
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/mss/base.py", line 114, in save
    to_png(sct.rgb, sct.size, output)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/mss/screenshot.py", line 133, in rgb
    self.raw[2::4], self.raw[1::4], self.raw[0::4]
ValueError: attempt to assign bytes of size 7056000 to extended slice of size 1764000

[BoboTiG]

The module installation is fine :)
Could you retry with the last update?

[edelmanjm]

My main Mac is out right now, let me get back to you in a few days.

[BoboTiG]

Hello, any news? :)

[edelmanjm]

Still not working :/

Traceback (most recent call last):
  File "msstest2.py", line 9, in <module>
    myqueue.put(sct.shot())
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/mss/base.py", line 127, in shot
    return next(self.save(**kwargs))
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/mss/base.py", line 117, in save
    to_png(sct.rgb, sct.size, output)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/mss/screenshot.py", line 135, in rgb
    self.raw[2::4], self.raw[1::4], self.raw[0::4]
ValueError: attempt to assign bytes of size 7056000 to extended slice of size 1764000

[BoboTiG]

OK, one time again, could you retry with the up-to-date devbranch? Sorry for all those tries, I think we are going to find the good fix :)

[edelmanjm]

Nope :(

Traceback (most recent call last):
  File "msstest.py", line 9, in <module>
    myqueue.put(sct.shot())
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/mss/base.py", line 127, in shot
    return next(self.save(**kwargs))
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/mss/base.py", line 117, in save
    to_png(sct.rgb, sct.size, output)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/mss/screenshot.py", line 135, in rgb
    self.raw[2::4], self.raw[1::4], self.raw[0::4]
ValueError: attempt to assign bytes of size 7056000 to extended slice of size 1764000

[BoboTiG]

What is the content of sct.monitors ?

[edelmanjm]

[{'left': 0, 'top': 0, 'width': 1680, 'height': 1050}, {'left': 0, 'top': 0, 'width': 1680, 'height': 1050}]

...which totally isn't the size of my screen. It's 2880x1800.

[edelmanjm]

@BoboTiG sct.monitors is still reporting the wrong size.

[BoboTiG]

Could you open a new issue as it ils not related to this one? Specify if you are using scaling or whatever can help investigate and reproduce.