Memory usage does not work with class methods?

[rominf]

I want to profile time and memory usage of class method.
When I try to use partial from functools I got this error:

File "/usr/lib/python2.7/site-packages/memory_profiler.py", line 126, in memory_usage
  aspec = inspect.getargspec(f)
File "/usr/lib64/python2.7/inspect.py", line 815, in getargspec
  raise TypeError('{!r} is not a Python function'.format(func))
TypeError: <functools.partial object at 0x252da48> is not a Python function

By the way exactly the same approach works fine with timeit function.

When I try to use lambda as was I got this error:

File "/usr/lib/python2.7/site-packages/memory_profiler.py", line 141, in memory_usage
  ret = parent_conn.recv()
IOError: [Errno 4] Interrupted system call

How can I handle class methods with memory_profiler? Are there any (even dirty) ways?

I asked this question on SO: http://stackoverflow.com/questions/16593246/how-to-use-memory-profiler-python-module-with-class-methods

UPD: fixed broken link to SO

[fabianp]

Thanks for the report, that's indeed a nasty bug.

[rominf]

Thank you for your answer!

Offtopic:
Excuse me for an offtopic question. I'm a student. Deadline for passing laboratory work in data structures and algorithms (binary search trees, balanced (AVL) trees, hash) is tomorrow evening and I finished this work completely excluding memory usage measuring.
Are there dirty way to fix this? Or do you know another way to profile methods (I've tried heapy, but it really complicated, I gave up)? I just need maximum memory usage, i.e.:

memory_list = memory_usage((func, (), {}))
memory = max(memory_list) - min(memory_list)

I thought about measuring memory with console API and decorators, parsing output, ... But that is silly.
Excuse me once again for inappropriate question.

[sjagoe]

@rominf A quick and probably dirty way to do what you want.

First install psutil.

This assumes that your function does not return anything (as in your example). Adjust as necessary. The function below takes a callable (the fn argument), and any args and keyword args to pass to it, and will return the change in resident set size, and change in virtual memory size, in bytes.

def call_with_memory(fn, *args, **kwargs):
    process = psutil.Process(os.getpid())
    start_rss, start_vms = process.get_memory_info()
    fn(*args, **kwargs)
    end_rss, end_vms = process.get_memory_info()
    return (end_rss - start_rss), (end_vms - start_vms)

[rominf]

@sjagoe Thank you for suggestion. Unfortunately it doesn't work as I want: it returns (0, 0) all the time :(
For example if I have:

def f():
    s = 'a'*10000

it returns (0, 0), not something like (in C style) sizeof(char)*10000 = 10000 bytes
I've tried to disable garbage collector, use global var with global keyword, measure end_rss, end_vms inside of fn, it doesn't work either. It worked only when I created lists inside fn, but that's not enough.

[sjagoe]

@rominf This discussion no longer belongs here. You can contact me by email (see my profile).

[sjagoe]

@fabianp Apologies for allowing this issue-hijacking for going on so long. I have moved my information to the SO thread.

I would suggest deleting the off topic comments here given that the same content is in the linked SO thread.

[fabianp]

@sjagoe no problem, I'm just sorry I don't have time to look into this at the moment.

[astrojuanlu]

This is missing a SSCCE.