Suggestion: Enable in script

[rahulraj80]

Hi @stas00 :

This works wonderfully in Jupyter/Notebook environment.

Can we enable this (take snapshot of variables at T1, log changes in variables, and cleanup/finish on demand at T2) in scripts as well?

This will be useful in long running scripts with user interactions.

Currently, the code fails (by design, not bug) as the ipython.kernel is null when the code runs in script.

Would be happy to test if something like this gets enabled.

[stas00]

Do you mean you would like to use it as a context manager? in jupyter each cell is a context plus the whole notebook is another context. So in the script you'd have to define your own context. Is that what you mean?

with IpyExperimentsCtx():
    some code 

How do you suggest to keep track of variables without ipython's built-in functionality for that?

[rahulraj80]

Context manager based best practice is definitely there for memory management, but possibly one can create that code structure without IpyE as well.

I was actually thinking independent of context and on the lines of taking snapshot of the variables at the start of the experiment, and then comparing it later, when the user asks for freeing up memory, or if there is an imminent OOM - something on the lines of the finish() code :

var_names_new = list(set(var_names_cur) - set(self.var_names_start) - set(self.var_names_keep)) 

From my rudimentary understanding of the code, it seems the cell logger is doing the heavy lifting right now. Can it use the script itself as context if ipython.kernel is None? Or maybe there is a more elegant solution.

[rahulraj80]

Flow in my mind:

==== script.py =====

import ipyexperiments as ipye
<other imports>

some_variables = ....

exp1 = ipye.IPyExperimentsXXX()  # Take a snapshot of current variables (track modification) 

new_variables = ....
<other stuff>

<user interactions>
        <if asked to free>
               - finish() experiments, 
                         - List variables modified + New variables
                         - capture list of variables deleted for use in round #2 (if needed) 
                         - share reclaim stats
               - If asked for, re-initiate experiment. 
                         - exp1 = ipye.IPyExperimentsXXX()  # Take an additional snapshot of variables
                          - Keep track of Phase #1 variables, in case it helps speed up comparison later.

[stas00]

• IPyExperiments is the main container that provides the experiment management.
• CellLogger currently does 2 things - managing the context for each cell and in addition to ipyexperiments provides the peak memory measurements functionality. I would like to get rid of threads - and then all the measurements functions could be extracted into its own class - not sure when pytorch will get around implementing a proper max memory functions. So currently, it's a bit yucky.

One immediate issue I see with your suggestion - is that it'd be tricky to find the right stack frame to get the local vars from. Here is a little proof of concept:

import inspect

def get_locals(levels=1):
    frame = inspect.currentframe()
    for l in range(levels): frame = frame.f_back
    return [x for x in frame.f_locals.keys() if not x.startswith('_')]

class Test:

    def __init__(self):
        self.before = get_locals(2)
        self.after  = []

    def diff(self):
        self.after = get_locals(2)
        return sorted(list(set(self.after)-set(self.before)))

t = Test()
xyz = 1
print(t.diff()) # prints ['t', 'xyz']

This works since we have a fixed number of frames. but call the same last few lines from within another function, and we need to figure out to move yet another frame back, and so on. I suppose we could always assume to go to the top level, where it has no more f_back defined, but that would be very hardwiring and might not give what people want - if they are after locals in whatever frame they decided to invoke the experiment from. I guess we could also measure the distance (number of frames inside the creation of the object) and invoke that... not sure.

in the case of ipython it manages the user variable namespace, regardless of where you are calling from.

I also remember encountering all kinds of problems with trying to track the variable change and remember giving up on it as the behavior wasn't consistent depending on situations.

But otherwise if you want to give it a try:

1. you can replace the ipython code in ipyexperiments.py with something based on the above code - i.e. you need a new get_var_names based on inspect as in the sample code above.
2. And then in cell_logger.py you need to replace the callback registration inside the start/stop functions with a context manager creation and destruction - or something similar. but you can also ignore it if you don't care for peak memory usage, that's currently only done in CellLogger.
3. of course, on the way remove any imports of IPython code to decouple from that environment - which is what you will be doing in items 1 and 2 above.

See if it satisfies your needs.

[rahulraj80]

Thanks for the ideas. Let me see if I can edit the code sufficiently.

Thanks again.