Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JoblibCollisionWarning not shown in an interactive session #649

Open
lesteve opened this issue Mar 14, 2018 · 1 comment
Open

JoblibCollisionWarning not shown in an interactive session #649

lesteve opened this issue Mar 14, 2018 · 1 comment

Comments

@lesteve
Copy link
Member

lesteve commented Mar 14, 2018

Seen in #648, the doc describing the behaviour with JoblibCollisionWarning does not correspond to the observed behaviour in an IPython console or Jupyter notebook. It seems to be more or less fine if the code is in a python file.

I'll try to add more details soon.

@lesteve lesteve changed the title JoblibCollisionWarning old behaviour gone JoblibCollisionWarning problem in an interactive session Mar 14, 2018
@lesteve
Copy link
Member Author

lesteve commented Mar 15, 2018

# test-collision.py
from joblib import Memory

memory = Memory('/tmp/test')
memory.clear(warn=False)


def func(x):
    print('Running one(%s)' % x)
    return 1


one = memory.cache(func)


def func(x):
    print('Running two(%s)' % x)
    return 2


two = memory.cache(func)

print('first call')
print('-'*80)
print('result one:', one(1))
print('result two:', two(1))
print('second call')
print('-'*80)
print('result one:', one(1))
print('result two:', two(1))
$ ipython test-collision.py
first call
--------------------------------------------------------------------------------
________________________________________________________________________________
[Memory] Calling __main__--tmp-test-joblib.func...
func(1)
Running one(1)
_____________________________________________________________func - 0.0s, 0.0min
result one: 1
/tmp/test-joblib.py:25: JobLibCollisionWarning: Possible name collisions between functions 'func' (/tmp/test-joblib.py:7) and 'func' (/tmp/test-joblib.py:15)
  print('result two:', two(1))
WARNING:root:[MemorizedFunc(func=<function func at 0x7f8412fd6a60>, location=/tmp/test/joblib)]: Clearing function cache identified by __main__--tmp-test-joblib/func
________________________________________________________________________________
[Memory] Calling __main__--tmp-test-joblib.func...
func(1)
Running two(1)
_____________________________________________________________func - 0.0s, 0.0min
result two: 2
second call
--------------------------------------------------------------------------------
result one: 2
result two: 2

Copying and pasting the same snippet in a IPython session:

first call
--------------------------------------------------------------------------------
________________________________________________________________________________
[Memory] Calling __main__--home-lesteve-dev-joblib-__ipython-input__.func...
func(1)
Running one(1)
_____________________________________________________________func - 0.0s, 0.0min
result one: 1
WARNING:root:[MemorizedFunc(func=<function func at 0x7f5eb12a1048>, location=/tmp/test/joblib)]: Clearing function cache identified by __main__--home-lesteve-dev-joblib-__ipython-input__/func
________________________________________________________________________________
[Memory] Calling __main__--home-lesteve-dev-joblib-__ipython-input__.func...
func(1)
Running two(1)
_____________________________________________________________func - 0.0s, 0.0min
result two: 2
second call
--------------------------------------------------------------------------------
result one: 2
result two: 2

To sum up the difference is a JoblibCollisionWarning or a normal warning. Granted the former has a little bit more details explaining the problem but in the grand scheme of things this may not make a big difference.

To be honest I am not sure what the behaviour should be (cc @GaelVaroquaux who may have an opinion on this edge case). It is nice to have a warning but it looks like the calling one the second time will load the cached result from two which seems not so great.

@lesteve lesteve changed the title JoblibCollisionWarning problem in an interactive session JoblibCollisionWarning not shown in an interactive session Mar 15, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant