Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

%%time magic prevents assignment #11659

Closed
dwhswenson opened this issue Mar 22, 2019 · 17 comments · Fixed by #11698
Closed

%%time magic prevents assignment #11659

dwhswenson opened this issue Mar 22, 2019 · 17 comments · Fixed by #11698
Milestone

Comments

@dwhswenson
Copy link

The %%time cell magic is preventing me from binding names. I found this in notebook code on Travis (py 3.6) and reproduced it locally (py 3.7) in notebook and in ipython at the terminal. It doesn't seem to apply to all cell magics:

Python 3.7.2 | packaged by conda-forge | (default, Mar 19 2019, 20:46:22)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.4.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: %%time
   ...: foo = "bar"
   ...:
   ...:
CPU times: user 3 µs, sys: 0 ns, total: 3 µs
Wall time: 6.91 µs

In [2]: foo
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-2-f1d2d2f924e9> in <module>
----> 1 foo

NameError: name 'foo' is not defined

In [3]: %%prun
   ...: baz = "qux"
   ...:
   ...:
         3 function calls in 0.000 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    0.000    0.000 {built-in method builtins.exec}
        1    0.000    0.000    0.000    0.000 <string>:2(<module>)
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}

In [4]: baz
Out[4]: 'qux'

Confirmed that 7.3.0 worked fine locally.

@dwhswenson
Copy link
Author

Looking through the PRs merged as part of 7.4.0, my best guess is this is an unexpected consequence of #11542 -- at least, I can see how this (and perhaps #11665, if pos existed as a list before the cell ran) would be caused by changing how scope is handled in cell magics. And I don't see anything else that looks like a likely cause.

But I don't know anything about the internals of magics: @Carreau, could this be the problem?

The workaround is just to stay on 7.3.0.

@Carreau
Copy link
Member

Carreau commented Mar 26, 2019

You are right that this is due to #11542 , I'll see if I can come up with a fix.

@cdeil
Copy link

cdeil commented Apr 1, 2019

%%time is used a lot, this affects many users & notebooks.
@Carreau or anyone - is it possible to do a patch release for this soon?
(for now I'll go back to v7.3 and tell others to do the same)

@wanick
Copy link

wanick commented Apr 2, 2019

"%time" and "%%time" behave differently
изображение

изображение

изображение

but run_call_magic && run_line_magic are identical
изображение

It is not clear how it ever worked before

@wanick
Copy link

wanick commented Apr 2, 2019

изображение

By the way, in "%%timeit" the situation is exactly the same, but at least it behaves predictably

изображение

@dwhswenson
Copy link
Author

@wanick : I believe the %%timeit behavior was the same before 7.4.0 (and made sense to me: timeit does multiple runs -- which one should be the result?) Of course, there's no such ambiguity with %%time.

@Carreau
Copy link
Member

Carreau commented Apr 2, 2019 via email

@wanick
Copy link

wanick commented Apr 2, 2019

@dwhswenson , True... I meant the same decorators and difference behavior

@Carreau, I solved this problem like this.

изображение

If this is a solution, I can make a pull request

@filannim
Copy link

Thank you guys. It all makes sense to me, I'm just wondering whether I need to update/edit all my Jupyter notebook now, or you're planning to restore the expected/canonical behaviour.

Thanks!

@Carreau
Copy link
Member

Carreau commented Apr 10, 2019

We're planning to fix it as soon as we have time; it will likely make it to the 7.5 release that I'll try to make before end of month.

@Carreau Carreau added this to the 7.5 milestone Apr 10, 2019
Carreau added a commit to Carreau/ipython that referenced this issue Apr 23, 2019
There was some side effect in some magic to allow passing
needs_local_scope. We know pass the global_scope to cell magice when
using @needs_local_scope, but at the same time this means that we can't
rely on the existance of local_ns from within the magic to know wheter
we are cell-called or line called.

Fixes ipython#11659
@Carreau
Copy link
Member

Carreau commented Apr 23, 2019

I should stop using "We", I'm practically the only developer reviewing and managing IPython itself.
This should have been fixed in a patch release but I only got time today and we're close to a month since last release so this is going to be in the next minor (7.5) if the fix is correct.

Thus I would appreciate review and testing of #11698, and once merged I'll start the process to do a 7.5 for before PyCon tutorials.

JamesSample added a commit to NIVANorge/Mobius that referenced this issue Apr 23, 2019
Due to ipython/ipython#11659. When fixed, can remove iPython from the `'conda install'` command.
@Carreau
Copy link
Member

Carreau commented Apr 23, 2019

Thanks @minrk, BTW the above was not a critic toward you, and you are supposed to be on leave, so go enjoy your sleepless nights :-)

@davidmakovoz
Copy link

davidmakovoz commented Jul 9, 2019

I'm experiencing the same issue. I just finished a fresh install of Anaconda on windows and discovered this problem. However it only occurs with my base environment:
conda version : 4.7.5
conda-build version : 3.18.5
python version : 3.7.3.final.0

I created (for a different reason) an environment with python 3.6 and python 3.7 (conda create -n testspacy37 python=3.x) and %%time works there

@schneiderfelipe
Copy link

Hi,

Should %%time and %%timeit behave differently with respect to variable assignments?

Basically, I can't access variables defined within a %%timeit block, but within a %%time block it works fine:

Python 3.6.8 (default, Jan 14 2019, 11:02:34) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.7.0 -- An enhanced Interactive Python. Type '?' for help.
In [1]: import numpy as np
In [2]: size = 20
In [3]: D = np.random.rand(size, size)
In [4]: I = np.random.rand(size, size, size, size)
In [5]: %%timeit
   ...: Gloop = np.zeros((size, size))
   ...: for p in range(size):
   ...:     for q in range(size):
   ...:         for r in range(size):
   ...:             for s in range(size):
   ...:                 Gloop[p, q] += (2 * I[p, q, r, s] - I[p, r, q, s]) * D[r, s]
   ...:
229 ms ± 2.44 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
In [6]: Gloop
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-6-b2508f9fc33d> in <module>
----> 1 Gloop

NameError: name 'Gloop' is not defined

In [7]: %%time
   ...: Gloop = np.zeros((size, size))
   ...: for p in range(size):
   ...:     for q in range(size):
   ...:         for r in range(size):
   ...:             for s in range(size):
   ...:                 Gloop[p, q] += (2 * I[p, q, r, s] - I[p, r, q, s]) * D[r, s]
   ...:
CPU times: user 264 ms, sys: 0 ns, total: 264 ms
Wall time: 263 ms

In [8]: Gloop
Out[8]:
array([[107.84826652,  87.35710355,  97.97752913,  86.50282191,
           ...
        101.92858435, 108.97229643, 103.99891131, 107.81191102]])

Does this have to do with the present issue?
By the way, I'm using Ubuntu 18.04.3 LTS:

% uname -a
Linux mothership 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
% python3 --version
Python 3.6.8
% pip3 --version
pip 9.0.1 from /usr/lib/python3/dist-packages (python 3.6)
% ipython3 --version
7.7.0

@Carreau
Copy link
Member

Carreau commented Sep 8, 2019

Making both work the same is likely tricky, %%timeit is a wrapper around the Python timeit module; andone of the issues is that it runs things many times. So a global statement like n += 1 will have weird effects.

We actually do not run things in global scope, but create an anonymous function and run this function. So in your example, Gloop is not really globlal.

I'm pretty sure starting your timeit with global Gloop should work around the issue. It's just quite hard.

If you findtime to send a pull-request to fix it I'll be happy to try to review it; I unfortunately won't have the time myself.

@zoyasaeed-piaic
Copy link

@wanick you explained %time %%time really well thanks for that.
i have a query why my wall time = 0
I tried your example as well but same wall time 0
can you please guide what I am doing wrong.
Annotation 2020-04-12 210440

@lkl2050
Copy link

lkl2050 commented Nov 27, 2021

same error on M1 Mac

Python 3.8.12
conda 4.11.0
ipython3 7.30.0

Funny thing is this issue actually ruined the notebook entirely. Even after I deleted the %%time command in the cell, the global variables were still removed after running the new cell

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

9 participants