Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

%%time magic prevents assignment #11659

Closed
dwhswenson opened this issue Mar 22, 2019 · 17 comments
Closed

%%time magic prevents assignment #11659

dwhswenson opened this issue Mar 22, 2019 · 17 comments
Milestone

Comments

@dwhswenson
Copy link

@dwhswenson dwhswenson commented Mar 22, 2019

The %%time cell magic is preventing me from binding names. I found this in notebook code on Travis (py 3.6) and reproduced it locally (py 3.7) in notebook and in ipython at the terminal. It doesn't seem to apply to all cell magics:

Python 3.7.2 | packaged by conda-forge | (default, Mar 19 2019, 20:46:22)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.4.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: %%time
   ...: foo = "bar"
   ...:
   ...:
CPU times: user 3 µs, sys: 0 ns, total: 3 µs
Wall time: 6.91 µs

In [2]: foo
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-2-f1d2d2f924e9> in <module>
----> 1 foo

NameError: name 'foo' is not defined

In [3]: %%prun
   ...: baz = "qux"
   ...:
   ...:
         3 function calls in 0.000 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    0.000    0.000 {built-in method builtins.exec}
        1    0.000    0.000    0.000    0.000 <string>:2(<module>)
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}

In [4]: baz
Out[4]: 'qux'

Confirmed that 7.3.0 worked fine locally.

@dwhswenson
Copy link
Author

@dwhswenson dwhswenson commented Mar 26, 2019

Looking through the PRs merged as part of 7.4.0, my best guess is this is an unexpected consequence of #11542 -- at least, I can see how this (and perhaps #11665, if pos existed as a list before the cell ran) would be caused by changing how scope is handled in cell magics. And I don't see anything else that looks like a likely cause.

But I don't know anything about the internals of magics: @Carreau, could this be the problem?

The workaround is just to stay on 7.3.0.

Loading

@Carreau
Copy link
Member

@Carreau Carreau commented Mar 26, 2019

You are right that this is due to #11542 , I'll see if I can come up with a fix.

Loading

@cdeil
Copy link

@cdeil cdeil commented Apr 1, 2019

%%time is used a lot, this affects many users & notebooks.
@Carreau or anyone - is it possible to do a patch release for this soon?
(for now I'll go back to v7.3 and tell others to do the same)

Loading

@wanick
Copy link

@wanick wanick commented Apr 2, 2019

"%time" and "%%time" behave differently
изображение

изображение

изображение

but run_call_magic && run_line_magic are identical
изображение

It is not clear how it ever worked before

Loading

@wanick
Copy link

@wanick wanick commented Apr 2, 2019

изображение

By the way, in "%%timeit" the situation is exactly the same, but at least it behaves predictably

изображение

Loading

@dwhswenson
Copy link
Author

@dwhswenson dwhswenson commented Apr 2, 2019

@wanick : I believe the %%timeit behavior was the same before 7.4.0 (and made sense to me: timeit does multiple runs -- which one should be the result?) Of course, there's no such ambiguity with %%time.

Loading

@Carreau
Copy link
Member

@Carreau Carreau commented Apr 2, 2019

Loading

@wanick
Copy link

@wanick wanick commented Apr 2, 2019

@dwhswenson , True... I meant the same decorators and difference behavior

@Carreau, I solved this problem like this.

изображение

If this is a solution, I can make a pull request

Loading

@filannim
Copy link

@filannim filannim commented Apr 10, 2019

Thank you guys. It all makes sense to me, I'm just wondering whether I need to update/edit all my Jupyter notebook now, or you're planning to restore the expected/canonical behaviour.

Thanks!

Loading

@Carreau
Copy link
Member

@Carreau Carreau commented Apr 10, 2019

We're planning to fix it as soon as we have time; it will likely make it to the 7.5 release that I'll try to make before end of month.

Loading

@Carreau Carreau added this to the 7.5 milestone Apr 10, 2019
Carreau added a commit to Carreau/ipython that referenced this issue Apr 23, 2019
There was some side effect in some magic to allow passing
needs_local_scope. We know pass the global_scope to cell magice when
using @needs_local_scope, but at the same time this means that we can't
rely on the existance of local_ns from within the magic to know wheter
we are cell-called or line called.

Fixes ipython#11659
@Carreau
Copy link
Member

@Carreau Carreau commented Apr 23, 2019

I should stop using "We", I'm practically the only developer reviewing and managing IPython itself.
This should have been fixed in a patch release but I only got time today and we're close to a month since last release so this is going to be in the next minor (7.5) if the fix is correct.

Thus I would appreciate review and testing of #11698, and once merged I'll start the process to do a 7.5 for before PyCon tutorials.

Loading

JamesSample added a commit to NIVANorge/Mobius that referenced this issue Apr 23, 2019
Due to ipython/ipython#11659. When fixed, can remove iPython from the `'conda install'` command.
@minrk minrk closed this in #11698 Apr 23, 2019
@Carreau
Copy link
Member

@Carreau Carreau commented Apr 23, 2019

Thanks @minrk, BTW the above was not a critic toward you, and you are supposed to be on leave, so go enjoy your sleepless nights :-)

Loading

@davidmakovoz
Copy link

@davidmakovoz davidmakovoz commented Jul 9, 2019

I'm experiencing the same issue. I just finished a fresh install of Anaconda on windows and discovered this problem. However it only occurs with my base environment:
conda version : 4.7.5
conda-build version : 3.18.5
python version : 3.7.3.final.0

I created (for a different reason) an environment with python 3.6 and python 3.7 (conda create -n testspacy37 python=3.x) and %%time works there

Loading

@schneiderfelipe
Copy link

@schneiderfelipe schneiderfelipe commented Sep 4, 2019

Hi,

Should %%time and %%timeit behave differently with respect to variable assignments?

Basically, I can't access variables defined within a %%timeit block, but within a %%time block it works fine:

Python 3.6.8 (default, Jan 14 2019, 11:02:34) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.7.0 -- An enhanced Interactive Python. Type '?' for help.
In [1]: import numpy as np
In [2]: size = 20
In [3]: D = np.random.rand(size, size)
In [4]: I = np.random.rand(size, size, size, size)
In [5]: %%timeit
   ...: Gloop = np.zeros((size, size))
   ...: for p in range(size):
   ...:     for q in range(size):
   ...:         for r in range(size):
   ...:             for s in range(size):
   ...:                 Gloop[p, q] += (2 * I[p, q, r, s] - I[p, r, q, s]) * D[r, s]
   ...:
229 ms ± 2.44 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
In [6]: Gloop
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-6-b2508f9fc33d> in <module>
----> 1 Gloop

NameError: name 'Gloop' is not defined

In [7]: %%time
   ...: Gloop = np.zeros((size, size))
   ...: for p in range(size):
   ...:     for q in range(size):
   ...:         for r in range(size):
   ...:             for s in range(size):
   ...:                 Gloop[p, q] += (2 * I[p, q, r, s] - I[p, r, q, s]) * D[r, s]
   ...:
CPU times: user 264 ms, sys: 0 ns, total: 264 ms
Wall time: 263 ms

In [8]: Gloop
Out[8]:
array([[107.84826652,  87.35710355,  97.97752913,  86.50282191,
           ...
        101.92858435, 108.97229643, 103.99891131, 107.81191102]])

Does this have to do with the present issue?
By the way, I'm using Ubuntu 18.04.3 LTS:

% uname -a
Linux mothership 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
% python3 --version
Python 3.6.8
% pip3 --version
pip 9.0.1 from /usr/lib/python3/dist-packages (python 3.6)
% ipython3 --version
7.7.0

Loading

@Carreau
Copy link
Member

@Carreau Carreau commented Sep 8, 2019

Making both work the same is likely tricky, %%timeit is a wrapper around the Python timeit module; andone of the issues is that it runs things many times. So a global statement like n += 1 will have weird effects.

We actually do not run things in global scope, but create an anonymous function and run this function. So in your example, Gloop is not really globlal.

I'm pretty sure starting your timeit with global Gloop should work around the issue. It's just quite hard.

If you findtime to send a pull-request to fix it I'll be happy to try to review it; I unfortunately won't have the time myself.

Loading

@zoyasaeed-piaic
Copy link

@zoyasaeed-piaic zoyasaeed-piaic commented Apr 12, 2020

@wanick you explained %time %%time really well thanks for that.
i have a query why my wall time = 0
I tried your example as well but same wall time 0
can you please guide what I am doing wrong.
Annotation 2020-04-12 210440

Loading

@lkl2050
Copy link

@lkl2050 lkl2050 commented Nov 27, 2021

same error on M1 Mac

Python 3.8.12
conda 4.11.0
ipython3 7.30.0

Funny thing is this issue actually ruined the notebook entirely. Even after I deleted the %%time command in the cell, the global variables were still removed after running the new cell

Loading

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

9 participants