Inconsistent underscore behavior in IPython

[reckoner]

Python 3.6.4 |Anaconda, Inc.| (default, Jan 16 2018, 18:10:19)
Type 'copyright', 'credits' or 'license' for more information
IPython 6.5.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: _,a=1,10

In [2]: _
Out[2]: 1

In [3]: 1/3
Out[3]: 0.3333333333333333

In [4]: _
Out[4]: 0.3333333333333333




In [5]: _,a = 1,10

In [6]: _
Out[6]: 1

In [7]: 1/3
Out[7]: 0.3333333333333333

In [8]: _
Out[8]: 1

Is there some reason that Out[8] should not match Out[4]?

I originally thought this was a Jupyter Notebook issue (jupyter/notebook#3943) but now I think it is an IPython Issue.

[minrk]

There is a bug here, and it's the behavior in the first example. IPython only intends to create the _ variable if you haven't defined it. So if you do _, a = 1, 10, that disables the _ as an alias for Out[max(Out.keys())] because your code has explicitly defined what _ should mean. This is the behavior you are seeing later on in your session. There appears to be a bug where we don't trigger this "don't override user-defined _" logic correctly if your assignment to _ happens in the first input.

[Carreau]

it's slightly more tricky as we need rotate _ -> __ -> ___ as well, and we do identity check internally to know wether our internal tracking should update the user namespace.

The problem is that the 1 from In[1]: _, a= 1,0 and the 1 from In[2]: _ are inherently the same object (in this case), but also because of interning.

So the internal logic really believe that _ is the previous result and thus should can be overwritten. You can check that interning works as well.

In [1]: _,a = 1,2
In [2]: 1
Out[2]: 1

In [3]: 1/2
Out[3]: 0.5

In [4]: _
Out[4]: 0.5

It looks like the second time it does the right thing, but in fact, it does not update _ because __ or ___ do not match.

That is going to be really tricky to fix.

[wstomv]

... the _ as an alias for Out[max(Out.keys())] ...

As a word of warning (for those, like me, struggling with this issue), the suggestion that underscore is an alias of Out[max(Out.keys())] is not completely true.

The safer way to get the result of the most recent execution that yielded a result is Out.get(max(Out.keys(), default=0)).

Note that Out could be empty. In that case, we need to tweak max with a default value. And then indexing with [...] doesn't work; so, we need to use get(...) (which returns None when the key is not present).

Another note: This does not get you the result (if any) of the most recent execution as such, because executions that did not yield a result (such as print(...)) do not appear in Out and are 'skipped' by underscore.

[jcrist]

We just ran into this same issue with ibis. ibis defines a magical _ variable that users can use to build up expressions. If the user imports _ in the first cell, it's overwritten in later cells. But if they import it after a most-recent-result _ already exists, then everything works fine:

In [1]: from ibis import _

In [2]: _
Out[2]: <ibis.expr.deferred.Deferred at 0x7f3415de6b30>

In [3]: 1
Out[3]: 1

In [4]: _  # overwritten!
Out[4]: 1

In [1]: 1  # run a cell to pre-populate `_`
Out[1]: 1

In [2]: from ibis import _

In [3]: _
Out[3]: <ibis.expr.deferred.Deferred at 0x7fd7409caec0>

In [4]: 1
Out[4]: 1

In [5]: _  # not overwritten
Out[5]: <ibis.expr.deferred.Deferred at 0x7fd7409caec0>