New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement update_display #10048

Merged
merged 7 commits into from Nov 18, 2016

Conversation

Projects
None yet
5 participants
@minrk
Member

minrk commented Nov 8, 2016

display(obj, display_id='xxx')

sets a display id.

update_display(obj, display_id='xxx')

updates the display in-place.

This is implemented through adding the transient dict to display-data, where display_id resides.

Still todo:

  • handle ipykernel not supporting the extra arguments (current stable)
  • handle-API for setting a display-id automatically and updating it
  • test exercise
  • example notebook
Add update_display
and display(display_id='...')

for updating display_data outputs in-place.
Show outdated Hide outdated IPython/core/display.py
"""
from IPython.core.interactiveshell import InteractiveShell
InteractiveShell.instance().display_pub.publish(
data=data,
metadata=metadata,
transient=transient,
**kwargs

This comment has been minimized.

@rgbkrk

rgbkrk Nov 8, 2016

Member

Leaving room for later I see. 😄

@rgbkrk

rgbkrk Nov 8, 2016

Member

Leaving room for later I see. 😄

@rgbkrk

rgbkrk approved these changes Nov 8, 2016

Show outdated Hide outdated IPython/core/display.py
metadata = kwargs.pop('metadata', None)
transient = kwargs.setdefault('transient', {})
if 'display_id' in kwargs:
transient['display_id'] = kwargs.pop('display_id')

This comment has been minimized.

@rgbkrk

rgbkrk Nov 8, 2016

Member

👍

@rgbkrk

rgbkrk Nov 8, 2016

Member

👍

@rgbkrk

rgbkrk approved these changes Nov 16, 2016

Show outdated Hide outdated IPython/core/display.py
def update_display(*objs, **kwargs):
"""Update an existing display"""
kwargs['update'] = True
return display(*objs, **kwargs)

This comment has been minimized.

@rgbkrk

rgbkrk Nov 16, 2016

Member

👍 I like that I can use display with update = True and not have to use update_display directly as a function. 😄 /cc @ivanov

@rgbkrk

rgbkrk Nov 16, 2016

Member

👍 I like that I can use display with update = True and not have to use update_display directly as a function. 😄 /cc @ivanov

minrk added some commits Nov 18, 2016

only pass transient if needed
avoids unnecessary errors with older ipykernel

passing display_id or transient will trigger TypeError: unsupported arg on ipykernel that doesn't support transient data
add DisplayHandle for updating displays
- display(display_id=True) generates new display_id
- display(display_id=anything) returns DisplayHandle
@minrk

This comment has been minimized.

Show comment
Hide comment
@minrk

minrk Nov 18, 2016

Member

Here's an example demonstrating this with current master of ipykernel, notebook now that those PRs have been merged.

This is my interpretation of proposals from @fperez and @jasongrout about creating a handle and auto-generating a display_id with a boolean flag (in this case, I used display_id=True rather than a separate track=True flag):

  • display(obj, display_id=True) generates a new display_id and returns a DisplayHandle
  • display(obj, display_id='given-id') returns a DisplayHandle with given display_id
  • display(obj) returns None
  • update_display(obj, display_id='id') display_id is required
  • DisplayHandle.display(obj) is equivalent to display(obj, display_id=handle.display_id)
  • DisplayHandle.update(obj) is equivalent to update_display(obj, display_id=handle.display_id)

So the quickest way to create and update a display without needing to generate an id yourself:

handle = display(x, display_id=True)
handle.update(y)
handle.update(z)

Possible functionality that is currently excluded:

  1. the ability for items to define and record their own display_id, such as:
    • display(x) being equivalent to display(x, display_id=x._ip_display_id_())
    • update_display(x) discovering display_id from x itself (same as above, really)
    • allowing ExecuteResults to have display_id (again, ultimately boils down to getting display_id from the object)
Member

minrk commented Nov 18, 2016

Here's an example demonstrating this with current master of ipykernel, notebook now that those PRs have been merged.

This is my interpretation of proposals from @fperez and @jasongrout about creating a handle and auto-generating a display_id with a boolean flag (in this case, I used display_id=True rather than a separate track=True flag):

  • display(obj, display_id=True) generates a new display_id and returns a DisplayHandle
  • display(obj, display_id='given-id') returns a DisplayHandle with given display_id
  • display(obj) returns None
  • update_display(obj, display_id='id') display_id is required
  • DisplayHandle.display(obj) is equivalent to display(obj, display_id=handle.display_id)
  • DisplayHandle.update(obj) is equivalent to update_display(obj, display_id=handle.display_id)

So the quickest way to create and update a display without needing to generate an id yourself:

handle = display(x, display_id=True)
handle.update(y)
handle.update(z)

Possible functionality that is currently excluded:

  1. the ability for items to define and record their own display_id, such as:
    • display(x) being equivalent to display(x, display_id=x._ip_display_id_())
    • update_display(x) discovering display_id from x itself (same as above, really)
    • allowing ExecuteResults to have display_id (again, ultimately boils down to getting display_id from the object)

minrk added some commits Nov 18, 2016

@minrk minrk changed the title from [WIP] Implement update_display to Implement update_display Nov 18, 2016

@minrk minrk added this to the 6.0 milestone Nov 18, 2016

Show outdated Hide outdated IPython/core/display.py
@@ -78,7 +79,7 @@ def _display_mimetype(mimetype, objs, raw=False, metadata=None):
# Main functions
#-----------------------------------------------------------------------------
def publish_display_data(data, metadata=None, source=None, transient=None, **kwargs):
def publish_display_data(data, metadata=None, source=None, *, transient=None, **kwargs):

This comment has been minimized.

@willingc

willingc Nov 18, 2016

Member

@minrk What does the * represent? Or typo?

@willingc

willingc Nov 18, 2016

Member

@minrk What does the * represent? Or typo?

Show outdated Hide outdated IPython/core/display.py
def update_display(obj, *, display_id, **kwargs):

This comment has been minimized.

@willingc

willingc Nov 18, 2016

Member

Would be good to explain * in docstring here and above. Thanks!

@willingc

willingc Nov 18, 2016

Member

Would be good to explain * in docstring here and above. Thanks!

This comment has been minimized.

@minrk

minrk Nov 18, 2016

Member

It is in the docstring, in that * is Python's syntax to denote the end of positional args and display_id is labeled as keyword-only in the docstring. What would you recommend for comments regarding adopting new Python syntax?

@minrk

minrk Nov 18, 2016

Member

It is in the docstring, in that * is Python's syntax to denote the end of positional args and display_id is labeled as keyword-only in the docstring. What would you recommend for comments regarding adopting new Python syntax?

This comment has been minimized.

@rgbkrk

rgbkrk Nov 18, 2016

Member

Wow, didn't realize that was available as a feature in Python. 😎

@rgbkrk

rgbkrk Nov 18, 2016

Member

Wow, didn't realize that was available as a feature in Python. 😎

This comment has been minimized.

@minrk

minrk Nov 18, 2016

Member

I added some comments on the methods to clarify these. Thanks!

@minrk

minrk Nov 18, 2016

Member

I added some comments on the methods to clarify these. Thanks!

@rgbkrk rgbkrk merged commit 94d6f5b into ipython:master Nov 18, 2016

3 checks passed

codecov/patch 96.59% of diff hit (target 0.00%)
Details
codecov/project 71.32% (+0.17%) compared to 18c42b1
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@minrk

This comment has been minimized.

Show comment
Hide comment
@minrk

minrk Nov 18, 2016

Member

For @Carreau, @takluyver: I made use of the fact that we are Python 3-only now by making some new args explicitly keyword-only but still in the signature. Hooray for progress!

We can refine/revise the APIs in subsequent PRs, but this one is complete enough for people to play with and build opinions about, I think. Thanks, @rgbkrk!

Member

minrk commented Nov 18, 2016

For @Carreau, @takluyver: I made use of the fact that we are Python 3-only now by making some new args explicitly keyword-only but still in the signature. Hooray for progress!

We can refine/revise the APIs in subsequent PRs, but this one is complete enough for people to play with and build opinions about, I think. Thanks, @rgbkrk!

@minrk minrk deleted the minrk:update-display branch Nov 18, 2016

@willingc

This comment has been minimized.

Show comment
Hide comment
@willingc

willingc Nov 18, 2016

Member

@minrk Thanks for the clarification re: '*' and arguments. I hadn't seen it used in the wild yet ;-)

Member

willingc commented Nov 18, 2016

@minrk Thanks for the clarification re: '*' and arguments. I hadn't seen it used in the wild yet ;-)

@Carreau

This comment has been minimized.

Show comment
Hide comment
@Carreau

Carreau Nov 18, 2016

Member

Sweet !

@willingc , you will also see / in docstrings:

In [2]: input?
Signature: input(prompt=None, /)
Docstring:
Read a string from standard input.  The trailing newline is stripped.

The prompt string, if given, is printed to standard output without a
trailing newline before reading input.

If the user hits EOF (*nix: Ctrl-D, Windows: Ctrl-Z+Return), raise EOFError.
On *nix systems, readline is used if available.

It means "end of positional only arguments". Above it means that prompt default to None, but that you can't say input(prompt='foo'), you have to use input('foo'). It's an even more rare beast than * :-)

Member

Carreau commented Nov 18, 2016

Sweet !

@willingc , you will also see / in docstrings:

In [2]: input?
Signature: input(prompt=None, /)
Docstring:
Read a string from standard input.  The trailing newline is stripped.

The prompt string, if given, is printed to standard output without a
trailing newline before reading input.

If the user hits EOF (*nix: Ctrl-D, Windows: Ctrl-Z+Return), raise EOFError.
On *nix systems, readline is used if available.

It means "end of positional only arguments". Above it means that prompt default to None, but that you can't say input(prompt='foo'), you have to use input('foo'). It's an even more rare beast than * :-)

@rgbkrk

This comment has been minimized.

Show comment
Hide comment
@rgbkrk

rgbkrk Nov 19, 2016

Member

Oh does this mean no update display in Python 2 for now? I misconstrued earlier sentiment that we'd keep doing a python 2 kernel for a bit and enforcing python 3 for services (notebook, hub).

Member

rgbkrk commented Nov 19, 2016

Oh does this mean no update display in Python 2 for now? I misconstrued earlier sentiment that we'd keep doing a python 2 kernel for a bit and enforcing python 3 for services (notebook, hub).

@takluyver

This comment has been minimized.

Show comment
Hide comment
@takluyver

takluyver Nov 19, 2016

Member

We're planning to keep supporting IPython 5 for Python 2 users for some time, but make IPython 6 require Python 3. As this is a new feature in IPython, Min has targeted it for IPython 6. We could maybe make a distinction between user-facing features and infrastructure features, and allow backporting some of the latter. I'd be -0.25 on doing that.

Separately, at some point the applications will require Python 3. Hub already does. For those, we don't plan to make an LTS version with Python 2 support as we have for IPython.

Member

takluyver commented Nov 19, 2016

We're planning to keep supporting IPython 5 for Python 2 users for some time, but make IPython 6 require Python 3. As this is a new feature in IPython, Min has targeted it for IPython 6. We could maybe make a distinction between user-facing features and infrastructure features, and allow backporting some of the latter. I'd be -0.25 on doing that.

Separately, at some point the applications will require Python 3. Hub already does. For those, we don't plan to make an LTS version with Python 2 support as we have for IPython.

@minrk

This comment has been minimized.

Show comment
Hide comment
@minrk

minrk Nov 21, 2016

Member

I'm okay targeting this for 5.x as well, if people have that preference. I just wanted to note that my experience was slightly improved by the fact that IPython 6 has dropped py2 support. The differences are small, so it wouldn't be a big deal to backport.

It is only ipython/ipython that's dropped py2 so far, so the ipykernel, notebook changes will still propagate to py2 users. A library that wants to support this in py2 could do ip.display_pub.publish(data=,metadata=,transient=).

Member

minrk commented Nov 21, 2016

I'm okay targeting this for 5.x as well, if people have that preference. I just wanted to note that my experience was slightly improved by the fact that IPython 6 has dropped py2 support. The differences are small, so it wouldn't be a big deal to backport.

It is only ipython/ipython that's dropped py2 so far, so the ipykernel, notebook changes will still propagate to py2 users. A library that wants to support this in py2 could do ip.display_pub.publish(data=,metadata=,transient=).

@rgbkrk

This comment has been minimized.

Show comment
Hide comment
@rgbkrk

rgbkrk Nov 21, 2016

Member

Ok, great. I'll use ip.display_pub.publish for now within the lib I'm working with that supports both.

Member

rgbkrk commented Nov 21, 2016

Ok, great. I'll use ip.display_pub.publish for now within the lib I'm working with that supports both.

@Carreau

This comment has been minimized.

Show comment
Hide comment
@Carreau

Carreau Nov 21, 2016

Member

I would be ok backporting. My Backporting bot that you can just @-mention is not ready yet though.

Member

Carreau commented Nov 21, 2016

I would be ok backporting. My Backporting bot that you can just @-mention is not ready yet though.

@Carreau

This comment has been minimized.

Show comment
Hide comment
@Carreau

Carreau Nov 21, 2016

Member

TIL @mention is a reserved name on github so you don't need to take extra-care when writing it.

Member

Carreau commented Nov 21, 2016

TIL @mention is a reserved name on github so you don't need to take extra-care when writing it.

@rgbkrk

This comment has been minimized.

Show comment
Hide comment
@rgbkrk

rgbkrk May 4, 2017

Member

@Carreau would you be willing to backport this one? It's the feature I want to use and support across PySpark installations for updating job statuses.

Member

rgbkrk commented May 4, 2017

@Carreau would you be willing to backport this one? It's the feature I want to use and support across PySpark installations for updating job statuses.

@Carreau Carreau modified the milestones: 5.4, 6.0 May 4, 2017

@Carreau

This comment has been minimized.

Show comment
Hide comment
@Carreau

Carreau May 4, 2017

Member

I think that is reasonable enough to make API used by many package like TQDM consistant between stable version. Let's see if it applies cleanly to 5.x

@meeseeksdev backport

Member

Carreau commented May 4, 2017

I think that is reasonable enough to make API used by many package like TQDM consistant between stable version. Let's see if it applies cleanly to 5.x

@meeseeksdev backport

@meeseeksdev

This comment has been minimized.

Show comment
Hide comment
@meeseeksdev

meeseeksdev bot May 4, 2017

Oops, something went wrong applying the patch... Please have a look at my logs.

meeseeksdev bot commented May 4, 2017

Oops, something went wrong applying the patch... Please have a look at my logs.

@Carreau

This comment has been minimized.

Show comment
Hide comment
@Carreau

Carreau May 4, 2017

Member

Ok, may need manual backport.

Member

Carreau commented May 4, 2017

Ok, may need manual backport.

Carreau added a commit to Carreau/ipython that referenced this pull request May 5, 2017

Backport PR #10048 on branch 5.x
Implement update_display (94d6f5b)

Carreau added a commit that referenced this pull request May 10, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment