Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add 'id' to Audio widget's HTML for javascript interaction #11670

Merged
merged 3 commits into from Apr 22, 2019

Conversation

@NumesSanguis
Copy link

commented Apr 3, 2019

Problem: At present, the Audio widget can play a .wav, numpy data among others. However, once initialized, we cannot interact with this Widget through code. This is a problem when creating e.g. an interactive audio dashboard.

Solution: By adding an id to the HTML code, we can grab the Audio widget through JavaScript.

With this pull request, you can do the following:

Cell 1

import numpy as np
from IPython.display import Audio

# 2 seconds sine wave
sr = 22050 # sample rate
T = 2.0    # seconds
t = np.linspace(0, T, int(T*sr), endpoint=False) # time variable
x = 0.5*np.sin(2*np.pi*440*t)                # pure sine wave at 440 Hz

Audio(x, rate=sr, element_id="my-audio")

Output: Audio widget

Cell 2

%%javascript

// Get the Audio widget from the DOM
var aud_js = document.getElementById("my-audio");
aud_js.play();

Result: Code controlled audio play


While this solution allows for Audio widget interaction, such as 'play', 'pause' and 'currentTime' (seeking), it might be more desirable that this is implemented at a Widget level.

Even more desirable might be to expose these JavaScript functions through a Python interface, preventing the need for the user to write any JavaScript code. The library Panel, from PyViz has taken this approach: pyviz/panel#215

@NumesSanguis

This comment has been minimized.

Copy link
Author

commented Apr 3, 2019

To give an idea what adding this makes possible in terms of Dashboarding (plotting with HoloViews, Bokeh backend):
holoviews-interactive-audio_mouse

  • The green line represents where the audio currently is, which is updated through a callback in JavaScript when 'currentTime' changes, which updates a Python variable.
  • The blue line follows the cursor
  • When clicked, JavaScript code is executed to set the Audio widget's 'currentTime' based on the x-value of the cursor.
@Carreau

This comment has been minimized.

Copy link
Member

commented Apr 17, 2019

Apologies for the delay in responding,

this looks great, I'll try to find some time to review, but I don't see any reason to refuse this on principle.

@@ -89,7 +89,8 @@ class Audio(DisplayObject):
"""
_read_flags = 'rb'

def __init__(self, data=None, filename=None, url=None, embed=None, rate=None, autoplay=False, normalize=True):
def __init__(self, data=None, filename=None, url=None, embed=None, rate=None, autoplay=False, normalize=True, *,
element_id=None):

This comment has been minimized.

Copy link
@Carreau

Carreau Apr 19, 2019

Member

@SylvainCorlay @martinRenou @jasongrout ; is there a conventional name for element_id or is that an acceptable name ?

I've also made it kwarg only.

@Carreau

This comment has been minimized.

Copy link
Member

commented Apr 19, 2019

One minor question, and if that's ok for dev more familiar with widgets i'm +1.

@Carreau Carreau requested review from SylvainCorlay and jasongrout Apr 19, 2019

@SylvainCorlay

This comment has been minimized.

Copy link
Member

commented Apr 20, 2019

@NumesSanguis independently of this PR, you may be interested in the Audio widget from ipywidgets which is trully a jupyter widget and not a rich mime type renderer.

Advantages is that you can put them in widget layouts, and data is passed as a binary buffer instead of a base64 encoded buffer over the wire.

Finally, combine with ipywebrtc, you can use it to record audio channel of video stream, stream it back etc...

@SylvainCorlay

This comment has been minimized.

Copy link
Member

commented Apr 20, 2019

Also, I am fine with adding an element id to the display message. I don't have an opinion on the name. element_id is probably fine. (could be dom_id, dom_element_id?).

Others may think this is an abstraction leak :)

@Carreau Carreau merged commit d3ffae3 into ipython:master Apr 22, 2019

0 of 2 checks passed

continuous-integration/appveyor/pr Waiting for AppVeyor build to complete
Details
continuous-integration/travis-ci/pr The Travis CI build is in progress
Details
@jasongrout

This comment has been minimized.

Copy link
Member

commented Apr 23, 2019

People will just have to be aware that it won't work as expected if the same id is on the page twice (which is very possible if a user executes similar code twice in jlab, or opens two views of the same notebook).

@Carreau Carreau added this to the 7.5 milestone Apr 23, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.