Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add 'id' to Audio widget's HTML for javascript interaction #11670

Merged
merged 3 commits into from Apr 22, 2019

Conversation

NumesSanguis
Copy link

@NumesSanguis NumesSanguis commented Apr 3, 2019

Problem: At present, the Audio widget can play a .wav, numpy data among others. However, once initialized, we cannot interact with this Widget through code. This is a problem when creating e.g. an interactive audio dashboard.

Solution: By adding an id to the HTML code, we can grab the Audio widget through JavaScript.

With this pull request, you can do the following:

Cell 1

import numpy as np
from IPython.display import Audio

# 2 seconds sine wave
sr = 22050 # sample rate
T = 2.0    # seconds
t = np.linspace(0, T, int(T*sr), endpoint=False) # time variable
x = 0.5*np.sin(2*np.pi*440*t)                # pure sine wave at 440 Hz

Audio(x, rate=sr, element_id="my-audio")

Output: Audio widget

Cell 2

%%javascript

// Get the Audio widget from the DOM
var aud_js = document.getElementById("my-audio");
aud_js.play();

Result: Code controlled audio play


While this solution allows for Audio widget interaction, such as 'play', 'pause' and 'currentTime' (seeking), it might be more desirable that this is implemented at a Widget level.

Even more desirable might be to expose these JavaScript functions through a Python interface, preventing the need for the user to write any JavaScript code. The library Panel, from PyViz has taken this approach: holoviz/panel#215

@NumesSanguis
Copy link
Author

To give an idea what adding this makes possible in terms of Dashboarding (plotting with HoloViews, Bokeh backend):
holoviews-interactive-audio_mouse

  • The green line represents where the audio currently is, which is updated through a callback in JavaScript when 'currentTime' changes, which updates a Python variable.
  • The blue line follows the cursor
  • When clicked, JavaScript code is executed to set the Audio widget's 'currentTime' based on the x-value of the cursor.

@Carreau
Copy link
Member

Carreau commented Apr 17, 2019

Apologies for the delay in responding,

this looks great, I'll try to find some time to review, but I don't see any reason to refuse this on principle.

@@ -89,7 +89,8 @@ class Audio(DisplayObject):
"""
_read_flags = 'rb'

def __init__(self, data=None, filename=None, url=None, embed=None, rate=None, autoplay=False, normalize=True):
def __init__(self, data=None, filename=None, url=None, embed=None, rate=None, autoplay=False, normalize=True, *,
element_id=None):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@SylvainCorlay @martinRenou @jasongrout ; is there a conventional name for element_id or is that an acceptable name ?

I've also made it kwarg only.

@Carreau
Copy link
Member

Carreau commented Apr 19, 2019

One minor question, and if that's ok for dev more familiar with widgets i'm +1.

@SylvainCorlay
Copy link
Member

@NumesSanguis independently of this PR, you may be interested in the Audio widget from ipywidgets which is trully a jupyter widget and not a rich mime type renderer.

Advantages is that you can put them in widget layouts, and data is passed as a binary buffer instead of a base64 encoded buffer over the wire.

Finally, combine with ipywebrtc, you can use it to record audio channel of video stream, stream it back etc...

@SylvainCorlay
Copy link
Member

Also, I am fine with adding an element id to the display message. I don't have an opinion on the name. element_id is probably fine. (could be dom_id, dom_element_id?).

Others may think this is an abstraction leak :)

@Carreau Carreau merged commit d3ffae3 into ipython:master Apr 22, 2019
@jasongrout
Copy link
Member

People will just have to be aware that it won't work as expected if the same id is on the page twice (which is very possible if a user executes similar code twice in jlab, or opens two views of the same notebook).

@Carreau Carreau added this to the 7.5 milestone Apr 23, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants