Audio widget accept Byte / Numpy data (request)

[NumesSanguis]

Background

With the pull request of #215 , Panel supports an Python callable audio widget. However, it seems artificially limited to accepting only a String with a path to an audio file when setting "value". Please allow other data as well, to make it more comparable functionality-wise with the Jupyter Notebook Audio widget.

Attempt

I've been giving an attempt myself, but I'm too unfamiliar with the code base of Panel. This was resulting in errors:

import os
import numpy as np
import param
from base64 import b64encode

class Audio2(pn.widgets.Audio):
    
    data_array = param.Array(default=None)
    
    def _process_param_change(self, msg):
        msg = super(Audio2, self)._process_param_change(msg)
        if 'value' in msg and os.path.isfile(msg['value']):
            fmt = msg['value'].split('.')[-1]
            with open(msg['value'], 'rb') as f:
                data = f.read()
            template = 'data:audio/{mime};base64,{data}'
            data = b64encode(data)
            msg['value'] = template.format(data=data.decode('utf-8'),
                                           mime=fmt)
        
        # numpy data
        elif 'data_array' in msg:  # 'value' in msg and isinstance(msg['value'], np.ndarray):
            template = 'data:audio/{mime};base64,{data}'
            msg['value'] = template.format(data=msg['data_array'],
                                           mime="wav")
            
        return msg

At first I was trying to change msg['value'] directly, but I don't know how to extend the String check to also accept Byte/Numpy data. Then I tried to add an additional attribute, but that gave me: AttributeError: unexpected attribute 'data_array' to Audio.

Reuse IPython code

Since IPython is also BSD licensed, some code from here could be reused: https://github.com/ipython/ipython/blob/08aaa2e197a34dc820e7fef32b473d24b4547630/IPython/lib/display.py

Extra

Would be nice if you can pass in a String to audio file or audio data without using a named argument.

[jbednar]

Sounds like a good idea; happy to accept a PR supporting that.

[NumesSanguis]

Could you give a overview of which files in Panel are related to Widgets and which ones likely need change to support this request? Because with 0 knowledge of the code base, it would take a long time to write proper code that doesn't break other parts of Panel. Especially since the Audio widget is still undocumented (couldn't find anything about it in the docs)?

[jbednar]

The only file you should need to change is the one class in panel/widgets/misc.py, which only has a few lines of code, and nothing you put in there should break the rest of Panel as long as it's syntactically correct. Right now, you should be able to see that it's opening a file based on the given string, where you want to pass in the raw bytes, which should be a simple change.

[NumesSanguis]

I thought it was a simple change as well, but I'm afraid of the param.String() check. It doesn't allow duck typing and just checking what the user has input. Changing this can influence other people's code. Please see under the code block in my first post.

[jbednar]

Sorry; I didn't read all the way back up for my last reply. To get it working, you can change param.String to param.Parameter. That will accept anything the user supplies, but then we should pin down the parameter, e.g. by creating a new Parameter type that accepts a URL, a path, or a byte stream. I think that would be useful to have in general.