Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Audio output is blocked on iOS (and probably elsewhere) #14

Open
cdcv opened this Issue Oct 5, 2018 · 10 comments

Comments

Projects
None yet
2 participants
@cdcv
Copy link

cdcv commented Oct 5, 2018

Thanks for the fantastic repo.

I have code that will correctly output speech audio when a button is pressed using kathy.speak() as in the repo example, using desktop chrome browser. On iOS (chrome and safari), no audio is played when the button is pressed, and the following console error is displayed:

Unhandled Promise Rejection: NotAllowedError: The request is not allowed by the user agent or the platform in the current context, possibly because the user denied permission.

This appears to be because the user action of hitting a button is not correctly link to the playing of the sound. This is described, eg here:
scottschiller/SoundManager2#178

Is there a solution to this?

Thanks!

@LoekvanKooten

This comment has been minimized.

Copy link

LoekvanKooten commented Oct 9, 2018

This will solve all your issues:

       `// Does not work in Safari/mobile
        // var blob = new Blob([arrayBuffer]);
        // var url = URL.createObjectURL(blob);

        // Should work in Safari/on mobile
        var blob = new Blob([arrayBuffer], {type: 'audio/mpeg'});
        // All browsers, except Firefox
        try {
            url = webkitURL.createObjectURL(blob);
        }
        // Firefox
        catch(err) {
            url = URL.createObjectURL(blob);
        }`
@cdcv

This comment has been minimized.

Copy link
Author

cdcv commented Oct 10, 2018

LoekvanKooten, thank you VERY much for your help. This could save me (and I'm guessing a lot of other people) a LOT of time.
Could you let me/the world know how to use this. Thanks!
I'm not sure how you mean to indicate that this could should be used. Is this a change to the repo itself? If so, seems like an awesome PR. Alternatively, is this code that should be included in the calling js?
Do you have an example of how / where to use this code, and what the calling function(s) would look like?

Thanks again for your help.

Best

@LoekvanKooten

This comment has been minimized.

Copy link

LoekvanKooten commented Oct 10, 2018

No problem at all! Just open up ChattyKathy.js and outcomment/delete the two lines below "does not work in Safari/mobile". Then replace those with everything below "should work in Safari/on mobile".

Make sure you refresh your browser cache before using the new ChattyKathy.js. I believe Ctrl + F9 does the trick; else call ChattyKathy.js? or ChattyKathy.js?123 (123 is just a random number) instead of ChattyKathy.js That will force a browser refresh.

@cdcv

This comment has been minimized.

Copy link
Author

cdcv commented Oct 10, 2018

Wonderful. Thanks very much for the help.

@cdcv

This comment has been minimized.

Copy link
Author

cdcv commented Oct 10, 2018

Hi LoekvanKooten,
Unfortunately, I have not been able to get your solution to work. Not sure if you have any suggestions / thoughts on why.
I made the changes that you suggested.
The code (still) works correctly in a desktop browser, playing speech text without issues.
In a mobile iOS browser (either mobile Chrome or mobile Safari) the rest of the app works, and other audio works, but the speech audio isn't played and I still get the same error:

Unhandled Promise Rejection: NotAllowedError: The request is not allowed by the user agent or the platform in the current context, possibly because the user denied permission.

Thanks again!

@LoekvanKooten

This comment has been minimized.

Copy link

LoekvanKooten commented Oct 11, 2018

This baffles me, really. On my end everything works, in all browsers. Is it possible this error is caused by something else than ChattyKathy.js?

@LoekvanKooten

This comment has been minimized.

Copy link

LoekvanKooten commented Oct 13, 2018

Try inserting

localStorage.clear();

in front of

localStorage.setItem("chattyKathyDictionary", JSON.stringify(localPlaylist));

This is a very dirty workaround to empty the cache for each sound you play. The thing is that after a while, this cache apparently overflows, causing an error and muting audio for good.

@cdcv

This comment has been minimized.

Copy link
Author

cdcv commented Oct 13, 2018

I discovered the issue. On (at least some) mobile browsers, it is required for an onclick function to .play() the audioElement created in ChattyKathy. Fo example, I made the audioElement global, and then attached a call to audioElement.play() to the onclick of a UI button. The user only needs to click the button once. Subsequent calls to ChattyKathy will work (I hope this stays true in the future) after the audioElement has been played using .play() once, even if the first time produces no sound. Apparently on desktop Chrome this is not (currently) required.

@cdcv

This comment has been minimized.

Copy link
Author

cdcv commented Oct 13, 2018

Here is example code taken from ChattyKathy.js

audioElement = {}; // make this global so that it can be clickable by user
function ChattyKathy(settings) {

settings = getValidatedSettings(settings);

// Add audio node to html
var elementId = "audioElement" + new Date().valueOf().toString();
audioElement = document.createElement('audio');
audioElement.setAttribute("id", elementId);
document.body.appendChild(audioElement);
@LoekvanKooten

This comment has been minimized.

Copy link

LoekvanKooten commented Oct 13, 2018

Fantastic! That solved my one remaining issue: needing to press each button twice in Safari.

The whole code now becomes:

// In Safari, a global audioElement must be created that is empty
var audioElement = {}; // make this global so that it can be clickable by user

function ChattyKathy(settings) {`

settings = getValidatedSettings(settings);

// Add audio node to html
var elementId = "audioElement" + new Date().valueOf().toString();
var audioElement = document.createElement('audio');
audioElement.setAttribute("id", elementId);
document.body.appendChild(audioElement);

// In Safari, a global audioElement must be created that is empty
// After creation, we play the empty element once to prep the system to play actual audio after this
audioElement.play();
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.