Can't get microphone capture to work reliably in Chrome

[fadookie]

Nature of issue?

• Found a bug
• Existing feature enhancement
• New feature request

Most appropriate sub-area of p5.js?

• Color
• Core/Environment/Rendering
• Data
• Events
• Image
• IO (Microphone)
• Math
• Typography
• Utilities
• WebGL
• Other (specify if possible)

Which platform were you using when you encountered this?

• Mobile/Tablet (touch devices)
• Desktop/Laptop
• Others (specify if possible)

Details about the bug:

• p5.js version: p5.js v0.9.0 July 01, 2019
• Web browser and version: Crome 78.0.3904.70 (Official Build) (64-bit)
• Operating System: macOS 10.14.6 (18G103)
• Steps to reproduce this:

I can't get microphone capture work reliably on my latest sketch: https://editor.p5js.org/fadookie/sketches/RM4wd2g8U

In the web editor, it works fine. But as soon as I download or show in fullscreen or presentation mode, I sometimes am unable to get any audio line in EVEN after waiting for a click event as suggested in the docs.

Sometimes it works without any user interaction, sometimes it never works, seemingly at random! And it's confusing because in the event the audio line in fails to start, I get a success callback from AudioIn#start rather than an error callback. I've never seen my error callback fire even once. This may be Chrome's fault, but it's making p5.js barely usable for making audio-reactive sketches.

Does anyone have any advice? Is there a more sure-fire way to wait for "user interaction" other than just waiting for a click on the canvas? Should I be re-creating the AudioIn? How can I tell my users what to do so it works? It also seems like there's a bug in #start such that errors don't actually get bubbled up.

[welcome]

Welcome! 👋 Thanks for opening your first issue here! And to ensure the community is able to respond to your issue, be sure to follow the issue template if you haven't already.

[lmccart]

Hm I'm not able to reproduce this, the sketch works fine for me. One thing to note is that when you run this sketch in the editor, clicking the play button counts as a click so it should start pretty immediately. If you run this without the editor you will need to click on the page. If you don't want play button click this behavior, you could have a button on the page that needs to be clicked on to start the mic.

In any case, I'm not seeing the issue with mic not starting. I'm on a similar version of mac and chrome as you. It could be you have a chrome extension that is messing with things. Can you try running this is an incognito window?

[Samuelfaure]

Same bug on my side. I'm on Arch linux, Chromium 78.
Incognito window changes nothing.

[stalgiag]

I am able to recreate on Windows 10, Chrome 78.

For me, it does work sometimes but breaks in a seemingly random fashion. p5.sound appears to be making an AudioContext prior to any gesture.

I made a copy of your example that doesn't create the p5.AudioIn object until user gesture which has improved the success-to-fail ratio such that the sketch works for me 95% of the time. But it appears that either p5.Sound is starting an AudioContext without any interaction from the sketch part of the time, or Chrome is catching it part of the time.

If I open my console with the above linked example, it works after mouse input if I don't see the Chrome AudioContext warning, but if I do see the Chrome AudioContext warning then the gesture will have no effect. Note that in that sketch I uploaded the un-minified version of the sound library so that you can do a stack trace. I also tested in a build from the master branch of the p5-sound repo. The stack trace for the AudioContext start errors points to a different location in the most recent build but the problem is still the same.

[Samuelfaure]

I'm able to hack around the problem by :
1/ using the instance mode
2/ adding a "resume" button to relaunch the context if there was an issue (although switching to instance mode fixed 95% of the problem)

[stalgiag]

Yes as @Samuelfaure points you can "resume" the context as a workaround.

I was able to achieve (seemingly) 100% functionality by calling getAudioContext().resume() after user gesture. This thread lead me there

[Samuelfaure]

Here is an easy example of instance mode + resume button if any vue.js adept needs it https://github.com/Samuelfaure/neon_genesis_cactus/blob/master/src/components/SoundControls.vue

[limzykenneth]

Shall we move this issue over to the p5.Sound repo so a in library workaround can be found there?

[kevinstadler]

please transfer to the appropriate repository, this isn't it