Webcam initalization assumption

[joreg]

hey dom and torin,

i see an issue where on some machines the webpage would run and show the camera but the prediction-loop is not initialzed. i tracked this down to what i believe is a wrong assumption in your setup() routine. i am not really familiar with js so maybe i'm reading things wrong, but here goes:

• first thing in setup() is opening the websocket
• i assume this also directly attempts to connect to the websocketserver
• if the connection is opened the client sends a list of available cameras
• meanwhile setup() continues with a bunch of stuff and at the end tries to start the webcam and then initializes the prediction-loop but only if the webcam is already running! if the webcam is not running yet at this point, we currently have no chance of initializing the prediction-loop

i wonder if instead, at the end of setup() we don't start the camera but always start the prediction-loop. in the loop we check if the camera is running. so whenever the camera is then started or changed, prediction would start?!

[domisjustanumber]

Hey @joreg thanks for taking the time to dig into this. Getting webcams to reliably start and return data has been something I've been tinkering with for a while, and this is a new way of it breaking that I haven't seen before!

That's a good idea on how to catch it - I can move the webcam started detection into the prediction loop - then if the webcam takes a while to start, there are more chances to catch it later.

If you want to make the change, feel free to submit a PR for it and/or I'll be doing some work on this project over the holidays so can put the change in then.

[joreg]

so here is how i fixed it on our end for now: vvvv/VL.MediaPipe@0b56dff?diff=unified&w=1

not sure if this is the best way, but it seems to work so far.

[domisjustanumber]

Oh amazing, thank you!

[deseipel]

I'm also having web cam issues when running it. Seems like the camera its trying to use doesn't default to the correct resolution. But I'm not sure how I can set that in the MediaPipe container. And the image segmentation container is supposed to take the same cam input? I did come up with a workaround using NDI in/out.

[domisjustanumber]

Hey @joreg I rolled some of the suggested changes you made into the latest release, so hopefully the webcam loads more reliably and faster now. Thanks for the tip and you're welcome to copy any changes to your vvvv implementation!

https://github.com/torinmb/mediapipe-touchdesigner/releases/tag/v0.4.1

[domisjustanumber]

@deseipel this release should also (hopefully) fix your webcam startup/resolution issue https://github.com/torinmb/mediapipe-touchdesigner/releases/tag/v0.4.1

[joreg]

@domisjustanumber thanks! i've just updated our pack with your latest code. no more custom modifications on our end.