Real-Time Detection of Vertical Tongue Movements via Mobile Camera — Choosing the Right Approach

[Alex-en568]

Hey everyone.

I'm working on a task involving the detection of articulatory tongue movements using a smartphone's front-facing camera. Specifically, I need to reliably classify vertical tongue motion (up / down / neutral position) in real time. The environment is everyday — standard indoor lighting, no additional equipment. The target audience is children, which adds variability in terms of anatomy and behavior in front of the camera.

I've experimented with MediaPipe, but its tongue tracking is pretty lacking. I'd love to hear whether there are approaches that actually work in practice, or whether this is fundamentally a hard-constrained problem under these shooting conditions. Could something be borrowed from adjacent fields — say, deformable object tracking or industrial detection, where similar problems have already been solved reasonably well?

Has anyone actually worked on detection or tracking of intraoral objects (tongue, teeth) on mobile hardware that could realistically achieve high detection accuracy?

Thanks in advance.

[natsuki-engr]

This is a fascinating problem, but I suspect the Vue Discussions board may not be the best place to get high-quality answers, since most contributors here specialize in Vue.

You may have better luck asking in MediaPipe or machine learning communities. Also, Project Babble might be worth investigating, as it's one of the more visible projects exploring camera-based tongue tracking and could point you toward relevant techniques or prior research.
https://babble.diy

Hope you find a solution, and good luck with the project!

[Alex-en568]

Thank you so much for your reply!

[kuldeeprajput-dev]

if you find the answer help full, and problem solve mark as answered