Mouse jumps confusingly when lighting changes #22
Labels
affects:desktop-app
Affects the Electron app
affects:web-library
Affects usage of the tracky-mouse.js library
bug
Something isn't working
Even switching between windows often causes the mouse to jump, because the head tracking gets lost, because the lighting changes, because the overall luminosity is different between the windows.
This happens when lighting isn't great, i.e. you don't have a lamp facing you, and the light of the screen is a significant factor in the overall amount of light reflecting off you.
I plan to include some guidance about lighting into the app, but I wonder if this could be improved by normalizing the luminosity histogram, perhaps selectively, like with a vignette for a multiplier of how much each pixel matters, approximating where the user's face likely is, without being too specific / relying on facemesh too much. My reasoning for focusing on the user for lighting normalization is that the user's face is what's important to normalize to keep tracking working, and the background scenery may not change nearly so much, since it's further behind, further away from the screen, and more affected by other light sources. If the background were to be counted equally to the face, the image of the face may not be normalized enough.
The text was updated successfully, but these errors were encountered: