Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Warn about poor lighting conditions, or no head detected #26

Open
1 of 4 tasks
1j01 opened this issue Aug 11, 2022 · 0 comments
Open
1 of 4 tasks

Warn about poor lighting conditions, or no head detected #26

1j01 opened this issue Aug 11, 2022 · 0 comments
Labels
affects:desktop-app Affects the Electron app affects:web-library Affects usage of the tracky-mouse.js library enhancement New feature or request

Comments

@1j01
Copy link
Owner

1j01 commented Aug 11, 2022

  • It should detect if the video stream is too dim, and show a warning message suggesting you improve the lighting conditions, such as with a lamp facing the user.
  • It should also detect if it's receiving (essentially) pitch black, and give a different message, because it's more likely a dust/privacy cover occluding the camera.
  • It should also show a warning in a similar way if no head is detected.
  • It should detect back-lighting (e.g. ceiling lights, when camera is below the user's head): a blob of bright light around the user's head, and significant fluctuation the apparent brightness of the user's face. That's my attempt at simplifying the definition for computer vision, but it's probably not good enough. I mean, if the back-lighting is really bad, the user would be just a silhouette, but it also shouldn't detect dark skin as a lighting issue, so I'd hesitate to add any absolute threshold of face luminance. That said, maybe dark skin is harder to light properly, and front-lighting would be more important? I don't know.

It should show the webcam view temporarily, so you can see what it's talking about, and see what's going on.
Either 1. the app window should be brought to front (but not focused), and hidden (after a second) if the issue becomes resolved (unless the window got focused by the user, in which case it should stay visible), or 2. the message and webcam view should be shown on the overlay window, and should avoid the mouse, either hiding mostly and blurring, or moving out of the way, so that you don't think you can click on them. Or you should be able to click on them, but that's more complicated.
(Showing the webcam on the overlay screen would mean... using WebRTC?)

These warnings should not show up immediately, only if the condition persists for a few seconds. Although jumps in lighting can cause problems (#22) (and automatic brightness may bring it within normal levels before a reasonable period of laxity), 1. it would probably be too annoying if these warnings showed fast enough to catch that happening, and 2. it's more the change that causes that issue, not the lack of absolute luminosity; that is, it could cause a bad mouse jump while brightening too.

This should be tested with regression testing, i.e. using recorded footage, not just tested live based on the present lighting conditions.

@1j01 1j01 added enhancement New feature or request affects:web-library Affects usage of the tracky-mouse.js library affects:desktop-app Affects the Electron app labels Aug 11, 2022
1j01 added a commit that referenced this issue May 9, 2024
Partially addresses #26
Also working towards #41
and #49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
affects:desktop-app Affects the Electron app affects:web-library Affects usage of the tracky-mouse.js library enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant