Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Browser extension to bridge between the desktop app and web pages #27

Open
1j01 opened this issue Aug 12, 2022 · 2 comments
Open

Browser extension to bridge between the desktop app and web pages #27

1j01 opened this issue Aug 12, 2022 · 2 comments
Labels
affects:browser-extension Affects the browser extension enhancement New feature or request

Comments

@1j01
Copy link
Owner

1j01 commented Aug 12, 2022

I want to make a browser extension, which would:

  1. Enhance system-level dwell clicking with knowledge of buttons, links, and other controls on web pages, with hover halos and centered dwell click indicators.
  2. Make it so you don't need to disable dwell clicking in the desktop app to use a web app that provides dwell clicking (for web applications that include the tracky-mouse.js library, or else support some protocol yet to be defined).
  3. Provide the equivalent of the desktop application for Chrome OS.
  4. Automatically enhance webpages to be friendlier toward facial mouse input, by preventing menus from closing based on hover, enlarging elements etc., probably using site-specific enhancements.
@1j01 1j01 added enhancement New feature or request affects:browser-extension Affects the browser extension labels Aug 12, 2022
@1j01
Copy link
Owner Author

1j01 commented Apr 21, 2024

thinking about detecting controls, briefly:

  • can I make it work for just the tracky mouse UI main window quickly?
    • nah, probably should wait and address the harder problem of communicating with webpages first
  • when communicating with webpages, how will it translate page coordinates to screen coordinates?
    • well theoretically javascript has window.screenLeft etc. but I'm not sure these would be accurate in all cases
    • could render fiduciaries to the page (at viewport corners) and have the desktop app read the pixels of the screen to establish a frame of reference, with unique fiduciaries for each page, or unique per "alignment request"
    • there's also page scaling (browser zoom level) and screen DPI to consider

@1j01
Copy link
Owner Author

1j01 commented Apr 21, 2024

oh, actually, I think my plan was to avoid translating page coordinates into screen coordinates by having the browser extension render the halos on the page, rather than the desktop app rendering the halos in the screen overlay window.
that's like 100x simpler, right? don't need to worry about screen DPI or page scaling (pretty much), let alone rendering fiduciaries and accessing screen pixels
I feel like I almost super overcomplicated this future for my future self by writing that half-baked note

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
affects:browser-extension Affects the browser extension enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant