Linux Support

[gozzarda]

I imagine you have considered this previously, but having had a quick look I don't see any dependencies that would prevent this project building for Linux now that .NET is available. I may be completely wrong about that, but thought it would be quicker for you to tell me whether Linux support is possible rather than me going digging.

This seems like a really cool project and I would love to see it support more platforms than just Windows.

[nash8350]

There needs to be hardware support for Linux. Which hardware are you using? I believe Tobii EyeX only offers support for Windows at this time. TalonVoice has a custom driver for MacOS but not Linux. It still proves it's possible to reverse engineer it. I know there are open source projects with drivers for TrackIR. Of course, Linux already has webcam support.

The only other Windows dependency is the UI. We'd have to make a cross-platform UI to support Linux or MacOS. That should be doable though.

[gozzarda]

Thanks for the reply!

I would be looking to use the Tobii EyeX and a webcam. I have a couple of applications in mind, but mostly I would love this as a productivity tool for myself.

Tobii seem to have recently released an alpha SDK for Linux: https://developer.tobii.com/consumer-eye-trackers/stream-engine/getting-started/

For development, you can probably forgo a UI at least at first in favour of a CLI.

I would like to offer my time to help develop this feature, but don't have enough of it to do so for the next month at least.

[nash8350]

Wow thats really cool to see Linux support! Yeah I'd be happy to have some help when you have time. A CLI version would probably work fine as long as the window manager provides API hooks to set the cursor position.

[dudman]

I was looking through the code here, and it dosn't look so much, perhaps a python app to coordinate and intergrate the tobii + eviacam would work. I havn't got he tobii 4c yet, but really am considering it. I saw https://www.youtube.com/watch?v=3wzgTG9CGRw the api working on ubuntu., with a git and some instructions.

[lorenzleutgeb]

Hello!

I would just like to express my interest in a Linux compatible version. After I stumbled upon precisiongazemouse.com I already compiled and installed Enable Viacam ("eviacam") and ordered a Tobii eyeX 4C unit which is being shipped at the moment.

I am using Linux for ten years now (Fedora 30 with Wayland at the moment), and I work as a software engineer, so I think am able to contribute to the effort of porting PrecisionGazeMouse to Linux.

@dudman, I think that your proposal of coming up with a substitute for PrecisionGazeMouse in Python sounds like a very pragmatic approach, but I would like to ask you to elaborate a bit more, based on the following ideas.

OS/WM Interface

How will we interface with the operating system or window manager? What kind of interface is it? Would we have to implement a virtual mouse driver? Or do we just use a library like PyUserInput (which does not support Wayland)? Would it be something like wayland_client?

Edit: I think the way to go would be to use uinput (or the corresponding Python bindings for it) to publish events. I found a Python 3 code example using libevdevs facility to create a uinput device.

Enable Viacam

How will we interface with Enable Viacam? I haven't looked at it's source, code, but PrecisionGazeMouse does not seem to interface with it. @nash8350 could you confirm that you just run eviacam alongside PrecisionGazeMouse? Or alternatively please point us at the code that interfaces the two.

Edit: On another note, I just discovered the function tobii_gaze_origin_subscribe in the Stream Engine. It might be suitable to replace Enable Viacam. My thinking is that in order to perform gaze tracking, the eyeX also must have very accurate positioning data for the eyes/pupils. This position might also be suitable to do the "precise pointing".

Tobii eyeX 4C

How will we interface with the Tobii eyeX 4C hardware? I think that we'll have to wrap the Stream Engine provided by Tobii ourselves to get Python bindings. It seems that some people already tried that, but mostly for other APIs and on Windows:

• There are many Python bindings for the Tobii APIs on Windows: balancana/gazesdk, MichaelMauderer/python-eyex tvanasse/Tobii-EyeX-Python-Bindings ZxYuan/PyTobiiEyeX
• A gist with Python code that imports tobii_research. Consider the applicable license. The shared library that comes with the Python package does not look like it is the Stream Engine. Also, there's a Tobii employee stating that it won't work.
• A project using the Stream Engine to implement a mouse for Windows.

I think that the most interesting part of the code that powers the demo (YouTube link) you shared is the subscription callback. But instead of just moving an object on the screen we might in addition also send pointer movement (which then ties back to the OS/WM Interface section above).

If we want to use Python we should probably autogenerate bindings.

[lorenzleutgeb]

Quick update: I got the Tobii eyeX 4C running in a prototype-fasion. My "main driver" (the program that makes everything work out somehow) is written in Rust. It links against Tobii's stream engine shared library and uses that to subscribe to gaze updates. These are then translated to pointer movements and forwarded to Wayland via xdotool. I couldn't get uinput to work.

[dudman]

@lorenzleutgeb Great stuff... sorry to say I havn't been able to get the hardware yet, difficult to justify with the wife, right now. When I looked through the source for precisiongazemouse it seemed that there was an api for the tobi hardware but the viacam software just emulated a mouse and moved the pointer. You've got some good links and research there, I think your making good progress understanding the parts, more than I've managed to date. Will be interested to follow along and see what you get working, and how you solve the different pieces :)

[nash8350]

Hey @lorenzleutgeb and @dudman thanks for continuing this thread. I'm not an expert at Linux UI programming I mostly do headless apps on Linux so I'm not as familiar with Wayland or X. I know Java offers a cross-platform mouse pointer API, but I'm not sure if it supports Wayland. Qt appears to have some support for Wayland. The xdotool looks very promising so if it's working that's great. Please keep us up to date on your progress!

It's true that PrecisionGazeMouse does not directly interface with eViacam. The way it works is through using a hotkey as a signal. By default the hotkey to enable eViacam is F11. So when we press down the movement hotkey F3 (or whatever you choose), it also presses F11 to enable eViacam. When you key up on F3, it also keys up F11. It's not the best interface and I'd prefer an API integration but that's all I have at the moment.

I haven't considered using tobii_gaze_origin_subscribe as a precision pointer. That's an interesting idea worth trying. I have tried their head tracking (in the EyeX only mode). It wasn't very accurate, which is why I opted for eViacam.

[shervinemami]

You should look into the Python library "pynput", it's very easy to use and emulates both keyboard and mouse, and is cross platform.

[nash8350]

Great tip thanks @shervinemami! @lorenzleutgeb I investigated your suggestion to use tobii_gaze_origin_subscribe as a precision pointer this weekend. The Tobii.Interaction library provides a higher level EyePositionStream which I used in my test. Unfortunately, there is too much jitter to use as a precision pointer. Even with a smoothing filter, it's got a jitter with a radius of about 20px which is too much for selecting text or clicking on small links or buttons. In contrast, eViacam has no noticeable jitter.

[lorenzleutgeb]

@lorenzleutgeb I investigated your suggestion to use tobii_gaze_origin_subscribe as a precision pointer this weekend. The Tobii.Interaction library provides a higher level EyePositionStream which I used in my test. Unfortunately, there is too much jitter to use as a precision pointer. Even with a smoothing filter, it's got a jitter with a radius of about 20px which is too much for selecting text or clicking on small links or buttons. In contrast, eViacam has no noticeable jitter.

I am using the Tobii device as a warp pointer, and eViacam (now also experimenting with a head-mounted gyroscope) as a precision pointer.

[noseglasses]

@lorenzleutgeb, could you please explain how you currently use the Tobii device as warp pointer. Did you manage to use your setup as a productive replacement for mouse movement?

i am also looking for a Linux solution and PrecisionGazeMouse seems very promising.

[spudnic]

Any update on this I would love to help if you need more hands.