Touchscreen support in input proxy #2281

atlhaua · Aug 29, 2016

Qubes OS version :

3.1

Affected TemplateVMs :

all

Expected behavior:

Touchscreen used to work fine on the first installation.

Actual behavior:

after testing and tweaking the system for a while I decided to re-installed Qubes to have a fresh start. The only differences this time are that I have added a sys-usb service VM and that I have installed Qubes on an internal drive instead of an external one connected through USB.

Steps to reproduce the behavior:

explained above.

General notes:

I have found out that the touchscreen may be set up as an USB device and therefore isolated from dom0. This might explain why it is not working. Is there a way to make the touchscreen available to dom0 and the other VM's while keeping sys-usb?

[user@sys-usb /]$ lsusb
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
**Bus 002 Device 005: ID 03eb:8b03 Atmel Corp.** 
Bus 002 Device 004: ID 064e:9700 Suyin Corp. Asus Integrated Webcam
Bus 002 Device 003: ID 8087:07dc Intel Corp. 
Bus 002 Device 002: ID 8087:8000 Intel Corp. 
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Yes, take a look at the USB documentation. You have to use either qubes.InputKeyboard or qubes.InputMouse, depending on which type of device your touchscreen is (probably the latter, but possibly both).

Please note that this type of query is not appropriate for qubes-issues. Please send any further replies to the qubes-users mailing list instead.

andrewdavidwong · Aug 29, 2016

Yes, take a look at the USB documentation. You have to use either qubes.InputKeyboard or qubes.InputMouse, depending on which type of device your touchscreen is (probably the latter, but possibly both).

Please note that this type of query is not appropriate for qubes-issues. Please send any further replies to the qubes-users mailing list instead.

Actually, input proxy do not support touchscreen devices currently. It shouldn't be hard do add it, but there are multiple higher priority tasks. But if anyone want to work on this, it require:

researching and adding appropriate data to initial proxy message: https://github.com/QubesOS/qubes-app-linux-input-proxy/blob/master/src/protocol.h (make sure to add new fields at the end of the structure - after "name" field, for protocol compatibility); this will most likely include min/max coordinates on each axis
make sure that dom0 part will still work even if remote sent no additional info - take a look at how "name" field was introduced
add support for appropriate message types (probably add --touchscreen option or such)
add appropriate qrexec service files, systemd units, and udev rules triggering them

It may be useful to examine the device using evtest first.

marmarek · Aug 30, 2016

Actually, input proxy do not support touchscreen devices currently. It shouldn't be hard do add it, but there are multiple higher priority tasks. But if anyone want to work on this, it require:

researching and adding appropriate data to initial proxy message: https://github.com/QubesOS/qubes-app-linux-input-proxy/blob/master/src/protocol.h (make sure to add new fields at the end of the structure - after "name" field, for protocol compatibility); this will most likely include min/max coordinates on each axis
make sure that dom0 part will still work even if remote sent no additional info - take a look at how "name" field was introduced
add support for appropriate message types (probably add --touchscreen option or such)
add appropriate qrexec service files, systemd units, and udev rules triggering them

It may be useful to examine the device using evtest first.

Hi again,

I found a workaround solution assining the USB bus that manages the touchscreen to dom0 (on sys-usb services tab). The disadvantage in terms of security is that the other devices assigned to this bus (USB port and built-in camera are also assigned to dom0.

The news is that after upgrading to Qubes 3.2, this workaround solution does not work anymore. Thinking that it might be related to having Xfce instead of KDE I swaped the desktop environment to KDE as instructed in your help pages but it seems that was not the issue.
I do not know if this the right place for posting this issue but since marmarek reopened it, I thought I might as well update it and enquire for a possible solution/workaround.
Thanks in advance.
Atlahua

atlhaua · Feb 21, 2017

Hi again,

I found a workaround solution assining the USB bus that manages the touchscreen to dom0 (on sys-usb services tab). The disadvantage in terms of security is that the other devices assigned to this bus (USB port and built-in camera are also assigned to dom0.

The news is that after upgrading to Qubes 3.2, this workaround solution does not work anymore. Thinking that it might be related to having Xfce instead of KDE I swaped the desktop environment to KDE as instructed in your help pages but it seems that was not the issue.
I do not know if this the right place for posting this issue but since marmarek reopened it, I thought I might as well update it and enquire for a possible solution/workaround.
Thanks in advance.
Atlahua

I have a laptop that accidentally* has a touchscreen and I have looked at it.

Maybe the reason why @atlhaua has the issues it rd.qubes_hide_all_usb kernel parameter, which has appeared in a clean install.

I've also looked at the touch USB device. Command lsusb -l sees it as mouse. Probably the main difference from standard mouse is absolute coordinates vs. relative movements, plus maybe multipointer/multitouch support. There is some output from evtest: https://gist.github.com/v6ak/0732c1bc714473c65aa259e4a9e948f1

The input proxy seems to have at least some parts implemented. The sender part, well, at least produces some output (not sure if it is correct, but maybe sender part is rather cat with some header). The receiver part has --tablet support, but it does not seem to support it. The code seems to have some preliminary support, but there is comment absmax is missing. Maybe there is even more missing.

I have also tried saving the proxy output to a file and then playing it to dom0, but it didn't work. Another surprising thing: When I tried to run receiver directly in dom0 on the saved file, it has always returned zero (even with --mouse or --keyboard). ~~IIUC the code, it should have returned non-zero error code.~~ EDIT: The function validate_and_forward_event returns -1 on invalid EV_SYN or on unknown event, but it returns 1 on other ignored events, silently ignoring them. The caller function (i.e., process_events) considers any positive return value as success, so it is not propagated. Unfortunately, there does not seem to be any logging of skipped events.

EDIT: Added few links and fixed some typo(s).

*) Well, touchscreen is by no means critical for me.

v6ak · May 13, 2017

I have a laptop that accidentally* has a touchscreen and I have looked at it.

Maybe the reason why @atlhaua has the issues it rd.qubes_hide_all_usb kernel parameter, which has appeared in a clean install.

I've also looked at the touch USB device. Command lsusb -l sees it as mouse. Probably the main difference from standard mouse is absolute coordinates vs. relative movements, plus maybe multipointer/multitouch support. There is some output from evtest: https://gist.github.com/v6ak/0732c1bc714473c65aa259e4a9e948f1

The input proxy seems to have at least some parts implemented. The sender part, well, at least produces some output (not sure if it is correct, but maybe sender part is rather cat with some header). The receiver part has --tablet support, but it does not seem to support it. The code seems to have some preliminary support, but there is comment absmax is missing. Maybe there is even more missing.

I have also tried saving the proxy output to a file and then playing it to dom0, but it didn't work. Another surprising thing: When I tried to run receiver directly in dom0 on the saved file, it has always returned zero (even with --mouse or --keyboard). ~~IIUC the code, it should have returned non-zero error code.~~ EDIT: The function validate_and_forward_event returns -1 on invalid EV_SYN or on unknown event, but it returns 1 on other ignored events, silently ignoring them. The caller function (i.e., process_events) considers any positive return value as success, so it is not propagated. Unfortunately, there does not seem to be any logging of skipped events.

EDIT: Added few links and fixed some typo(s).

*) Well, touchscreen is by no means critical for me.

but there is comment absmax is missing. Maybe there is even more missing.

Yes, exactly. According to your evtest output, this should be somehow added to the protocol handshake:

  Event type 3 (EV_ABS)
    Event code 0 (ABS_X)
      Value   3134
      Min        0
      Max     4095
      Resolution      12
    Event code 1 (ABS_Y)
      Value    548
      Min        0
      Max     4095
      Resolution      21
    Event code 47 (ABS_MT_SLOT)
      Value      0
      Min        0
      Max        9
    Event code 53 (ABS_MT_POSITION_X)
      Value      0
      Min        0
      Max     4095
      Resolution      12
    Event code 54 (ABS_MT_POSITION_Y)
      Value      0
      Min        0
      Max     4095
      Resolution      21
    Event code 57 (ABS_MT_TRACKING_ID)
      Value      0
      Min        0
      Max 65535

The tricky part is that we want the initial message be a constant-size, but according to the above, there may be variable number of ranges. Maybe some arbitrary upper limit would work? Like 16, or even 8?

The function validate_and_forward_event returns -1 on invalid EV_SYN or on unknown event, but it returns 1 on other ignored events, silently ignoring them. The caller function (i.e., process_events) considers any positive return value as success, so it is not propagated. Unfortunately, there does not seem to be any logging of skipped events.

Yes, this is partially intentional. For example if you attach some mouse+keyboard device but you only want to allow mouse (using qrexec policy), any other event would produce log entry, allowing to quickly fill up the whole disk...
Maybe we should add some --verbose / --debug option?

marmarek · May 14, 2017

but there is comment absmax is missing. Maybe there is even more missing.

Yes, exactly. According to your evtest output, this should be somehow added to the protocol handshake:

  Event type 3 (EV_ABS)
    Event code 0 (ABS_X)
      Value   3134
      Min        0
      Max     4095
      Resolution      12
    Event code 1 (ABS_Y)
      Value    548
      Min        0
      Max     4095
      Resolution      21
    Event code 47 (ABS_MT_SLOT)
      Value      0
      Min        0
      Max        9
    Event code 53 (ABS_MT_POSITION_X)
      Value      0
      Min        0
      Max     4095
      Resolution      12
    Event code 54 (ABS_MT_POSITION_Y)
      Value      0
      Min        0
      Max     4095
      Resolution      21
    Event code 57 (ABS_MT_TRACKING_ID)
      Value      0
      Min        0
      Max 65535

The tricky part is that we want the initial message be a constant-size, but according to the above, there may be variable number of ranges. Maybe some arbitrary upper limit would work? Like 16, or even 8?

The function validate_and_forward_event returns -1 on invalid EV_SYN or on unknown event, but it returns 1 on other ignored events, silently ignoring them. The caller function (i.e., process_events) considers any positive return value as success, so it is not propagated. Unfortunately, there does not seem to be any logging of skipped events.

Yes, this is partially intentional. For example if you attach some mouse+keyboard device but you only want to allow mouse (using qrexec policy), any other event would produce log entry, allowing to quickly fill up the whole disk...
Maybe we should add some --verbose / --debug option?

Yes, this is partially intentional. For example if you attach some mouse+keyboard device but you only want to allow mouse (using qrexec policy), any other event would produce log entry, allowing to quickly fill up the whole disk... Maybe we should add some `--verbose` / `--debug` option?

This would be good. Plus some throttled logging (e.g., stop logging after 10 failures) might be also useful (but a bit controversial). Also, I understand that logging of input events can log something potentially sensitive (imagine typing your password on keyboard that was misdetected as a mouse…) I know this should be rare, but still… Maybe default logging could just warn like «Some key code from “sys-usb: some device” is not allowed, so it was skipped. This is either an attack or a bug. You can enable verbose logging to see more.».

v6ak · May 15, 2017

Yes, this is partially intentional. For example if you attach some mouse+keyboard device but you only want to allow mouse (using qrexec policy), any other event would produce log entry, allowing to quickly fill up the whole disk... Maybe we should add some `--verbose` / `--debug` option?

This would be good. Plus some throttled logging (e.g., stop logging after 10 failures) might be also useful (but a bit controversial). Also, I understand that logging of input events can log something potentially sensitive (imagine typing your password on keyboard that was misdetected as a mouse…) I know this should be rare, but still… Maybe default logging could just warn like «Some key code from “sys-usb: some device” is not allowed, so it was skipped. This is either an attack or a bug. You can enable verbose logging to see more.».

Please forgive me if this should be obvious, but how will we know when this fix will be in a "dom0 update", or if it has already arrived? Or, must we obtain it some other way?

And, how do we enable it? dom0 Settings Manager?

Thank you.

ncm · Apr 24, 2018

Please forgive me if this should be obvious, but how will we know when this fix will be in a "dom0 update", or if it has already arrived? Or, must we obtain it some other way?

And, how do we enable it? dom0 Settings Manager?

Thank you.

@ncm: There should be an automated comment on this issue announcing when the package is available in each repo.

andrewdavidwong · Apr 24, 2018

@ncm: There should be an automated comment on this issue announcing when the package is available in each repo.

@ncm: I'm afraid I don't know how you'll enable it. I'll leave that for someone else to answer.

andrewdavidwong · Apr 24, 2018

@ncm: I'm afraid I don't know how you'll enable it. I'll leave that for someone else to answer.

Automated announcement from builder-github

The package qubes-input-proxy_1.0.12-1+deb9u1 has been pushed to the r4.0 stable repository for the Debian template.
To install this update, please use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

QubesOS/qubes-issues

Join GitHub today

Touchscreen support in input proxy #2281

Comments

atlhaua commented Aug 29, 2016 • edited by andrewdavidwong Edited 1 time andrewdavidwong edited Feb 21, 2017 (most recent)

Qubes OS version :

Affected TemplateVMs :

Expected behavior:

Actual behavior:

Steps to reproduce the behavior:

General notes:

This comment has been minimized.

andrewdavidwong commented Aug 29, 2016

andrewdavidwong closed this Aug 29, 2016

andrewdavidwong added the invalid label Aug 29, 2016

marmarek added enhancement C: other P: minor and removed invalid labels Aug 30, 2016

This comment has been minimized.

marmarek commented Aug 30, 2016

marmarek reopened this Aug 30, 2016

marmarek added the help wanted label Aug 30, 2016

marmarek added this to the Far in the future milestone Aug 30, 2016

marmarek changed the title from Touchscreen stopped working after second installation of Qubes to Touchscreen support in input proxy Aug 30, 2016

This comment has been minimized.

atlhaua commented Feb 21, 2017

This comment has been minimized.

v6ak commented May 13, 2017 • edited Edited 1 time v6ak edited May 14, 2017 (most recent)

This comment has been minimized.

marmarek commented May 14, 2017

This comment has been minimized.

v6ak commented May 15, 2017

marmarek added a commit to marmarek/qubes-app-linux-input-proxy that referenced this issue Apr 1, 2018

marmarek added a commit to marmarek/qubes-app-linux-input-proxy that referenced this issue Apr 1, 2018

marmarek added a commit to marmarek/qubes-app-linux-input-proxy that referenced this issue Apr 1, 2018

marmarek added a commit to marmarek/qubes-app-linux-input-proxy that referenced this issue Apr 1, 2018

marmarek added a commit to marmarek/qubes-app-linux-input-proxy that referenced this issue Apr 1, 2018

marmarek added a commit to marmarek/qubes-app-linux-input-proxy that referenced this issue Apr 2, 2018

marmarek added a commit to marmarek/qubes-app-linux-input-proxy that referenced this issue Apr 14, 2018

marmarek added a commit to marmarek/qubes-app-linux-input-proxy that referenced this issue Apr 14, 2018

marmarek added a commit to marmarek/qubes-app-linux-input-proxy that referenced this issue Apr 14, 2018

marmarek added a commit to marmarek/qubes-app-linux-input-proxy that referenced this issue Apr 14, 2018

marmarek referenced this issue Apr 14, 2018

marmarek added a commit to marmarek/qubes-app-linux-input-proxy that referenced this issue Apr 14, 2018

marmarek added a commit to marmarek/qubes-app-linux-input-proxy that referenced this issue Apr 14, 2018

marmarek added a commit to marmarek/qubes-app-linux-input-proxy that referenced this issue Apr 14, 2018

marmarek added a commit to marmarek/qubes-app-linux-input-proxy that referenced this issue Apr 14, 2018

marmarek added a commit to marmarek/qubes-app-linux-input-proxy that referenced this issue Apr 14, 2018

marmarek added a commit to marmarek/qubes-app-linux-input-proxy that referenced this issue Apr 14, 2018

marmarek added a commit to marmarek/qubes-app-linux-input-proxy that referenced this issue Apr 14, 2018

marmarek referenced this issue in QubesOS/qubes-app-linux-input-proxy Apr 14, 2018

marmarek closed this in marmarek/qubes-app-linux-input-proxy@adaacde Apr 17, 2018

This comment has been minimized.

ncm commented Apr 24, 2018

This comment has been minimized.

andrewdavidwong commented Apr 24, 2018

This comment has been minimized.

andrewdavidwong commented Apr 24, 2018

qubesos-bot added r4.0-jessie-stable and removed r4.0-jessie-cur-test labels Jun 12, 2018

This comment has been minimized.

qubesos-bot commented Jun 12, 2018

qubesos-bot added r4.0-stretch-stable and removed r4.0-stretch-cur-test labels Jun 12, 2018

This comment has been minimized.

qubesos-bot commented Jun 12, 2018

qubesos-bot added r3.2-centos7-stable r3.2-fc25-stable and removed r3.2-centos7-cur-test r3.2-fc25-cur-test labels Jun 12, 2018

This comment has been minimized.

qubesos-bot commented Jun 12, 2018

qubesos-bot added r3.2-fc26-stable r3.2-fc27-stable r3.2-fc28-stable and removed r3.2-fc26-cur-test r3.2-fc27-cur-test r3.2-fc28-cur-test labels Jun 12, 2018

This comment has been minimized.

qubesos-bot commented Jun 12, 2018

qubesos-bot added r4.0-dom0-stable and removed r4.0-dom0-cur-test labels Jun 12, 2018

This comment has been minimized.

qubesos-bot commented Jun 12, 2018

qubesos-bot added r3.2-dom0-stable and removed r3.2-dom0-cur-test labels Jun 12, 2018

This comment has been minimized.

qubesos-bot commented Jun 12, 2018

qubesos-bot added r4.0-centos7-stable and removed r4.0-centos7-cur-test labels Jun 12, 2018

This comment has been minimized.

qubesos-bot commented Jun 12, 2018

qubesos-bot added r4.0-fc26-stable r4.0-fc27-stable r4.0-fc28-stable r3.2-buster-stable r3.2-jessie-stable and removed r4.0-fc26-cur-test r4.0-fc27-cur-test r4.0-fc28-cur-test r3.2-buster-cur-test r3.2-jessie-cur-test labels Jun 12, 2018

This comment has been minimized.

qubesos-bot commented Jun 13, 2018

atlhaua commented Aug 29, 2016 •

edited by andrewdavidwong

Edited 1 time

andrewdavidwong edited Feb 21, 2017 (most recent)

v6ak commented May 13, 2017 •

edited

Edited 1 time

v6ak edited May 14, 2017 (most recent)

marmarek closed this in `marmarek/qubes-app-linux-input-proxy@adaacde` Apr 17, 2018