Offset in Y coordinate with Windows 7 and Multitouch HID Device

[ghost]

Hi all,

It seems there is a bug with Windows 7 (not tested on others Windows) and Kivy about the Y position of the cursor. There is a constant offset between the Windows Cursor position and the one Kivy uses. (See attached pictures). Note that the offset is bigger in fullscreen that in windowed mode.

After having a quick look at the code, it seems that this offset is due to the caption size which is substracted to calculate the Y coordinate (line 165 in file wm_touch.py).

I can try to run additional tests if needed.

Regards.

Touchtracer in windowed mode:

Touchtracer in fullscreen mode:

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[jamiecollinson]

I was discussing this issue on the #kivy IRC channel with @tshirtman yesterday. I was having difficulty hitting buttons at the bottom of the window in an app I'm prototyping, and he suggested using the touchtracer example to check the behaviour.

I can confirm the same issue on a Windows 8.1 Asus laptop with a touchscreen. The behaviour I see is:

• The y position seems generally accurate in the top half of the screen, but there's an increasing offset (as indicated in the images above) as the touch event approaches the lower edge of the window.
• I see roughly the same behaviour windowed and fullscreen - it seems to depend more on the height of the window than whether it's windowed or not.
  ** On an 800px high window it's offset by ~20px at the bottom
• The x position also grows offset as the right hand edge of the window is approached, but to a much lesser degree than the y offset. On a 1280px wide window the x position reaches maybe a 5px offset at the right hand edge.
• On the top edge the y position seems accurate and exhibits no offset. On the left edge the x position seems accurate and exhibits no offset.
• Non kivy applications don't exhibit the same behaviour, and in general seem very accurate.

I'm happy to do any further investigation that would help - I could really use a fix for this for my application!

[jamiecollinson]

@Vianneyz did you get anywhere with this? I'm building an application for a surface pro 3 and just received a test device, which exhibits the same behaviour (it's only on touch rather than pen input, so that must not use wm_touch).

Anyone got a workaround or quick fix? At the moment I'm looking at avoiding controls at the bottom of the screen & increasing control sizes, which is not ideal.

[ghost]

Hi,
I applied a "dirty" hack to my local kivy installation. In the file wm_touch.py, I replaced the line 165 which is:
y = 1.0 - (t.screen_y() - win_rect.y - caption) / float(win_rect.h)
by
y = 1.0 - (t.screen_y() - win_rect.y) / float(win_rect.h)

There is still a little offset with this but the app become usable in full screen, even if controls are placed in the bottom of the screen.

[jamiecollinson]

Thanks - that works pretty well for me for controls at the bottom of the screen. Unfortunately a control in the very top right is now not reachable, so I'm going to try a dirty if/else on t.screen_y > win_rect.y / 2

[jamiecollinson]

@Vianneyz I've found that replacing line 165 with the following solves both the top and bottom edge issues for me on Windows 8.1:

if (t.screen_y() - win_rect.y) > (win_rect.h / 2):
    y = 1.0 - (t.screen_y() - win_rect.y) / float(win_rect.h)
else:
    y = 1.0 - (t.screen_y() - win_rect.y - caption) / float(win_rect.h)

Definitely a bit dirty, but happy with the result!

[ghost]

@jamiecollinson It does not work for me on Windows 7.1 in fullscreen. I can clearly see the gap when the touch is in the top half of the screen.

I had further investigations on the problem and here are my conclusions:
t.screen_x and t.screen_y seems to be comprised between 0 and screen resolution x or y -1.
With my config, I have a LCD with a resolution of 1680 x 1050 so, in full screen:
t.screen_x = [0 ; 1679] and t.screen_y = [0 ; 1049]
win_rect.w = 1680 and win_rect.h = 1050
If we want to have the full range for the coordinate [0 ; 1] the solution seems to be (for full screen):

x = int(t.screen_x() - win_rect.x) / float(win_rect.w - 1)
y = 1.0 - int(t.screen_y() - win_rect.y) / float(win_rect.h - 1)

The integer cast is important to avoid problems such as 1 / 1 =0.999998 or similar.

In windowed mode, this is different as we have to substract the size of caption, borders, etc.
It seems that we should use directly user32.GetClientRect instead of user32.WindowRect to have directly the coordinates and size of the usable area as explained here: http://www.firststeps.ru/mfc/winapi/72.gif

I don't succeed to use this function, the compiler / interpreter prints strange error "expected LP_RECT instance instead of pointer to RECT"... Maybe someone else will have more chance. I'm not a python developer but have some background in programming languages...

[tito]

@Vianneyz Can you share your patch even if it doesn't work? It can be a start for others peoples to contribute / help you to fix it as well

[jamiecollinson]

I'm afraid I was working on a client device and don't have access to it at the moment, so can't confirm @Vianneyz's comments on the problem when windowed. I don't recall spotting that as a problem during testing, but I was focused on how it performed when maximised.

I'm also confused by the suggestion to divide both by float(win_rect.w - 1):

• (probably a typo) shouldn't we use win_rect.h for the y co-ordinate
• why do we subtract 1? aren't win_rect.h and win_rect.w the number of pixels in each dimension rather than an absolute value?

[ghost]

@jamiecollinson Yes, you're right it's a typo error for the y coordinate. -> I edited my comment.

I substract 1 to have the coordinates [0 ; 1]. If we don't the coordinates are [0 ; 1[.
To be honnest, I don't know exactly how kivy handles the cursor and what are the min and max values for the coordinates (I guess 0 and 1, included, but...)

@tito I have no free time until January to work on that but I will try to do so.

[grmcom]

I am having the same issue on Windows 8.1 using an Acer t272hl touch monitor.
I also observed the issue on a windows 8 Dell Inspiron laptop with a touchscreen.
The offset is not present with the mouse.
Image of the lowest touch I can get on the acer touchscreen:

[grmcom]

A workaround for this issue is to add the following to config.ini.
You might have to alter the ratio and offset for your screen:

[postproc:calibration]
wm_touch = yratio=1.04,yoffset=-0.027

[tshirtman]

A calibration trick won't cut it if you want to have a portable solution, i'm having a look in the same direction as @Vianneyz , but hitting the same error, which is weird, because both fonctions have the same signature according to MS doc (https://msdn.microsoft.com/fr-fr/library/windows/desktop/ms633519(v=vs.85).aspx, https://msdn.microsoft.com/fr-fr/library/windows/desktop/ms633503(v=vs.85).aspx) and both are declared the same way in the module. Digging.

edit: the change is as simple as changing windll.user32.GetWindowRect(self.hwnd, byref(win_rect)) to windll.user32.GetClientRect(self.hwnd, byref(win_rect)) in wm_touch.py.

[grmcom]

Hi tshirtman, if I make that change I get this error:

File "C:\Users\Kivy\Desktop\Programing\Python\Kivy\Kivy 1.9-32bit\kivy27\kivy\input\providers
wm_touch.py", line 154, in update
windll.user32.GetClientRect(self.hwnd, byref(win_rect))
ctypes.ArgumentError: argument 2: <type 'exceptions.TypeError'>: expected LP_RECT instance inste
ad of pointer to RECT

[tshirtman]

Yes, this is what i get as well, i'm trying to understand what makes one call work and not the other.

[tshirtman]

this is getting confusing, seems kivy is doing something that confuses ctypes, because with this test code, the first call doesn't crash (though hwnd is 0, which is weird), but the call when i push the button does.

from kivy.app import App
from kivy.uix.button import Button
from ctypes import windll, wintypes, byref


class Test(App):
    def build(self):
        return Button(on_press=self.cb)

    def cb(self, *args):
        windll.user32.GetClientRect(hwnd, byref(r))
        print r

hwnd = windll.user32.GetActiveWindow()
print hwnd
r = wintypes.RECT()
windll.user32.GetClientRect(hwnd, byref(r))
print r
Test().run()

[tshirtman]

PR up there only fixes the originaly reported issue, that is the constant offset in the whole window when the app is fullscreen, i don't currently have a way to reproduce non-constant and x offset issues, but i think the more general solution would be to use GetClientRect, if i could find why it doesn't work.

[ghost]

@grmcom @tshirtman I remember having the same error (with GetClientRect) when I tried a few month ago to correct the error but as I have absolutely no knowledge of the kivy structure and I'm not a Python developper, It made it hard for me to solve the issue.