Implement keyboard and mouse hooks module (say hooks.py)

[vasily-v-ryabov]

pyHook contains C module. But hooks can be implemented with Python only (using ctypes).

See pyHook example for reference: http://stackoverflow.com/questions/4581772/using-pyhook-to-respond-to-key-combination-not-just-single-keystrokes

One more example of keyboard hook without pyHook: http://stackoverflow.com/questions/9817531/applying-low-level-keyboard-hooks-with-python-and-setwindowshookexa

Analogue in AutoIt:

HotKeySet('{F2}', '_hk1')

While 1
Sleep(100)
WEnd

Func _hk1()
Send("{CTRLDOWN}{SHIFTDOWN}t{SHIFTUP}{CTRLUP}")
EndFunc

Thanks TomashUA for the use case.

[vasily-v-ryabov]

One more Python module to look at: https://github.com/IronManMark20/hooked

[vasily-v-ryabov]

Another alternative: https://github.com/schurpf/pyhk

They are all for keyboard only. We need mouse hooks as well.

[airelil]

@vasily-v-ryabov, just to clarify. Do you mean mouse/keyboard hooks for processes started/connected to by pywinauto ? The AutoIt example above sets the hook for it's own process. It's a bit different.
In case we are hooking another process we have to check if there are special requirements for that: administrator rights, code signing...

[vasily-v-ryabov]

Low-level mouse and keyboard hooks are global. We may set it in operating system and then filter target process events in. I didn't check yet how it works for processes with different privileges.

Code signing is another interesting topic that we didn't touch at all.

[airelil]

Maybe you can put some use cases to better understand the requirements? I could think about hooking of Python interpreter process itself with a running Pywinauto script. (Similar to the specified AutoId example) This could allow to interrupt/pause/whatever the script execution in more easier manner, without trying to the catch interpreter window. However, I still don't see how it can be of use for other processes.

[vasily-v-ryabov]

Currently we have only one example using the outdated pyHook: recorder.py. Since mouse/keyboard hooks are global (i.e. system wide), we can get a window handle from any process (in theory) in the event callback. It means that our Python interpreter instance listens all processes with allowed permission (at least).

Let's imagine that you've run pywinauto script with keyboard event handler (say on {F5} key press) that calls a very long chain of actions inside a user software (managed by pywinauto from Python process). It's a kind of a shortcut for automated tasks. The script is working in the background.

In that case {F5} can be pressed in a user software window. It looks more convenient than opening Python interpreter console window and pressing {F5} there.

[airelil]

Looks like LowLevelKeyboarProc/LowLevelMouseProc is the way to go:

[airelil]

To finish my previous comment:
https://msdn.microsoft.com/en-us/library/windows/desktop/ms644985(v=vs.85).aspx
Aforementioned hooks called in the context of calling thread (good as it doesn't require a separate code in DLL and all this mess with addresses and pointers)
However as people mentioned in comments it got quite restricted with Windows 7:

1. should processed very fast
2. many restrictions by UAC

[vasily-v-ryabov]

Correct. That's all about WH_KEYBOARD_LL and WH_MOUSE_LL.
UAC restrictions belong to different privileges level of the Python and the app that is not the case. We assume to work with an app at the same privileges level.

Timing restrictions might be more serious for us. I didn't look deeper how it works in AutoIt. Maybe it has the same restrictions.

[vasily-v-ryabov]

On my laptop LowLevelHooksTimeout = 5 000 (5 seconds).

[ethanhs]

Just wanted to let you know, my hook library does support mouse hooks. I get the mouse position on each callback. You can add hotkeys (hotmice?) By setting a coordinate. It isnt perfect, as you should be able to tell whether it is inside a rectangle, but that just gets down to some math.

[vasily-v-ryabov]

Hi @ironmanmark20, thanks for pointing it out. It's not so obvious looking at the readme example.

[vasily-v-ryabov]

@ironmanmark20, the wiki says "There is no tolerance (yet) so the user must put their mouse exactly on the coordinates." Currently it's not the case we want. Better way is an ability to set a callback that receives any mouse event (at any coordinates) with the info about coordinates and mouse flags (i.e. to know which mouse button clicked, pressed down or released) as a callback params. In other words it would be nice to have everything like it's done in pyHook (but with no compilation and no dependencies for pywinauto). BTW, hooked has stricter license GPL v2 while pywinauto is under LGPL v2.1+. I'm not a licensing specialist, but having an included module with stricter license might be a potential problem.

[ethanhs]

In a couple hours i can give you what you want. I hide it to make things easier for beginners, but what you want exists in the code.

[ethanhs]

Please read the readme with this example. Also, run the other file on any computer with Python 2.7-3.5 or PyPy. How is that? The newest version (0.6) is LGPL, not GPL, so no worries there.

[airelil]

Hi @ironmanmark20, thanks for you suggestion. I tried to run your example on my Dell laptop and it didn't work straight away. It looks like I get different scan_codes on my machine, they come with huge values. You can refer to attached screenshot with my prints. However I have a more basic question. How do you see your collaboration with pywinauto ? Would you like to make it as a part of the project or you'd prefer to keep it as a separate package? Basically, we try to keep only minimal dependencies for the pywinauto project.

[ethanhs]

I would not mind if code was folded straight into the project. I can understand not wanting additional dependencies.

It is strange that you scan codes are so high. They are supposed to be pretty standardized, so unless you have a very rare (or old) keyboard (perhaps another language?) you should not be getting those codes. I based my codes on an standard 101 key English keyboard. another possibility would be a different python version or implementation. (I test on CPython, PyPy, IronPython). For example, PyPy returns long values for scan codes for some reason. Anyway, I guess we should test keyboard input automatically. Using some automation framework or something 😜 or perhaps just have a config file.

[airelil]

Cool. Plugging pyhooked into the project will really simplify the things. Regarding scan_codes, indeed, I have several input languages installed on my Win8.1 and it could be the reason. I tried the script on a VM with Win7 and with English keyboard only and it started printing LCtrl as expected but still didn't get to the foobar print though. Anyway, as for me, your module could be a good start point for the feature.

[vasily-v-ryabov]

@ironmanmark20, thanks for your efforts! We really appreciate it!
I will learn your code in more details when I have a chance.

From what I can see now we probably need to synchronize hotkeys aliases with SendKeysCtypes.py module. It means we may improve both SendKeysCtypes.py and hooked.py.

[vasily-v-ryabov]

There is interesting cross-platform package: SelfSpy.

[vasily-v-ryabov]

Reviewed the code under the gist. @ironmanmark20 did you notified about the comment? So duplicating it here just in case:

Method print_event looks too complicated, it would be great to re-structure that code. For example, PyPy and CPython branches have only one line difference, but all the code is duplicated for both.

Magic numbers like 0x00D or 0x0E is not a good practice. Pseudo-constants WH_KEYBOARD_LL and WH_MOUSE_LL should be used instead.

It's also not clear why you're using the same variable hook_id for 2 different calls to SetWindowsHookExA. I believe these 2 calls should be in different methods (when setting HotKey handler or mouse handler - separately).

low_level_handler_mouse should pass wParam and lParam to higher level callback so that the callback decides whether to get the info about coordinates and hwnd where the event happened.

[vasily-v-ryabov]

Current version is released in pywinauto 0.6.0. So closing this.