Virtual key input is blocked in Windows #4

chi3236 · 2017-01-15T14:38:25Z

We used WM_KEYINPUT function and keybd_event funtion to drive the truck automatically, but thevirtual key input with those functions is blocked. Specifically, virtual key input is working on other programs, such as notepad or ms word, but we could not handle the truck with virtual key input.

zappybiby · 2017-01-15T22:13:30Z

The fix for this is to replace the deprecated keybd_event with SendInput as described here: http://stackoverflow.com/a/41057051

I figured this out after looking at http://stackoverflow.com/a/40318692

zappybiby · 2017-01-15T23:11:00Z

Possibly something like this? ajchili@6f3dadc#diff-3644f6bac665e291284f6df8b749eda1R205

to be honest, I don't know what I'm doing... I tried though!! 👶

zappybiby · 2017-01-16T05:55:51Z

Update: Input w/ SendInput not working with my current implementation, however I still believe it is what is the issue here. ajchili@5ab03b4

Also, for the past couple hours now I have been trying to get GPUMat to work... keep getting error

OpenCV Error: The function/feature is not implemented You should explicitly call download method for cuda::GpuMat object...

chi3236 · 2017-01-16T07:18:56Z

I tried umat instead of mat for using gpu before, but there was an incompatibility problem. I gonna try it again

chi3236 · 2017-01-17T07:55:00Z

http://stackoverflow.com/questions/35138778/sending-keys-to-a-directx-game

By this Q&A, SendMessage function is not moking actual key input, and DirectX app cannot be controled by this function. Keybd_Input can be used for DirectX app but unfortunately it seems like that function is blokced by ETS2 client. We should try other methods like direct input hooking...

zappybiby · 2017-01-17T12:58:56Z

Maybe use postmessage instead of sendmessage?

From further reading on stackoverflow, SendInput should definately work. I can check using Spy++ when I get home to confirm this.

I'm thinking that using hardware scan codes might work, like they did here: http://stackoverflow.com/a/18854441

zappybiby · 2017-01-17T18:03:03Z

ajchili@b4ac41f This is my attempt at fixing input. Although I can't check to see if it works because the program outputs "go straight" once and then crashes :(

Edit: Also worth noting that we could try

ip.ki.wVk = 0; //We're doing scan codes instead
    ip.ki.dwExtraInfo = 0;

    //This let's you do a hardware scan instead of a virtual keypress
    ip.ki.dwFlags = KEYEVENTF_SCANCODE;
    ip.ki.wScan = 0x1E;  //Set a unicode character to use (A)

to use hardware scan codes instead of virtual key presses...

zappybiby · 2017-01-19T02:47:11Z

After more testing (ajchili@750a79d), I found that the program freezes after reading line 161 (cout << "go left ";). Unfortunately can't debug because it doesn't crash, just freezes after first "go straight" output...

chi3236 · 2017-01-19T10:01:17Z

What does "freeze" mean? Does program stop without crash or something? Maybe that is because of infinite loop without call stack push :(

zappybiby · 2017-01-21T07:30:59Z

Yeah, program freezes because of the issue described here: #6

Freezing means that the program stops working, but doesn't exit or return with an exception. It just hangs and eventually needs to be closed with task manager. I ended up using"-nudge" with Dr. Memory to give me some info on the problem. However I can't get it to display any useful stack trace...

In a recent commit (ajchili@6c96498), I edited the code to use hardware scan input, I'm really hoping that works. If I can fix this buffer overflow error, I think we have a good chance of having a working Windows version! Otherwise, we will need to use DirectInput

zappybiby · 2017-01-21T22:01:54Z

Still stuck on the buffer overflow error, but in the meantime I changed the input to follow the original SendMessage format you created. Basically, it goes in the original order that you created. I did change the "0x74" (F5) to "0x44" ('D'), however. I assume that was a typo, since F5 is not really relevant in changing directions.

See ajchili@bced2f5?diff=unified#diff-3644f6bac665e291284f6df8b749eda1

zappybiby · 2017-01-30T22:03:45Z

New Developments! I used Spy++ to see what inputs were being taken by ETS2.

Also, I used PE explorer to find out if ETS2 is importing any DirectX dll's. It is NOT using any DirectX dll's, and is instead using Win32 API. Which is great, since it means we will not have to account for DirectInput.

This means that keybd_event, SendMessage, and/or PostMessage should work when sending key inputs.
I confirmed this by downloading the USER32.dll (http://pastebin.com/FbBEZyUn)

Interestingly enough, SendMessage is not in there, only SendMessageW is shown. 'W' stands for Wide or Unicode.

Here is the Spy++ log when physically pressing the keys on my keyboard:

Real Keys

<000001> 001208CA P WM_KEYDOWN nVirtKey:'A' cRepeat:1 ScanCode:1E fExtended:0 fAltDown:0 fRepeat:0 fUp:0

<000002> 001208CA P WM_CHAR chCharCode:'97' (97) cRepeat:1 ScanCode:1E fExtended:0 fAltDown:0 fRepeat:0 fUp:0

<000003> 001208CA P WM_KEYUP nVirtKey:'A' cRepeat:1 ScanCode:1E fExtended:0 fAltDown:0 fRepeat:1 fUp:1

<000004> 001208CA P WM_KEYDOWN nVirtKey:'S' cRepeat:1 ScanCode:1F fExtended:0 fAltDown:0 fRepeat:0 fUp:0

<000005> 001208CA P WM_CHAR chCharCode:'115' (115) cRepeat:1 ScanCode:1F fExtended:0 fAltDown:0 fRepeat:0 fUp:0

<000006> 001208CA P WM_KEYUP nVirtKey:'S' cRepeat:1 ScanCode:1F fExtended:0 fAltDown:0 fRepeat:1 fUp:1

<000007> 001208CA P WM_KEYDOWN nVirtKey:'D' cRepeat:1 ScanCode:20 fExtended:0 fAltDown:0 fRepeat:0 fUp:0

<000008> 001208CA P WM_CHAR chCharCode:'100' (100) cRepeat:1 ScanCode:20 fExtended:0 fAltDown:0 fRepeat:0 fUp:0

<000009> 001208CA P WM_KEYUP nVirtKey:'D' cRepeat:1 ScanCode:20 fExtended:0 fAltDown:0 fRepeat:1 fUp:1

<000010> 001208CA P WM_KEYDOWN nVirtKey:'W' cRepeat:1 ScanCode:11 fExtended:0 fAltDown:0 fRepeat:0 fUp:0

<000011> 001208CA P WM_CHAR chCharCode:'119' (119) cRepeat:1 ScanCode:11 fExtended:0 fAltDown:0 fRepeat:0 fUp:0

<000012> 001208CA P WM_KEYUP nVirtKey:'W' cRepeat:1 ScanCode:11 fExtended:0 fAltDown:0 fRepeat:1 fUp:1

zappybiby · 2017-01-30T22:21:39Z

Looking at the Spy++ log, it is evident that we should be using PostMessage to send input. (See the 'P')

Using PE Explorer, I found out that ETS2 does NOT use any DirectX dll's in it's import tables. It instead uses Win32 API for input (e.g. PostMessageW, SendMessageW, and keybd_event). See bethesirius#4 (comment) Now that we know that ETS2 accepts postmessage, I am removing the SendInput stuff and replacing it with PostMessage. Unfortunately it's not working in-game, but Spy++ verifies that those inputs are being sent to the game window. The simulated keys are identical to the real keys with two exceptions: cRepeat is 0 when simulated, 1 when real, and LPARAM is 0 when simulated, given another value when real. FPS is unfortunately pretty bad, can't deal with this yet until input is working. For now I'm leaving "go straight" to have no messages sent, since it would keep spamming the keydown for A and D, causing lag.

zappybiby · 2017-01-31T00:08:59Z

Switched to PostMessage ajchili@e1bc37e

Spy++ shows that it is giving nearly identical presses when comparing real and simulated keys. I am not sure why this isn't working. Should probably increase Sleep() time, keys are being pressed far too quickly.

Simulated Keys

<003546> 001208CA P WM_KEYDOWN nVirtKey:'D' cRepeat:0 ScanCode:00 fExtended:0 fAltDown:0 fRepeat:0 fUp:0
<003547> 001208CA P WM_CHAR chCharCode:'100' (100) cRepeat:0 ScanCode:00 fExtended:0 fAltDown:0 fRepeat:0 fUp:0
<003548> 001208CA P WM_KEYUP nVirtKey:'D' cRepeat:0 ScanCode:00 fExtended:0 fAltDown:0 fRepeat:0 fUp:0
<003549> 001208CA P WM_CHAR chCharCode:'100' (100) cRepeat:0 ScanCode:00 fExtended:0 fAltDown:0 fRepeat:0 fUp:0
<003550> 001208CA P WM_KEYUP nVirtKey:'A' cRepeat:0 ScanCode:00 fExtended:0 fAltDown:0 fRepeat:0 fUp:0
<003551> 001208CA P WM_CHAR chCharCode:'97' (97) cRepeat:0 ScanCode:00 fExtended:0 fAltDown:0 fRepeat:0 fUp:0

chi3236 · 2017-02-01T15:46:09Z

Sorry for my late reply :( .
I am studying ML with Dr. Ng's lecture and working on object detection with Google Tensorflow. By the way, I pulled e1bc37e to my workstation, and I think auto steering will work if key press problem is fixed.
Thank you for your interest and I will apply object detection to my project soon. Also, linux branch and master branch will be merged in several days.

zappybiby · 2017-02-01T16:06:06Z

No problem!

You may want to make the SendInput an array. I've tried using both Scan codes and Virtual keys but it's still not working for me. Frustrating! I'm still not sure if the KEYEVENTF_SCANCODE flag is actually defined...

For example:

INPUT ip[2];
ip[0].type = INPUT_KEYBOARD;
ip[0].ki.time = 0;
ip[0].ki.wVk = 0; // scan codes
ip[0].ki.dwExtraInfo = 0;
// Releases 'D'
ip[0].ki.wScan = 0x44;
ip[0].ki.dwFlags = KEYEVENTF_KEYUP;
 
 // Releases 'A'
ip[1].ki.wScan = 0x41;
ip[1].ki.dwFlags = KEYEVENTF_KEYUP;
SendInput(2, &ip, sizeof(ip)); // Note: '&' probably not needed, already a pointer.
Sleep(100);

I think we will need to use SendInput because PostMessage does not alter LPARAM as far as I know. Also, using SendInput makes my program crash. PostMessage has great FPS.

zappybiby · 2017-02-10T18:52:11Z

Input works now: ajchili@b8746df

Thanks to Komat for suggesting to change the di8.keyboard/fusion.mouse in the controls.sii to sys.keyboard/sys.mouse.

Edit: I mean it works in that ETS2 is receiving the input and moving the truck. However, it is sending keys too fast, and adding Sleep() makes the program crash... So for now it's not working "correctly"

Edit2: Now with sys.mouse, we can change SendInput to use mouse instead, which would mimic input in linux branch.

Edit3: ajchili@efcd613 zeromemory is kind of redundant so getting rid of that.

zappybiby · 2017-02-22T17:30:39Z

We now know to use SetCursorPos to control the mouse. I think this issue can be closed now?

chi3236 · 2017-02-28T07:58:52Z

OK. Input in windows is now working. Thanks!

MatthewBerk · 2020-11-15T05:53:51Z

So I know this thread is three years old, though it helped me better understand win32api.keybd_event from Python's pywin32 package, and I hope my discoveries could help others who happen upon this thread.

I was trying to figure out the purpose of the scan code argument since I found putting 0 instead of the actual scan code of the key worked for applications like notepad and games like 7 days to die which seem to use DirectX. I came upon this thread and decided to try out the method on Euro Truck Simulator 2 and it lead to some interesting discoveries.

To keep it short:

i = "a"
win32api.keybd_event(0, win32api.MapVirtualKey(VK_CODE[i], 0), 0, 0)
# Works for some DirectX games like Euro Truck Simulator 2 but not other applications like notepad.
# Doesn't work for games like 7 days to die which is reported to use DirectX. 
sleep(3.05)
win32api.keybd_event(0, win32api.MapVirtualKey(VK_CODE[i], 0), win32con.KEYEVENTF_KEYUP, 0)

i = "w"
win32api.keybd_event(VK_CODE[i], win32api.MapVirtualKey(VK_CODE[i], 0), 0, 0)
# Works for some DirectX games like Euro Truck Simulator 2  and other applications like notepad.
# Works for games like 7 days to die and Stardew Valley.
sleep(3.05)
win32api.keybd_event(VK_CODE[i], win32api.MapVirtualKey(VK_CODE[i], 0), win32con.KEYEVENTF_KEYUP, 0)

i = "d"
win32api.keybd_event(VK_CODE[i], 0, 0,0) 
# Doesn't work for some DirectX games like Euro Truck Simulator 2  BUT works for other applications like notepad.
# Works for games like 7 days to die and Stardew Valley.
sleep(3.05)
win32api.keybd_event(VK_CODE[i], 0, win32con.KEYEVENTF_KEYUP, 0)

So, so far it seems to get the best results from win32api.keybd_event, should specify virtual key code and scan code
'win32api.keybd_event(VK_CODE[i], win32api.MapVirtualKey(VK_CODE[i], 0), 0, 0)'
Some games take vk code, some only accept scan code. So far have not found a downside to sending both.

I don't mean no downside to keybd_event, since an argument can be made due to its deprecation and documentation suggests using SendInput. Though in python at least, SendInput isn't in win32api but in a different package called ctypes
ctypes.windll.user32.SendInput. Probably different in c++ but I am working with Python for now.

Note: I have only tested this on one system, so unsure if get different results using different systems.

bethesirius added the Windows label Feb 19, 2017

chi3236 closed this as completed Feb 28, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Virtual key input is blocked in Windows #4

Virtual key input is blocked in Windows #4

chi3236 commented Jan 15, 2017 •

edited

zappybiby commented Jan 15, 2017 •

edited

zappybiby commented Jan 15, 2017

zappybiby commented Jan 16, 2017

chi3236 commented Jan 16, 2017

chi3236 commented Jan 17, 2017

zappybiby commented Jan 17, 2017

zappybiby commented Jan 17, 2017 •

edited

zappybiby commented Jan 19, 2017 •

edited

chi3236 commented Jan 19, 2017

zappybiby commented Jan 21, 2017 •

edited

zappybiby commented Jan 21, 2017

zappybiby commented Jan 30, 2017 •

edited

zappybiby commented Jan 30, 2017 •

edited

zappybiby commented Jan 31, 2017 •

edited

chi3236 commented Feb 1, 2017 •

edited

zappybiby commented Feb 1, 2017 •

edited

zappybiby commented Feb 10, 2017 •

edited

zappybiby commented Feb 22, 2017

chi3236 commented Feb 28, 2017

MatthewBerk commented Nov 15, 2020

Virtual key input is blocked in Windows #4

Virtual key input is blocked in Windows #4

Comments

chi3236 commented Jan 15, 2017 • edited

zappybiby commented Jan 15, 2017 • edited

zappybiby commented Jan 15, 2017

zappybiby commented Jan 16, 2017

chi3236 commented Jan 16, 2017

chi3236 commented Jan 17, 2017

zappybiby commented Jan 17, 2017

zappybiby commented Jan 17, 2017 • edited

zappybiby commented Jan 19, 2017 • edited

chi3236 commented Jan 19, 2017

zappybiby commented Jan 21, 2017 • edited

zappybiby commented Jan 21, 2017

zappybiby commented Jan 30, 2017 • edited

zappybiby commented Jan 30, 2017 • edited

zappybiby commented Jan 31, 2017 • edited

chi3236 commented Feb 1, 2017 • edited

zappybiby commented Feb 1, 2017 • edited

zappybiby commented Feb 10, 2017 • edited

zappybiby commented Feb 22, 2017

chi3236 commented Feb 28, 2017

MatthewBerk commented Nov 15, 2020

chi3236 commented Jan 15, 2017 •

edited

zappybiby commented Jan 15, 2017 •

edited

zappybiby commented Jan 17, 2017 •

edited

zappybiby commented Jan 19, 2017 •

edited

zappybiby commented Jan 21, 2017 •

edited

zappybiby commented Jan 30, 2017 •

edited

zappybiby commented Jan 30, 2017 •

edited

zappybiby commented Jan 31, 2017 •

edited

chi3236 commented Feb 1, 2017 •

edited

zappybiby commented Feb 1, 2017 •

edited

zappybiby commented Feb 10, 2017 •

edited