Ruby FFI freezes when setting up a Windows Hook Procedure

[AntonSynytsia]

Ruby FFI freezes an application when activating Hook Procedure. This issue was originally closed due to the project's closure: #283

I tried this code with the latest FFI commit, but the 'freezing' issue still persists:

require 'ffi'

module HookProc

  extend FFI::Library
  ffi_lib :user32, :kernel32
  ffi_convention :stdcall

  # Hook reference
  #   http://msdn.microsoft.com/en-us/library/windows/desktop/ms632589(v=vs.85).aspx
  #
  callback :procedure, [ :int, :uint, :long ], :int
  attach_function :callNextHookEx, :CallNextHookEx, [ :pointer, :int, :uint, :long ], :int, :blocking => true
  attach_function :setWindowsHookEx, :SetWindowsHookExA, [ :int, :procedure, :uint, :uint ], :pointer
  attach_function :unhookWindowsHookEx, :UnhookWindowsHookEx, [ :pointer ], :bool
  attach_function :getCurrentThreadId, :GetCurrentThreadId, [], :int

  @hhook = nil
  @tid = getCurrentThreadId

  # KeyboardProc callback function
  #   http://msdn.microsoft.com/en-us/library/windows/desktop/ms644984(v=vs.85).aspx
  #
  KeyboardProc = Proc.new() { |nCode, wParam, lParam|
    puts wParam
    state = lParam >> 30 # 0 - onKeyDown, -1 - onKeyUp, 1 - onKeyExtended
    unhook if wParam == 0x1b and state == 0 # on escape key down

    # Pass the hook information to the next hook procedure.
    #callNextHookEx(nil, nCode, wParam, lParam)
    # May return a non-zero value to prevent the system from passing the message
    # to the target window procedure.
    1
  }

  class << self

    def hook
      return false if @hhook != nil
      @hhook = setWindowsHookEx(2, KeyboardProc, 0, @tid)
      puts 'hook called!'
      true
    end

    def unhook
      return false if @hhook == nil
      unhookWindowsHookEx(@hhook)
      puts 'unhook called!'
      @hhook = nil
      true
    end

    def status
      @hhook != nil
    end

  end # class << self

end # module HookProc

Calling HookProc.hook works fine, but after pressing any key, i.e triggering the KeyboardProc callback, the application freezes. I tried similar code with Win32-API by Daniel Berger and it worked fine, without freezing an application. I encountered this issue on Windows 7 and Windows 10, Ruby 2.0.0p247 (both win32 and x64). I'm not aware of what's causing the freeze when using Ruby FFI and hope it is resolved.

Thanks

[rawhiteside]

I tried this code on Win 10 with JRuby 1.7.21. Using the ffi that ships with that (don't know the ffi version).

I get different behavior. The hook appears to install (good return value, nothing in GetLastError), but then does nothing. No lock up for me, but the puts in the KeyboardProc never appears. everything seems normal.

[AntonSynytsia]

Did you try pressing any keyboard keys after calling HookProc.hook? If not, then that's probably why nothing happened. Another reason could perhaps be the fact that your current thread ID is not the one that emulates keyboard input. Changing thread ID to the one that emulates your keyboard input might resolve it. You should also try using win32-api version of the code, which can be found in the closed issue #283. See if it works.

[rawhiteside]

Yes, I did. They went wherever they usually do: various windows on the screen.

I was reading more about hook procs, though. I was warned that the app doing the hooking has to run a message pump, or things won't work. Did you run from a command-line app, or a windowed one? I ran mine from a little windowed one.

[AntonSynytsia]

@rawhiteside I ran myne from SketchUp application, which already has a message loop - It worked with Win32-API, but froze with FFI.