Text Expansion #55

Open
Thr1ve opened this Issue Jul 30, 2015 · 32 comments

Comments

Projects
None yet
9 participants
@Thr1ve

Thr1ve commented Jul 30, 2015

Would there be any way to do text expansion with robotjs like you can with AutoHotkey?

I was able to get robotjs working with electron so I could listen for global keypresses, but electron (and nw.js) only seems to be able to listen for key combinations, not key sequences.

It would be amazing if we could simply turn any key-presses / mouse-movements into events and/or observables so we could deal with them purely in node-land ( and perhaps use something fancy like http://reactivex.io/ so we could just map/filter/etc the keypresses )

Robotjs is/will be great for sending keyboard commands etc., but I think we'd need a reliable way to listen for these events globally to come close to AutoHotkey. Is there perhaps a node module I haven't found that strictly listens for keypresses / mouse movements globally and outputs them as events ? Surely someone must have written something like this already.

@fohara

This comment has been minimized.

Show comment
Hide comment
@fohara

fohara Jul 30, 2015

@Thr1ve I haven't seen a node module, but there is libuiohook , which could probably be wrapped with nan (or something), similar to how robotjs itself is designed.

fohara commented Jul 30, 2015

@Thr1ve I haven't seen a node module, but there is libuiohook , which could probably be wrapped with nan (or something), similar to how robotjs itself is designed.

@mwagstaff

This comment has been minimized.

Show comment
Hide comment
@mwagstaff

mwagstaff Jul 30, 2015

+1. Built-in text expansion capabilities similar to AutoHotkey would be a massive benefit.

+1. Built-in text expansion capabilities similar to AutoHotkey would be a massive benefit.

@octalmage

This comment has been minimized.

Show comment
Hide comment
@octalmage

octalmage Jul 30, 2015

Owner

I generally use nw.js for hotkeys but as you mentioned its lacking. I'm hoping they improve that because I don't think I want to include hotkey support in RobotJS. If I do work on it, it will be a separate module. I think I'm going to do the same for window management.

Owner

octalmage commented Jul 30, 2015

I generally use nw.js for hotkeys but as you mentioned its lacking. I'm hoping they improve that because I don't think I want to include hotkey support in RobotJS. If I do work on it, it will be a separate module. I think I'm going to do the same for window management.

@octalmage

This comment has been minimized.

Show comment
Hide comment
@octalmage

octalmage Jul 30, 2015

Owner

@Thr1ve but RobotJS can get the mouse position currently.

Owner

octalmage commented Jul 30, 2015

@Thr1ve but RobotJS can get the mouse position currently.

@octalmage

This comment has been minimized.

Show comment
Hide comment
@octalmage

octalmage Jul 30, 2015

Owner

@fohara thanks for the link! I'll try this library and if it works well, I'll turn it into a node module. It would be very easy.

Owner

octalmage commented Jul 30, 2015

@fohara thanks for the link! I'll try this library and if it works well, I'll turn it into a node module. It would be very easy.

@fohara

This comment has been minimized.

Show comment
Hide comment
@fohara

fohara Jul 30, 2015

@octalmage excellent. I was able to compile and run the demohook example (./configure --enable-demo) from the repo on OS X and it seemed to work well.

fohara commented Jul 30, 2015

@octalmage excellent. I was able to compile and run the demohook example (./configure --enable-demo) from the repo on OS X and it seemed to work well.

@octalmage

This comment has been minimized.

Show comment
Hide comment
@octalmage

octalmage Jul 31, 2015

Owner

Awesome! Can't wait to try it out.

Owner

octalmage commented Jul 31, 2015

Awesome! Can't wait to try it out.

@Thr1ve

This comment has been minimized.

Show comment
Hide comment
@Thr1ve

Thr1ve Jul 31, 2015

@octalmage Completely agree on the separate module thing; sorry if I implied otherwise! In my mind, there would be a really simple, small, and separate module that would solely listen for any keyboard / mouse events and then emit them as a events / streams / observables / whatever.
Second, there would be a layer ( also separate from robotjs; totally with you re: keeping hotkeys separate ) that could recognize key combinations / sequences and send the appropriate commands to robotjs. I would think we could easily bend/hack something like Mousetrap to work for this?

@fohara Thanks! Can't say I know how to make that work, but it looks like a good time to learn!

I've been dreaming of a javascript version of AutoHotkey for so long... major props to @octalmage for what he's done so far 👍

Thr1ve commented Jul 31, 2015

@octalmage Completely agree on the separate module thing; sorry if I implied otherwise! In my mind, there would be a really simple, small, and separate module that would solely listen for any keyboard / mouse events and then emit them as a events / streams / observables / whatever.
Second, there would be a layer ( also separate from robotjs; totally with you re: keeping hotkeys separate ) that could recognize key combinations / sequences and send the appropriate commands to robotjs. I would think we could easily bend/hack something like Mousetrap to work for this?

@fohara Thanks! Can't say I know how to make that work, but it looks like a good time to learn!

I've been dreaming of a javascript version of AutoHotkey for so long... major props to @octalmage for what he's done so far 👍

@octalmage

This comment has been minimized.

Show comment
Hide comment
@octalmage

octalmage Jul 31, 2015

Owner

Gotcha! I for sure will to try to replace each AHK feature with Node.js modules. If libuiohook works out we'll have a great start for key listening. I'll keep y'all updated!

Owner

octalmage commented Jul 31, 2015

Gotcha! I for sure will to try to replace each AHK feature with Node.js modules. If libuiohook works out we'll have a great start for key listening. I'll keep y'all updated!

@octalmage

This comment has been minimized.

Show comment
Hide comment
@octalmage

octalmage Aug 1, 2015

Owner

@fohara did you encounter this error?

hook_run [882]: Accessibility API is disabled!
Failed to enable access for assistive devices.

I tried adding it to the list of allowed accessibility apps, but that isn't helping.

Owner

octalmage commented Aug 1, 2015

@fohara did you encounter this error?

hook_run [882]: Accessibility API is disabled!
Failed to enable access for assistive devices.

I tried adding it to the list of allowed accessibility apps, but that isn't helping.

@fohara

This comment has been minimized.

Show comment
Hide comment
@fohara

fohara Aug 1, 2015

@octalmage ah yes, I did encounter that accessibility error. My short term solution was to add iTerm (or any terminal app) to the allowed list. There is probably a more intuitive way of granting permission for this, but I haven't looked into it yet.

fohara commented Aug 1, 2015

@octalmage ah yes, I did encounter that accessibility error. My short term solution was to add iTerm (or any terminal app) to the allowed list. There is probably a more intuitive way of granting permission for this, but I haven't looked into it yet.

@octalmage

This comment has been minimized.

Show comment
Hide comment
@octalmage

octalmage Aug 1, 2015

Owner

Ah gotcha. This system is so silly. I had to remove and re-add iterm2.

Owner

octalmage commented Aug 1, 2015

Ah gotcha. This system is so silly. I had to remove and re-add iterm2.

@fohara

This comment has been minimized.

Show comment
Hide comment
@fohara

fohara Aug 1, 2015

@octalmage yeah, there must be a programmatic way to handle this. I remember seeing this before, I think with Sikuli perhaps. There was some flag that prompted permission for accessibility, I'll see if I can find it.

fohara commented Aug 1, 2015

@octalmage yeah, there must be a programmatic way to handle this. I remember seeing this before, I think with Sikuli perhaps. There was some flag that prompted permission for accessibility, I'll see if I can find it.

@octalmage

This comment has been minimized.

Show comment
Hide comment
@octalmage

octalmage Aug 1, 2015

Owner

This tool does it, we can look at the code:

https://github.com/jacobsalmela/tccutil

But yeah, libuiohook is so perfect. Just need to create an EventEmitter in V8 and we'll be good to go. Looking into this now.

Owner

octalmage commented Aug 1, 2015

This tool does it, we can look at the code:

https://github.com/jacobsalmela/tccutil

But yeah, libuiohook is so perfect. Just need to create an EventEmitter in V8 and we'll be good to go. Looking into this now.

@octalmage

This comment has been minimized.

Show comment
Hide comment
@octalmage

octalmage Aug 1, 2015

Owner

It looks like emitting events isn't very straight forward but I figured it out. Woohoo!

Owner

octalmage commented Aug 1, 2015

It looks like emitting events isn't very straight forward but I figured it out. Woohoo!

@fohara

This comment has been minimized.

Show comment
Hide comment
@fohara

fohara Aug 1, 2015

@octalmage nice! Looking forward to seeing how the event emitting works for this.

fohara commented Aug 1, 2015

@octalmage nice! Looking forward to seeing how the event emitting works for this.

@octalmage

This comment has been minimized.

Show comment
Hide comment
@octalmage

octalmage Aug 7, 2015

Owner

So I've made good progress on this, but the library outputs the copyright when it's included:

Do you think I should contact the author to see if it's ok to remove it? Or just remove it? I don't want to release the project with this copyright notice, it's ugly.

Owner

octalmage commented Aug 7, 2015

So I've made good progress on this, but the library outputs the copyright when it's included:

Do you think I should contact the author to see if it's ok to remove it? Or just remove it? I don't want to release the project with this copyright notice, it's ugly.

@fohara

This comment has been minimized.

Show comment
Hide comment
@fohara

fohara Aug 7, 2015

@octalmage good news on the progress! Regarding the copyright notice, you should be able to suppress it by building the lib with the --enable-quiet config, if I'm understanding this correctly.

fohara commented Aug 7, 2015

@octalmage good news on the progress! Regarding the copyright notice, you should be able to suppress it by building the lib with the --enable-quiet config, if I'm understanding this correctly.

@octalmage

This comment has been minimized.

Show comment
Hide comment
@octalmage

octalmage Aug 7, 2015

Owner

@fohara oh duh, thanks!

Owner

octalmage commented Aug 7, 2015

@fohara oh duh, thanks!

@octalmage

This comment has been minimized.

Show comment
Hide comment
@octalmage

octalmage Oct 15, 2015

Owner

I ran into a block with the c++ EventEmitter. I'm going to rewrite the library with a getKeyState type function. Then I'll figure out the events.

Owner

octalmage commented Oct 15, 2015

I ran into a block with the c++ EventEmitter. I'm going to rewrite the library with a getKeyState type function. Then I'll figure out the events.

@IvoPereira

This comment has been minimized.

Show comment
Hide comment
@IvoPereira

IvoPereira Feb 19, 2016

Any updates on this @octalmage?

Any updates on this @octalmage?

@octalmage

This comment has been minimized.

Show comment
Hide comment
@octalmage

octalmage Feb 19, 2016

Owner

This isn't a priority at the moment, I wouldn't expect it anytime soon. Sorry!

Owner

octalmage commented Feb 19, 2016

This isn't a priority at the moment, I wouldn't expect it anytime soon. Sorry!

@Venryx

This comment has been minimized.

Show comment
Hide comment
@Venryx

Venryx Jul 28, 2016

Looking for a replacement for AutoHotkey, and this is saddening... : (

  1. For now, I think I'm going to try using edge.js (http://tjanczuk.github.io/edge)) to embed C# code, which'll hook into the windows dll's for keyboard input, and trigger nodejs callbacks when they're pressed. (This should also be nice in case I want to migrate over more os-involved operations from my autohotkey scripts, like window manipulation)

  2. If that doesn't work/is too difficult, I'll see if I can use node-ffi (https://github.com/node-ffi/node-ffi) to interface with the windows dll's directly.

  3. If that doesn't work, I'll probably just use the input listener system that's already being used for this other nodejs automation repo: https://github.com/relipse/JsRoboKey

(Actually, I might just try using #3 to begin with; depends mostly on whether I can find some easy C# code to capture global hotkeys and send them back to nodejs.)

Anyway, I'll post back here with my results! (Since this is a pretty significant feature for an automation system)

P.S. I realize the approaches above (#1 and #2 anyway) would be Windows-only solutions, so would probably not be wanted as a part/listed-dependency for this project. And that you already have something of a solution using nw.js. But I figured it'd be helpful to at least provide one path of adding hotkeys for other Windows users browsing these threads and not seeing a way explained yet. (I looked briefly at the nw.js project, and didn't see the way you captured hotkeys using it)

Venryx commented Jul 28, 2016

Looking for a replacement for AutoHotkey, and this is saddening... : (

  1. For now, I think I'm going to try using edge.js (http://tjanczuk.github.io/edge)) to embed C# code, which'll hook into the windows dll's for keyboard input, and trigger nodejs callbacks when they're pressed. (This should also be nice in case I want to migrate over more os-involved operations from my autohotkey scripts, like window manipulation)

  2. If that doesn't work/is too difficult, I'll see if I can use node-ffi (https://github.com/node-ffi/node-ffi) to interface with the windows dll's directly.

  3. If that doesn't work, I'll probably just use the input listener system that's already being used for this other nodejs automation repo: https://github.com/relipse/JsRoboKey

(Actually, I might just try using #3 to begin with; depends mostly on whether I can find some easy C# code to capture global hotkeys and send them back to nodejs.)

Anyway, I'll post back here with my results! (Since this is a pretty significant feature for an automation system)

P.S. I realize the approaches above (#1 and #2 anyway) would be Windows-only solutions, so would probably not be wanted as a part/listed-dependency for this project. And that you already have something of a solution using nw.js. But I figured it'd be helpful to at least provide one path of adding hotkeys for other Windows users browsing these threads and not seeing a way explained yet. (I looked briefly at the nw.js project, and didn't see the way you captured hotkeys using it)

@octalmage

This comment has been minimized.

Show comment
Hide comment
@octalmage

octalmage Jul 28, 2016

Owner

Check out:

https://github.com/electron/electron/blob/master/docs/api/global-shortcut.md

https://github.com/nwjs/nw.js/wiki/shortcut

I use both of these in different apps and they work great for my purpose!

Owner

octalmage commented Jul 28, 2016

Check out:

https://github.com/electron/electron/blob/master/docs/api/global-shortcut.md

https://github.com/nwjs/nw.js/wiki/shortcut

I use both of these in different apps and they work great for my purpose!

@WilixLead

This comment has been minimized.

Show comment
Hide comment
@WilixLead

WilixLead Dec 10, 2016

Hello guys. I just implement libuiohook for node.js.
Now we can hook all keyboard and mouse events globally.
This module also support electron and nw.js
https://www.npmjs.com/package/iohook

PS. It is cross platform module already tested in Mac and Ubuntu. Should good work in windows and other systems, what supported from libuiohook

Hello guys. I just implement libuiohook for node.js.
Now we can hook all keyboard and mouse events globally.
This module also support electron and nw.js
https://www.npmjs.com/package/iohook

PS. It is cross platform module already tested in Mac and Ubuntu. Should good work in windows and other systems, what supported from libuiohook

@Venryx

This comment has been minimized.

Show comment
Hide comment
@Venryx

Venryx Apr 26, 2017

Thanks for making the iohook library WilixLead!

Unfortunately, I'm getting an error trying to use it: WilixLead/iohook#16

Venryx commented Apr 26, 2017

Thanks for making the iohook library WilixLead!

Unfortunately, I'm getting an error trying to use it: WilixLead/iohook#16

@Venryx

This comment has been minimized.

Show comment
Hide comment
@Venryx

Venryx Apr 26, 2017

I also tried this very similar package: https://github.com/ayoubserti/node-iohook

But it also had an error, while installing. In that case, it's because I didn't have the Visual C++ Build Tools installed. I didn't feel like restarting, and installing the build tools, to try the module out. (at least while other options existed)

Which brings me to my third attempt: https://github.com/tomzx/gkm

This one finally works! However, it's not preferred, since it uses Java; but for Windows users, it's probably the easiest option atm. (at least, if you can't get the iohook or node-iohook libraries working)

Venryx commented Apr 26, 2017

I also tried this very similar package: https://github.com/ayoubserti/node-iohook

But it also had an error, while installing. In that case, it's because I didn't have the Visual C++ Build Tools installed. I didn't feel like restarting, and installing the build tools, to try the module out. (at least while other options existed)

Which brings me to my third attempt: https://github.com/tomzx/gkm

This one finally works! However, it's not preferred, since it uses Java; but for Windows users, it's probably the easiest option atm. (at least, if you can't get the iohook or node-iohook libraries working)

@Venryx

This comment has been minimized.

Show comment
Hide comment
@Venryx

Venryx Apr 27, 2017

It would be nice to have a Wiki page listing various libraries that work well with robotjs, for achieving various automation functions.

If it's okay with @octalmage, I've gone ahead and created the page here: https://github.com/octalmage/robotjs/wiki/Plugins-(or-compatible-modules)

I plan to create some modules myself later, as I'd like to eventually replace AutoHotkey entirely. (AutoHotkey is nice, but it's scripting language is terrible)

Venryx commented Apr 27, 2017

It would be nice to have a Wiki page listing various libraries that work well with robotjs, for achieving various automation functions.

If it's okay with @octalmage, I've gone ahead and created the page here: https://github.com/octalmage/robotjs/wiki/Plugins-(or-compatible-modules)

I plan to create some modules myself later, as I'd like to eventually replace AutoHotkey entirely. (AutoHotkey is nice, but it's scripting language is terrible)

@mviens

This comment has been minimized.

Show comment
Hide comment
@mviens

mviens Sep 14, 2017

Venryx,

This is also a goal of mine. Love what AHK can do, but hate AHK's language. Plus version 2 has been in development for 6+ years. And even if/when that is released, it does not really fix the language issues, but mostly remove the horrible Command syntax in favor of function() syntax.

I have wanted to use JS for years to replace my product which is written in AHK. I plan to use NodeJS, TypeScript, Electron and Aurelia, but since I have a full-time job this conversion goes very slowly. I would really enjoying partnering with anyone here on replacing AHK with a cross-platform equivalent.

If interested, please contact me directly at mikeviens@gmail.com.

mviens commented Sep 14, 2017

Venryx,

This is also a goal of mine. Love what AHK can do, but hate AHK's language. Plus version 2 has been in development for 6+ years. And even if/when that is released, it does not really fix the language issues, but mostly remove the horrible Command syntax in favor of function() syntax.

I have wanted to use JS for years to replace my product which is written in AHK. I plan to use NodeJS, TypeScript, Electron and Aurelia, but since I have a full-time job this conversion goes very slowly. I would really enjoying partnering with anyone here on replacing AHK with a cross-platform equivalent.

If interested, please contact me directly at mikeviens@gmail.com.

@Venryx

This comment has been minimized.

Show comment
Hide comment
@Venryx

Venryx Sep 15, 2017

Hi mviens.

I have the same goal, however I've come to believe it's best to build such a system incrementally, based on the foundation here in robotjs. This is because robotjs already has a userbase, and the language/build-process is good, so we can just build up modules around it.

For example, I plan to make a module which creates a system-icon at the bottom-right of screen, from which the user can start, stop, or edit a given script file.

I plan to make another which adds the ability to query, move, and otherwise control various windows. So for example, an easy way to check the title of the active window, minimize it, etc.

We could build a new system from the ground up, but because the NodeJS module system is so flexible (and because gaining traction for a new software system is so hard), I think it's best to just build on this project here (robotjs) and transform it into the full-fledged software we want.

By the way, node-ffi is very convenient for hooking into native dll functions. You can do so entirely in JS land! (even callbacks; I've used it to implement a full-fledged JS keyboard hook, which I now use everyday for global hotkeys)

Venryx commented Sep 15, 2017

Hi mviens.

I have the same goal, however I've come to believe it's best to build such a system incrementally, based on the foundation here in robotjs. This is because robotjs already has a userbase, and the language/build-process is good, so we can just build up modules around it.

For example, I plan to make a module which creates a system-icon at the bottom-right of screen, from which the user can start, stop, or edit a given script file.

I plan to make another which adds the ability to query, move, and otherwise control various windows. So for example, an easy way to check the title of the active window, minimize it, etc.

We could build a new system from the ground up, but because the NodeJS module system is so flexible (and because gaining traction for a new software system is so hard), I think it's best to just build on this project here (robotjs) and transform it into the full-fledged software we want.

By the way, node-ffi is very convenient for hooking into native dll functions. You can do so entirely in JS land! (even callbacks; I've used it to implement a full-fledged JS keyboard hook, which I now use everyday for global hotkeys)

@mviens

This comment has been minimized.

Show comment
Hide comment
@mviens

mviens Sep 15, 2017

What does this project (https://github.com/octalmage/robotjs) plan on doing differently than the already built/working project (https://github.com/robot/robot-js)? It seems like the other project has not been updated in a while, does appears to be well thought out and capable.

mviens commented Sep 15, 2017

What does this project (https://github.com/octalmage/robotjs) plan on doing differently than the already built/working project (https://github.com/robot/robot-js)? It seems like the other project has not been updated in a while, does appears to be well thought out and capable.

@hallss93

This comment has been minimized.

Show comment
Hide comment
@hallss93

hallss93 Feb 9, 2018

Use RobotJS to troubleshoot Electron's GlobalShortcut.
Use shortcuts for numerals (0-9) but as keys not issued plus values when pressed.
A solution to instantiate the keyTap shortly after an event capture:
globalShortcut.register ('6', () => { that.logger = "" + that.logger + 6 robot.keyTap ("numpad_6"); })

So a solution to use as a Global Shortcut! :)

hallss93 commented Feb 9, 2018

Use RobotJS to troubleshoot Electron's GlobalShortcut.
Use shortcuts for numerals (0-9) but as keys not issued plus values when pressed.
A solution to instantiate the keyTap shortly after an event capture:
globalShortcut.register ('6', () => { that.logger = "" + that.logger + 6 robot.keyTap ("numpad_6"); })

So a solution to use as a Global Shortcut! :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment