Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Entire Abbreviation Not Deleted Before Expansion #819

Open
4 of 16 tasks
andonagio opened this issue Mar 24, 2023 · 26 comments
Open
4 of 16 tasks

Entire Abbreviation Not Deleted Before Expansion #819

andonagio opened this issue Mar 24, 2023 · 26 comments
Labels
autokey-gtk bug phrase expansion Issues related to Phrases

Comments

@andonagio
Copy link

andonagio commented Mar 24, 2023

AutoKey is a Xorg application and will not function in a Wayland session. Do you use Xorg (X11) or Wayland?

Xorg

Has this issue already been reported?

  • I have searched through the existing issues.

Is this a question rather than an issue?

  • This is not a question.

What type of issue is this?

Bug

Choose one or more terms that describe this issue:

  • autokey triggers
  • autokey-gtk
  • autokey-qt
  • beta
  • bug
  • critical
  • development
  • documentation
  • enhancement
  • installation/configuration
  • phrase expansion
  • scripting
  • technical debt
  • user interface

Other terms that describe this issue if not provided above:

No response

Which Linux distribution did you use?

Xubuntu 22.04

Which AutoKey GUI did you use?

GTK

Which AutoKey version did you use?

0.96

How did you install AutoKey?

From the *.deb package downloaded from GitHub

Can you briefly describe the issue?

I use phrase expansions where the abbreviation is deleted and then Autokey uses the clipboard (Ctrl + V) to paste the expansion. Sometimes it works perfectly, but other times it fails to delete the first character of the abbreviation. For instance, if I type

hth

I'll get either

Hope that helps

or

hHope that helps

It's a toss-up. I'm guessing that either a) there's a race condition where Autokey sometimes tries to paste in the expansion before the last delete, or b) the last delete is not getting properly sent to the application (maybe the deletes are being sent in too quick succession for the application to properly register each one?). Most of the related issues on GitHub seems to involve scripts, but I'm just using plain expansions with Ctrl + V.

Can the issue be reproduced?

Sometimes

@josephj11
Copy link
Contributor

josephj11 commented Mar 24, 2023

What's the target window application? Some apps are really slow. Does this occur in a terminal or a plain text editor (low latency apps)? Pasting with the clipboard (as you're already doing) usually fixes this.

Can you try using autokey-qt to see if that makes a difference?

If you're doing everything right (and it looks like you are), then you'll have to try a script where you can add delays and even send one character at a time in the worst case.

To debug this, try running an AutoKey trace to make sure AutoKey is sending the correct output.

@josephj11 josephj11 added bug phrase expansion Issues related to Phrases autokey-gtk labels Mar 24, 2023
@andonagio
Copy link
Author

andonagio commented Mar 24, 2023

I predominantly use it when writing emails in the Outlook web interface in Firefox. I did a few test runs in said app. Here's the terminal output from one of the expansions that exhibited the aforementioned problem:

2023-03-24 19:22:17,278 DEBUG - autokey.service - Raw key: 'q', modifiers: [], Key: q
2023-03-24 19:22:17,278 DEBUG - autokey.service - Window visible title: 'Mail - XXXXXXX - Outlook — Mozilla Firefox', Window class: 'Navigator.firefox'
2023-03-24 19:22:17,279 DEBUG - autokey.service - Input queue at end of handle_keypress: deque(['q'], maxlen=150)
2023-03-24 19:22:17,373 DEBUG - autokey.service - Raw key: 'o', modifiers: [], Key: o
2023-03-24 19:22:17,373 DEBUG - autokey.service - Window visible title: 'Mail - XXXXXXX - Outlook — Mozilla Firefox', Window class: 'Navigator.firefox'
2023-03-24 19:22:17,375 DEBUG - autokey.service - Input queue at end of handle_keypress: deque(['q', 'o'], maxlen=150)
2023-03-24 19:22:17,498 DEBUG - autokey.service - Raw key: 'c', modifiers: [], Key: c
2023-03-24 19:22:17,498 DEBUG - autokey.service - Window visible title: 'Mail - XXXXXXX - Outlook — Mozilla Firefox', Window class: 'Navigator.firefox'
2023-03-24 19:22:17,498 DEBUG - autokey.service - Input queue at end of handle_keypress: deque(['q', 'o', 'c'], maxlen=150)
2023-03-24 19:22:17,748 DEBUG - autokey.service - Raw key: <Key.ENTER: '<enter>'>, modifiers: [], Key: Key.ENTER
2023-03-24 19:22:17,748 DEBUG - autokey.service - Window visible title: 'Mail - XXXXXXX - Outlook — Mozilla Firefox', Window class: 'Navigator.firefox'
2023-03-24 19:22:17,749 INFO - autokey.service - Matched Phrase "please-know-questions" having abbreviations "['qoc', 'qc']" against current input
2023-03-24 19:22:17,750 DEBUG - autokey.iomediator.iomediator - Send via clipboard
2023-03-24 19:22:17,750 DEBUG - autokey.service - Input queue at end of handle_keypress: deque([], maxlen=150)
2023-03-24 19:22:17,751 DEBUG - autokey.interface - Sending string via clipboard: Please let me know if you have any questions or concerns.
2023-03-24 19:22:17,752 DEBUG - autokey.interface - Sending via clipboard enqueued.
2023-03-24 19:22:17,753 DEBUG - autokey.interface - Send special key: [<Key.BACKSPACE: '<backspace>'>]
2023-03-24 19:22:17,755 DEBUG - autokey.interface - Send special key: [<Key.BACKSPACE: '<backspace>'>]
2023-03-24 19:22:17,757 DEBUG - autokey.interface - Send special key: [<Key.BACKSPACE: '<backspace>'>]
2023-03-24 19:22:17,759 DEBUG - autokey.interface - Send special key: [<Key.BACKSPACE: '<backspace>'>]
2023-03-24 19:22:17,767 DEBUG - autokey.iomediator.iomediator - Send via event interface
2023-03-24 19:22:17,768 DEBUG - autokey.interface - Send modified key: modifiers: ['<ctrl>'] key: v

I tried a plain text editor (in this case, Mousepad). The expansions seem to work fine there. I also tried Outlook web in Chromium and could not reproduce the problem, but I was able to reproduce it in a private Firefox window with extensions turned off, so maybe it's something specific to Firefox.

I get the same intermittent problem in Outlook/Firefox using the qt version.

Is there a way to add delays to an expansion?

@josephj11
Copy link
Contributor

This is one of our old arch Nemeses, slow applications (especially Firefox and LibreOffice).

Yep, but ... See #566. Until that gets implemented, the only alternative is writing a script (see link in my prior comment).

@andonagio
Copy link
Author

Interesting. Why Firefox, specifically? I don't see why it would be slower than the Chromium-based browsers.

I'll take a look at the links and at writing a script. Thanks!

@josephj11
Copy link
Contributor

You'll have to ask that green dragon. :) (*)

I'm speaking from years of empirical evidence, not from any knowledge of internals.

Our wiki has a ton of scripting examples, API examples and other useful information. And you can always post on Gitter if you have anything to discuss with us or would like some assistance with scripting.

  • The Mozilla mascot

@aldoxvr
Copy link

aldoxvr commented Apr 21, 2023

I think I found the root of the problem and that it also affects when information uses the
function interface.py->__sendString() inside a command line.

The autokey system works asynchronously but does not follow an established sequence,
I added a screencast where you can see that the string that is being sent does not do so
completely and this depends on the clock cycles of each cpu.

vokoscreenNG-2023-04-21_10-19-10.webm

I also added a piece of code where I thought it could solve it and waited for it to finish
sending the entire chain to move on to the interface.py->__sendKey() function.

    def __sendString(self, string):
        .
        .
        .
        logger.debug("Finish send keys")
        time.sleep(1)
        self.flg_finish = True

    def __sendKey(self, keyName):
        while not self.flg_finish:
            time.sleep(0.2)
        logger.debug("Send special key: [%r]", keyName)
        self.__sendKeyCode(self.__lookupKeyCode(keyName))
        self.flg_finish = True

I added the script it was trying to run as well as the lines of code it modified with no
successful result.

  import time
  keyboard.send_keys("ssh-add -l<enter>")
  time.sleep(1)
  cmd = '''find /tmp/ -type s -perm 775 -name agent.\* -printf "%-25p %TY-%Tm-%Td %TH:%TM\n"  2> /dev/null | sort<enter>'''
  keyboard.send_keys(cmd)
  time.sleep(3)
  keyboard.send_keys("export SSH_AUTH_SOCK=/tmp/ssh-")

Unfortunately I don't have the time to be able to apply the patch where the system must send
entire string that is being ordered and then move on to send de special command. I hope you can
solve it with this information.

@josephj11
Copy link
Contributor

To preface this: I have minimal Python coding skills (but I have done a lot in other languages) and I don't understand most of the AutoKey codebase. (I am not an AutoKey developer.)

I certainly hope you have something here. This is something we run into all the time.
It also affects our other asynchronous API calls - notably those interacting with the clipboard.

However, I'm having a lot of difficulty understanding your explanatory comments and I'm not sure where your code fits (in interface.py?).

Just looking at the code snippet above, nothing sets self.flg_finish to false.

Your while loop continues until self.flg_finish becomes true, but subsequent code sets it to true again anyway.

Since the while loop doesn't affect the setting of self.flg_finish, it must be assigned in another concurrent thread, etc., but I don't see that variable at all in the current interface.py source.

What am I missing?

BTW, a test branch was created a year or two ago with long delays added. This helped with this problem, but did not entirely solve it. However, it made AutoKey embarrassingly slow and clunky, almost to the point of unusability.

@Elliria
Copy link
Contributor

Elliria commented Apr 22, 2023

I think you're right that there needs to be a False in there, @josephj11. If my understanding of the code is correct, then the description below is how it works, but I could be misunderstanding the code, because I'm still very much a Python n00b, so don't necessarily hang your hat on my nail.

The send_string function currently looks like this:

    def send_string(self, string):
        self.__enqueue(self.__sendString, string)

This would get bolted onto the end of it:

        logger.debug("Finish send keys")
        time.sleep(1)
        self.flg_finish = True

This is what the send_string function would look like when that's done:

    def send_string(self, string):
        self.__enqueue(self.__sendString, string)
        logger.debug("Finish send keys")
        time.sleep(1)
        self.flg_finish = True

What that would do is to log a comment, pause for a moment, and then set the value of the self.flg_finish variable to the Boolean True value.

That's not all, though. We then would move on to the sendKey function, which currently looks like this:

    def __sendKey(self, keyName):
        logger.debug("Send special key: [%r]", keyName)
        self.__sendKeyCode(self.__lookupKeyCode(keyName))

This would get bolted onto the beginning of it:

        while not self.flg_finish:
            time.sleep(0.2)

And this would get bolted onto the end of it:

        self.flg_finish = True

This is what the sendKey function would look like when that's done:

    def __sendKey(self, keyName):
        while not self.flg_finish:
            time.sleep(0.2)
        logger.debug("Send special key: [%r]", keyName)
        self.__sendKeyCode(self.__lookupKeyCode(keyName))
        self.flg_finish = True

What that would do is to run a loop that pauses for a moment if the value of the self.flg_finish variable is not True, then move on only when it is, then log a comment, then send the keycode, and then set the value of the flg_finish variable to True.

It should work, but I think the reason it isn't working yet is that that final line of code in the sendKey function should be setting the value of the self.flg_finish variable to False so that it's not True again until after the next action is completed. I would think that if it's never set to False, then the condition is always good for it to send the next character, which is how those extra unwanted characters are sneaking in. So, perhaps the sendKey function should look like this instead:

    def __sendKey(self, keyName):
        while not self.flg_finish:
            time.sleep(0.2)
        logger.debug("Send special key: [%r]", keyName)
        self.__sendKeyCode(self.__lookupKeyCode(keyName))
        self.flg_finish = False

What that would do is to run a loop that pauses for a moment if the value of the self.flg_finish variable is not True, then move on only when it is, then log a comment, send the keycode, and then set the value of the flg_finish variable to False.

That should reset the Boolean value on each run. I haven't tested it, so this is just a theory.

@aldoxvr
Copy link

aldoxvr commented Apr 22, 2023

That's right, I'm missing some lines of code, but it doesn't work at all anyway.

Let's put it this way.

Code script

# Abbreviation: ";gtg"
# Text expansion
txtexp = "Grettings from Guanajuato Mexico<enter>"
keyboard.send_keys(txtexp)

I should expect this after type de abbreviation and trigger the expansion.

case1

# Grettings from Guanajuato Mexico

But I get this.

case2

# Grettings from Gua
# najuato Mexico

Another case is this.

case3

# ;gtGrettin from Guanajuato Mexico

And de worst scenario is when want to expand in apps like Libreoffice, Firefox

case4

# reGris Tohjl
# ff

The last scenario already happens in older versión of Ubuntu.

But after upgrade the distro got worse, and is because the function doesn't finish at all and
go to the next function.

It happens randomly and with the new X11 or WM that maybe optimize code or something.

It doesn't work put a lot of sleep because we rely on the cpu clock or something, the solution that the expanded text should run in sequence.

I think in some approximation but as i said, i don't have much time to debug and test, but i maybe take a look during the weekend.

@Elliria
Copy link
Contributor

Elliria commented Apr 22, 2023

Since you pointed out that it's happening only in certain programs, some of which we've seen others have the same issue in, I think that, as @josephj11 originally said up above, the best solution for this is to use the Function to Type Text Slowly when using those programs. You could write the script in such a way that it checks which program is open and only runs that function when it's writing to one of those programs and, otherwise, sends the text normally.

@aldoxvr
Copy link

aldoxvr commented Apr 22, 2023

I'ts not only in certain programs, impact in all the programs and it's depends of the hardware of the workstation, i know that @josephj11 do a great function but it is a band aid, the program should type as fast as a human be. I will search more deeper and try to find the root of the problem.

@Elliria
Copy link
Contributor

Elliria commented Apr 22, 2023

Okay, and we appreciate it. You're very welcome here and we'll do our best to help.

@josephj11
Copy link
Contributor

I don't believe that just adding delays will fix this. It helps, but it's not perfect. Luzifeius created a branch to test that a few years ago and it didn't work.

What I don't see is how that while loop ever terminates unless self.flg_finish is always true. If it indeed gets set to false before that loop, I don't see it getting set to true anywhere while that loop is running.

@Elliria
Copy link
Contributor

Elliria commented Apr 23, 2023

The send_string function finishes running by setting it to True. Nothing sets it to False, though, as you noticed. The while loop in the __sendKey function is just there to continue setting a delay if the send_string function hasn't finished yet. Once the send_string function finishes and sets the value to True, that satisfies the while loop and lets you get past it. The problem is that once the __sendKey function's while loop is happy and steps aside, the __sendKey function tries to set the value to True, which does nothing since it was already True thanks to the send_string function. What needs to happen, I think, is for the __sendKey function to set it to False so that those two functions can play nicely with one another.

@andonagio
Copy link
Author

I switched from Firefox Snap to Firefox Flatpak a few days ago, and interestingly, Autokey seems to work better with the latter. (In fact, Firefox in general seems to run more smoothly under Flatpak.) I haven't had this issue come up since. I'll continue to keep an eye on it, but it makes me wonder if the Snap packaging had anything to do with this. I'm still hoping for a general fix in a future Autokey release.

@josephj11
Copy link
Contributor

#566 might help with this.

Technical nit: I believe the new phrase text is emitted before the trigger abbreviation is removed, so the title here is (understandably) incorrect.

@Elliria
Copy link
Contributor

Elliria commented May 30, 2023

#566 looks like it would help.

The titles were added automatically by GitHub when I typed in the issue numbers. It's some sort of new feature. I realize now how that happened. When GitHub pops up the selector after you type in an issue number, if you hit Enter, it adds the highlighted text. If you just ignore it and continue on about your business, it just puts in the issue number. Good to know.

@josephj11
Copy link
Contributor

@Elliria Not your links - the overall issue title, but that feature is good to know about.

@m-guggenmos
Copy link

In my case using the following script solves the issue of non-deleted abbreviation characters:

def my_send_keys(abbr, string):
  for _ in range(len(abbr)):
    keyboard.send_keys("<backspace>")
  keyboard.send_keys(string)
  return

my_send_keys("trigger-string","expanded-string")

Note that you have to replace "trigger-string" and "expanded-string" with your respective strings.

@josephj11
Copy link
Contributor

@m-guggenmos Your code should only work if you have deselected the Remove typed abbreviation option. Normally that does almost the same thing as your code, but when applications can't process keystrokes quickly enough, the number of abbreviation characters left behind usually varies instance by instance.

@m-guggenmos
Copy link

@josephj11 Oh this is a good point. I actually hadn't deselected this option and hence the reason my entire abbreviation got deleted before expansion was because in this way twice the number of Backspace strokes was issued. This obviously might delete too many characters if there is other text before the abbreviation (I hadn't tested this).
I might have to play around with time delays after all...

@andonagio
Copy link
Author

Update: while this problem occurs far less frequently in Firefox flatpak vs snap, it does manifest itself occasionally. Also, a new problem has arisen: I paste the expansion via the Ctrl + v option, and occasionally in Firefox, what gets pasted is the pre-existing text already on the clipboard rather than the expanded text from Autokey. It seems that Firefox is registering Ctrl + v too early or too late in the substitution process.

@josephj11
Copy link
Contributor

@andonagio Firefox and LibreOffice are the two slowest apps. When using AutoKey with them, use scripts instead of phrases so you can add delays between each step. Worst case, use my type-slowly() function.

@andonagio
Copy link
Author

@andonagio Firefox and LibreOffice are the two slowest apps. When using AutoKey with them, use scripts instead of phrases so you can add delays between each step. Worst case, use my type-slowly() function.

I'm open to scripting, but your method only seems to make sense if I'm pasting using the keyboard rather than pasting using Ctrl + v. Is there a way to slow down the process of moving the expansion to the clipboard, hitting Ctrl + v, then moving the old clipboard contents back?

@josephj11
Copy link
Contributor

Yeah... The clipboard operations themselves can't be throttled, but you can add delays between them - and you usually have to anyway because clipboard calls are asynchronous, but return success immediately. Usually a time.sleep(0.1) does the trick, but you can try longer delays.

For most people, the big difference between pasting and typing is that typing can't easily use multibyte characters (Unicode...) and pasting can. Applications can usually handle one paste event quickly, but get bogged down with fast multiple keystroke events. With typing you can delay between each character if necessary. See this for how to type Unicode characters.

Sometimes I use insanely long delays (2.0 or more) while debugging a script so I can see what it does one step at a time and see where it's going wrong.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
autokey-gtk bug phrase expansion Issues related to Phrases
Projects
None yet
Development

No branches or pull requests

5 participants