Thunderbird plain text and html editor #455

sphh · 2020-09-16T12:56:04Z

I just noticed, that espanso behaves differently in Thunderbird's plain text and html editor.

I need multiline expansions, so according to #285 (comment) I have:

name: Thunderbird

filter_class: "Thunderbird"
backend: Clipboard
paste_shortcut: CtrlShiftV

matches:
  - trigger: ":kr"
    replace: |-
      Kind regards
      Max

I get the following results:

Plain message editor:

(Yes, nothing)

HTML message editor:

Kind regards
Max

  <body>
    Kind regards<br>
    Max<br>
  </body>

With

filter_class: "Thunderbird"
backend: Clipboard

(paste_shortcut: CtrlShiftV commented out)
I get the following results:

Plain message editor:

Kind regards
Max

HTML message editor:

Kind regards Max

  <body>
    Kind regards
    Max
  </body>

And with just

filter_class: "Thunderbird"

(backend: Clipboard also commented out):

Plain message editor:

Kind regards
Max

HTML message editor:

Kind regards

Max

  <body>
    <p>Kind regards</p>
    <p>Max<br>
    </p>
  </body>

I tried to differentiate the two message editors by their window properties, but no luck, because espanso detect returns the same for both:

Detected change, current window has properties:
==> Title: 'Write: (no subject) - Thunderbird - Max Mustermann <mm@mannomann.com>'
==> Class: 'Thunderbird'
==> Executable: ''

Any idea how to get a consistent behaviour for both editors?

The text was updated successfully, but these errors were encountered:

federico-terzi · 2020-09-16T16:58:56Z

Hey @sphh,
Thank you! That finally makes sense :) I was wondering why some users were able to use that workaround while others couldn't...

Any idea how to get a consistent behaviour for both editors?

That's a good question. Unfortunately, I don't have any idea to solve the problem at the moment.
Hopefully, when #214 gets implemented, this problem will be solved as well (as you could set up a match as a rich text match).

Cheers :)

sphh · 2021-01-28T10:01:27Z

I had a fresh look at this issue using the latest version 0.7.3.

I removed the entry for the backend, now using the default:

filter_class: "Thunderbird|TeXstudio"
#backend: Clipboard

Looks like espanso now inserts the replacement both into the html and plain text editors! That's very good!

But I noticed the following, when playing with markdown and html replacements: They work fine in the html editor, but if I use them in Thunderbird's plain text editor, I get the following:

matches:
  - trigger: ":kr"
    markdown: |-
      Kind regards
      Max

becomes in the plain text editor:

楋摮爠来牡獤䴠硡

(AI at work translating it on the fly to Chinese?)

The same for

matches:
  - trigger: ":kr"
    html: |-
      Kind regards<br />
      Max

⇒

楋摮爠来牡獤戼⁲㸯䴊硡

federico-terzi · 2021-01-28T18:50:26Z

Hey @sphh,

Thank you for the investigation!

But I noticed the following, when playing with markdown and html replacements: They work fine in the html editor, but if I use them in Thunderbird's plain text editor, I get the following:

That's interesting, does it happen on every application or in Thunderbird only?

If xclip could support multiple paste formats (last time I searched it wasn't clear how to do it), we could instruct espanso to copy both the Text and HTML versions in the clipboard and let the text editor decide which one was best suited.

Cheers :)

sphh · 2021-01-28T19:58:02Z

Good question regarding other applications!
Here are my results:

replace:
- works in TB html editor
- works in TB plain text editor
- works in Xed and the terminal
markdown: works in TB html editor
- works in TB html editor
- gives Chinese characters in TB plain text editor
- removes the trigger, but does not insert anything

Have you asked at https://github.com/astrand/xclip/ to support multiple targets?

These bug reports might be related: astrand/xclip#93 astrand/xclip#94

federico-terzi · 2021-02-02T12:12:05Z

@sphh Thank you for the detailed investigation! This information will be very useful.

Have you asked at https://github.com/astrand/xclip/ to support multiple targets?

No, I haven't. Besides the fact that the project does not seem very active, I'm investigating possible alternatives to xclip, as I'm currently working on a major refactor. Hopefully, we will be able to fix that issue and remove the xclip dependency :)

sphh · 2021-02-02T12:28:04Z

Thanks for taking care of this. Looking already forward towards the refactored program!

federico-terzi · 2021-02-02T12:33:12Z

@sphh Thanks, I'll make sure to create a tracking issue in the next few days so that we can all keep track of the refactoring

sphh · 2021-12-31T10:16:04Z

Thanks for the continuing work on this program!

I upgraded to 2.1.2-alpha and tested this again.

I created an application specific config file for Thunderbird:

filter_class: "Thunderbird"
backend: Auto
#backend: Clipboard
#backend: Inject

and dis/enabled the different backends (edited on the fly with Espanso notifying me, that the configuration was reloaded as soon as I save the file). All of them show the same behaviour:

The text is correctly expanded in a terminal, in CudaText, Spyder, Firefox, … and in Thunderbird HTML message edit windows.
The match is deleted, but no expansion inserted into Thunderbird plain message edit windows.

federico-terzi · 2022-01-01T10:51:30Z

Hey @sphh,

Thanks for investigating! The v2 comes with a built-in patch for thunderbird, which might be interfering with your experiment: https://github.com/federico-terzi/espanso/blob/master/espanso/src/patch/patches/linux/thunderbird_x11.rs#L37-L40

One thing you might try is editing the app-specific config for thunderbird you created, adding this line:

paste_shortcut: "SHIFT+INSERT"
backend: Clipboard

Let me know if that helps :) Also you might want to type #acfg# inside Thunderbird. That will show you the active configuration being used by Espanso. If you post it here, that might help debug this further

Cheers :)

sphh · 2022-01-01T12:07:44Z

Thanks.

I tested it with this configuration (.config/espanso/config/thunderbird.yml):

filter_class: "Thunderbird"
paste_shortcut: "SHIFT+INSERT"
backend: Clipboard
extra_includes:
  - "../match/_correspondance.yml"

#detect# returns

title: 'Write: (no subject) - Thunderbird'
exec: '/usr/lib/thunderbird/thunderbird'
class: 'Thunderbird'

and #acfg#:

[espanso config: "/home/user/.config/espanso/config/thunderbird.yml (PATCHED: thunderbird_x11)"]

backend: Clipboard
enable: true
paste_shortcut: Some("CTRL+SHIFT+V")
inject_delay: Some(15)
key_delay: Some(15)
apply_patch: true
word_separators: [" ", ",", ".", "?", "!", "\r", "\n", "\u{16}"]

preserve_clipboard: true
clipboard_threshold: 100
disable_x11_fast_inject: false
pre_paste_delay: 100
paste_shortcut_event_delay: 10
toggle_key: None
auto_restart: true
restore_clipboard_delay: 300 
backspace_limit: 5
search_trigger: Some("jkj")
search_shortcut: Some("ALT+SPACE")
keyboard_layout: None

show_icon: true
show_notifications: true
secure_input_notification: true

x11_use_xclip_backend: false
win32_exclude_orphan_events: true
win32_keyboard_layout_cache_interval: 2000

match_paths: [
    "/home/user/Data/dotConfig/espanso/match/_correspondance.yml",
    [...]
]

As you can see, paste_shortcut is not "SHIFT+INSERT", but the file _correspondance.yml is loaded, so I guess config/thunderbird.yml is picked up correctly.

federico-terzi · 2022-01-02T20:12:01Z

@sphh Thanks! Ok, that looks correct

As you can see, paste_shortcut is not "SHIFT+INSERT", but the file _correspondance.yml is loaded, so I guess config/thunderbird.yml is picked up correctly.

You're right! That's because the patch overrides the custom config you specified. To avoid that, you can add this option to your thunderbird.yml:

apply_patch: false

Let me know if that helps :)

sphh · 2022-01-02T20:33:41Z

Thanks.

With apply_patch: false #acfg# shows paste_shortcut: Some("SHIFT+INSERT"), but now I get the following behaviour:

Expansion in the plain message editor works as expected.
In the HTML message editor, the current clipboard is inserted instead of the expansed phrase (and there is a long delay between removing the shortcut and inserting the clipboard).

This behaviour is independent of the setting of the paste_shortcut.

But if I switch back to backend: Auto, everything is working fine.

My working config/thunderbird.yml is now:

filter_class: "Thunderbird"
apply_patch: false
backend: Auto
extra_includes:
  - "../match/_correspondance.yml"

and #acfg#:

[espanso config: "/home/user/.config/espanso/config/thunderbird.yml"]

backend: Auto
enable: true
paste_shortcut: None
inject_delay: None
key_delay: None
apply_patch: false
word_separators: [" ", ",", ".", "?", "!", "\r", "\n", "\u{16}"]

preserve_clipboard: true
clipboard_threshold: 100
disable_x11_fast_inject: false
pre_paste_delay: 100
paste_shortcut_event_delay: 10
toggle_key: None
auto_restart: true
restore_clipboard_delay: 300 
backspace_limit: 5
search_trigger: Some("jkj")
search_shortcut: Some("ALT+SPACE")
keyboard_layout: None

show_icon: true
show_notifications: true
secure_input_notification: true

x11_use_xclip_backend: false
win32_exclude_orphan_events: true
win32_keyboard_layout_cache_interval: 2000

match_paths: [
    "/home/user/Data/dotConfig/espanso/match/_correspondance.yml",
    [...]
]

federico-terzi · 2022-01-06T18:51:06Z

@sphh Thanks! That's interesting

But if I switch back to backend: Auto, everything is working fine.

So if you keep Auto, everything works as expected on both the Plain and HTML editors? If that's the case, we might need to remove the thunderbird patch :)

sphh · 2022-01-06T19:30:55Z

So if you keep Auto, everything works as expected on both the Plain and HTML editors?

Yes.

If that's the case, we might need to remove the thunderbird patch :)

I wouldn't be surprised, if it depends on the version of Thunderbird :( The version I currently use is 78.14.0 (64-bit) (on Linux Mint 20.2).

I just noticed another problem with expansions, but this problem shows in Thunderbird HTML editor and xed, Linux Mint's editor from the XApplication family. I could not identify any other editor showing this behaviour:

Sometimes the expansion phrase is deleted, but no expanded phrase inserted (I believe this happens with expanded phrases consisting of just one single character).
Sometimes the expansion phrase is deleted and the current contents of the clipboard is inserted (after a very long wait of some seconds).
Sometimes everything works as it should (I believe it works, if there is cursor position specified in the expanded phrase).

Is this the behaviour, the thunderbird patch try to resolve? If not, I am happy to open a separate bug.

federico-terzi · 2022-01-10T18:09:39Z

@sphh Thanks!

I wouldn't be surprised, if it depends on the version of Thunderbird :( The version I currently use is 78.14.0 (64-bit) (on Linux Mint 20.2).

I've originally added the patch due to this issue: #351, it seems I mistakenly added the paste_shortcut override even if it wasn't needed, so I'll try removing that first

Regarding the other problems you mentioned, perhaps slowing down the injection a bit might help:

inject_delay: 15
key_delay: 15

Cheers :)

…d patch. Fix #455

sphh · 2022-01-10T20:32:29Z

Unfortunately that does not help:

I can remove the key_delay: 15 (I even tried 30) and the phrase to be expanded still gets removed.

It does not matter, if I have inject_delay: 15 (or 30) in my configuration, it still inserts the actual contents of the clipboard and not the expanded phrase.

And I noticed something else: I have Thunderbird message editor and a terminal open (to edit Espanso's config file). I type ;copyright (which should be become a ©) in the Thunderbird message editor. As soon as I finish that phrase, it gets deleted and the contents of the clipboard inserted, but not ©. Then I switch to the terminal and press Backspace and suddenly ;copyright appears in the terminal …

BTW, sometimes the expansion works and it works fast. But whenever the current contents of the clipboard gets inserted as described above, it takes a while for it to appear.

EDIT: Detailed the behaviour of what gets inserted and added the section about the delay.

federico-terzi · 2022-01-11T19:07:40Z

@sphh I see, thanks!

Another thing you might try is increasing the pre_paste_delay to something like 500 or more. From the docs:

And also the restore_clipboard_delay option:

For more info: https://espanso.org/docs/next/configuration/options/#options-reference

Then I switch to the terminal and press Backspace and suddenly ;copyright appears in the terminal …

This is interesting, how do you switch to the terminal window? (using the mouse, keyboard shortcut, something else?). Pressing backspace immediately after an expansion "reverts" it due to the undo_backspace feature, which can be disabled. The strange thing is that Espanso should invalidate the "undoability" as soon as some actions occur, including changing the window... So this is likely a bug :)

sphh · 2022-01-11T20:04:27Z

Thank you for your patience! I just added pre_paste_delay: 500 to the Thunderbird specific configuration. It is working for now, nevertheless I will observe it.

I switch to the terminal window by mouse: I use focus follows mouse, so I just move the mouse pointer over the terminal and it gets the input focus. (BTW with the expansion working for now, I do not get the behaviour described).

sphh · 2022-01-12T13:01:30Z

Sorry, the all clear came to early. I again experience this behaviour!

To recap:

It does not happen in the plain text editor.
It does happen in the HTML editor.
It does not happen in the plain text HTML editor (add-on ThunderHTMLedit).

I increased all timeouts to very large values:

filter_class: "Thunderbird"
apply_patch: false
backend: Clipboard
pre_paste_delay: 1000
key_delay: 300
inject_delay: 300
restore_clipboard_delay: 1000
extra_includes:
  - "../match/_correspondance.yml"

federico-terzi · 2022-01-14T21:37:40Z

@sphh Thank you for the details!

I switch to the terminal window by mouse: I use focus follows mouse, so I just move the mouse pointer over the terminal and it gets the input focus. (BTW with the expansion working for now, I do not get the behaviour described).

This is likely the cause of the undo_backspace feature not working correctly. Espanso assumes that in order to change the active window, a user would either click with the mouse or type a key combination. Therefore, if that happens, the undo_backspace feature is invalidated. For 99% of cases, this works correctly, but definitely not in your case :)

Sorry, the all clear came to early. I again experience this behaviour!

No worries! this one is really tricky... I think we tried pretty much all the available options. One last attempt might be to try adding this configuration line to use an alternative clipboard backend (note this requires the latest 2.1.2-alpha version and xclip command being installed):

x11_use_xclip_backend: true

Cheers :)

sphh · 2022-01-14T22:43:08Z

Re the focus follow mouse: Your explanation makes absolute sense! I now tested it with and without clicking into the terminal window. I can confirm, that this behaviour only shows, if I do not click into the terminal window. But as a bonus I can switch back to the first window by pointing the mouse over it and undo_backspace the expansion!! (Now I also know, why I cannot undo the expansion after having visited another window.)

So far the x11_use_xclip_backend: true works for me in Thunderbird editor (both HTML and plain). And it works fast (the other backend had a very long delay before not replacing or replacing with the current clipboard contents).

Do you have any idea, why Thunderbird (and xed) makes such a fuss?

BTW this is now my config file for xed:

filter_class: "Xed"
x11_use_xclip_backend: true

federico-terzi · 2022-01-15T20:54:20Z

@sphh Nice!

Do you have any idea, why Thunderbird (and xed) makes such a fuss?

I have absolutely no idea :) Previous Espanso versions used xclip by default, but in the v2 I decided to migrate to the clip library, as it provides a way to work with the clipboard on X11 without launching a separate process. From my experience, for 95% of the times it's faster, but sometimes it hangs for multiple seconds with certain applications, which is clearly not optimal

The x11_use_xclip_backend option is a workaround until we can figure out a good way to solve the problem :)

sphh · 2022-01-15T21:30:12Z

Thanks for your explanation.

Now that there is a workaround (use xclip) – and if you don't want to keep it open as a reminder for an update of the clip library – , as far as I am concerned, this issue can be closed.

Put please announce – maybe with an pop-up? –, that xclip is not needed anymore, whenever the clip library can handle Thunderbird and Xed .

Thank you for your patience and support and this wonderful and very, very useful application!!!! I use it so often, that I don't notice it, because it's entirely automatic 😃

federico-terzi · 2022-01-17T18:30:55Z

@sphh Thanks! I've updated an existing issue on the clip repository: dacap/clip#27

Hopefully, we'll be able to figure something out :)

federico-terzi added the needs-triage More investigation is needed on this issue to understand the problem label Sep 16, 2020

federico-terzi added a commit that referenced this issue Jan 10, 2022

fix(core): remove paste_shortcut and backend override from Thunderbir…

85b8a90

…d patch. Fix #455

federico-terzi closed this as completed Jan 17, 2022

sphh mentioned this issue Nov 1, 2022

Incomplete or incorrect expansion of non-ASCII characters in Thunderbird #1343

Open

smeech mentioned this issue Feb 11, 2024

Still problems with German Umlaute in Thunderbird #1845

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Thunderbird plain text and html editor #455

Thunderbird plain text and html editor #455

sphh commented Sep 16, 2020

federico-terzi commented Sep 16, 2020 •

edited

sphh commented Jan 28, 2021

federico-terzi commented Jan 28, 2021

sphh commented Jan 28, 2021

federico-terzi commented Feb 2, 2021

sphh commented Feb 2, 2021

federico-terzi commented Feb 2, 2021

sphh commented Dec 31, 2021

federico-terzi commented Jan 1, 2022

sphh commented Jan 1, 2022

federico-terzi commented Jan 2, 2022

sphh commented Jan 2, 2022

federico-terzi commented Jan 6, 2022

sphh commented Jan 6, 2022

federico-terzi commented Jan 10, 2022

sphh commented Jan 10, 2022 •

edited

federico-terzi commented Jan 11, 2022

sphh commented Jan 11, 2022

sphh commented Jan 12, 2022

federico-terzi commented Jan 14, 2022

sphh commented Jan 14, 2022

federico-terzi commented Jan 15, 2022

sphh commented Jan 15, 2022

federico-terzi commented Jan 17, 2022

Thunderbird plain text and html editor #455

Thunderbird plain text and html editor #455

Comments

sphh commented Sep 16, 2020

federico-terzi commented Sep 16, 2020 • edited

sphh commented Jan 28, 2021

federico-terzi commented Jan 28, 2021

sphh commented Jan 28, 2021

federico-terzi commented Feb 2, 2021

sphh commented Feb 2, 2021

federico-terzi commented Feb 2, 2021

sphh commented Dec 31, 2021

federico-terzi commented Jan 1, 2022

sphh commented Jan 1, 2022

federico-terzi commented Jan 2, 2022

sphh commented Jan 2, 2022

federico-terzi commented Jan 6, 2022

sphh commented Jan 6, 2022

federico-terzi commented Jan 10, 2022

sphh commented Jan 10, 2022 • edited

federico-terzi commented Jan 11, 2022

sphh commented Jan 11, 2022

sphh commented Jan 12, 2022

federico-terzi commented Jan 14, 2022

sphh commented Jan 14, 2022

federico-terzi commented Jan 15, 2022

sphh commented Jan 15, 2022

federico-terzi commented Jan 17, 2022

federico-terzi commented Sep 16, 2020 •

edited

sphh commented Jan 10, 2022 •

edited