Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pasting long block from clipboard inserts wrong content for line breaks #4562

Closed
mc-butler opened this issue Jul 17, 2024 · 30 comments
Closed
Assignees
Labels
area: mcedit mcedit, the built-in text editor prio: low Minor problem or easily worked around
Milestone

Comments

@mc-butler
Copy link

Important

This issue was migrated from Trac:

Origin https://midnight-commander.org/ticket/4562
Reporter newbie-02 (newbie-02@….de)
Keywords paste, wrong, content

hi @ all, I'm new here, sorry for all I might do wrong,

short description: pasting a long block from a web-browser inserts one special block of 'foreign content' after / instead of multiple line breaks, making result unusable.

I'm aware it's a weird problem, did all I can do, persistent, thus ask for help, if not solvable already info if 'clean or bad on other systems' or hints for further track down can be helpful.

After I first noticed I tried lots of things, none helped, just now installed mc from scratch, issue persists.

System: actual Kali linux ( Debian ) on Intel ( Xeon ) hardware which works quite stable.

Reproducibility: copy some text from a webpage, as reference I marked the main text of https://randomascii.wordpress.com/2012/02/25/comparing-floating-point-numbers-2012-edition/ from 'Comparing Floating Point Numbers, 2012 Edition' to 'with all of its flaws' in actual version of Brave and copied by Ctrl-C.

If I paste into a text application, e.g. 'Mousepad', by Ctrl-V I get a meaningful display and can save the text.

Opening an empty text file with mc ( F4 ) and pasting there with Shift-Ctrl-V the pasting takes some time, with cursor flipping around on the screen, and then I get the start of the page as wanted, but starting at some point a special block of nonsense is inserted instead of most ( all? ) linebreaks. The point where the nonsense starts varies from try to try.

I'll try to attach the head of a copy from pasting into Mousepad and alternately mc.

The problem is persistent, the block is always the same, and behavior didn't change with fresh installation of mc ( ver. 4.8.31 from today - git clone https://github.com/MidnightCommander/mc.git - ./autogen.sh - sudo apt install build-dep mc ( S-Lang was missing ) - ./configure - make -j12 - sudo make install -j12 ).

As the unwanted block is part of another text on my system I think me or my system bad, as it's always the same I think must be configured / referenced somewhere, but couldn't find that block in all my files ( searched all files inc. configuration and hidden ) except in the origin and where I saved pasting tries with mc.

Even tried to work on another disk ( home instead of data ), same problem, same block.

Have been involved in two other pasting bugs, LO Calc sometimes not pasting content from other apps, was a windows 'occupied, wait and retry' problem, and GNOME Gnumeric not pasting webbrowser content, was a 'browser exports bad html' problem, IMHO none of them fits here.

Trying a workaround which there often helped, paste into pure text application, copy from there and then paste into mc doesn't work here, produces same / similar wrong content.

I'm clueless, can anybody give a hint?

To make it a good bug report, info as requested:

'search before create new ticket' - I searched 'paste' and 'wrong', 2 pages of reports, none matching my experience,

LC_MESSAGES=C mc -V:

GNU Midnight Commander 4.8.31
Built with GLib 2.78.4
Built with S-Lang 2.3.3 with terminfo database
Built with libssh2 1.11.0
With builtin Editor and Aspell support
With subshell support as default
With support for background operations
With mouse support on xterm and Linux console
With support for X11 events
With internationalization support
With multiple codepages support
With ext2fs attributes support
Virtual File Systems:

cpiofs, tarfs, sfs, extfs, ext2undelfs, ftpfs, sftpfs, shell

Data types:

char: 8; int: 32; long: 64; void *: 64; size_t: 64; off_t: 64;

LC_MESSAGES=C mc -F:

Home directory: /home/kali
Profile root directory: /home/kali

[System data]

Config directory: /etc/mc/
Data directory: /usr/share/mc/
File extension handlers: /usr/lib/mc/ext.d/
VFS plugins and scripts: /usr/lib/mc/

extfs.d: /usr/lib/mc/extfs.d/
shell: /usr/lib/mc/shell/

[User data]

Config directory: /home/kali/.config/mc/
Data directory: /home/kali/.local/share/mc/

skins: /home/kali/.local/share/mc/skins/
extfs.d: /home/kali/.local/share/mc/extfs.d/
shell: /home/kali/.local/share/mc/shell/
mcedit macros: /home/kali/.local/share/mc/mc.macros
mcedit external macros: /home/kali/.local/share/mc/mcedit/macros.d/macro.*

Cache directory: /home/kali/.cache/mc/


mc --configure-options:

'--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=${prefix}/include' '--mandir=${prefix}/share/man' '--infodir=${prefix}/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-option-checking' '--libdir=${prefix}/lib/x86_64-linux-gnu' '--runstatedir=/run' '--disable-maintainer-mode' '--disable-dependency-tracking' 'AWK=awk' 'X11_WWW=x-www-browser' '--libexecdir=/usr/lib' '--with-x' '--with-screen=slang' '--disable-rpath' '--disable-static' '--disable-silent-rules' '--enable-aspell' '--enable-vfs-sftp' '--enable-vfs-undelfs' '--enable-tests' 'build_alias=x86_64-linux-gnu' 'CFLAGS=-g -O2 -Werror=implicit-function-declaration -ffile-prefix-map=/build/reproducible-path/mc-4.8.31=. -fstack-protector-strong -fstack-clash-protection -Wformat -Werror=format-security -fcf-protection' 'LDFLAGS=-Wl,-z,relro -Wl,-z,now -Wl,--as-needed' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2'


search: did, see above,

distribution first: did fresh compile from sources,

steps: see head of this report,

expected: file like attached 'mc_paste_should.txt', note that that's only the first few paragraphs!

see instead: file like attached 'mc_paste_happening.txt', note that that's only the first few paragraphs! The issue strikes in long pasting only!

Note

Original attachments:

@mc-butler
Copy link
Author

Changed by newbie-02 (newbie-02@….de) on Jul 17, 2024 at 20:34 UTC

pasting as it should be:

@mc-butler
Copy link
Author

Changed by newbie-02 (newbie-02@….de) on Jul 17, 2024 at 20:35 UTC

paste as it happens with inserted nonsense:

@mc-butler
Copy link
Author

Changed by zaytsev (@zyv) on Jul 18, 2024 at 5:53 UTC (comment 1)

  • Resolution set to invalid
  • Status changed from new to closed
  • Milestone Future Releases deleted

I can't reproduce your problem on macOS with Apple Terminal.

In as far as I'm concerned, it looks like it's the terminal emulation application you are using that is cutting the text that is being given to mc, whereas mc itself is able to deal with any large blocks of text that are given to it.

Try a different terminal application (xterm, urxvt, ...).

@mc-butler
Copy link
Author

Changed by newbie-02 (newbie-02@….de) on Jul 18, 2024 at 7:24 UTC (comment 2)

  • Milestone set to Future Releases
  • Summary changed from pasting long block from clipboard inserts wrong content for line breaks to pasting long block from clipboard inserts wrong content for line breaks [ solved ]

[ solved ]

got it tracked down, I had a ~27kB

mc.macros

file in

~/.local/share/mc

with content like:

[editor]
kpenter=Enter:-1;InsertChar:40;InsertChar:120;InsertChar:110;InsertChar:41;InsertChar:32;InsertChar:226;InsertChar:136;InsertChar:146;Enter:-1;InsertChar:102;InsertChar:40;InsertChar:40;InsertChar:120;InsertChar:41; ... 

after removing it mc / mcedit works as expected.

No clue how this file creeped in, sorry for wasting your time, hope this topic can help others if they stumble in similar oddities. The structured work trough for this topic and a cross check on another machine helped me to find the source of problem.

Pls. close topic if / after someone added a comment about correct use of mc.macros .

@mc-butler
Copy link
Author

Changed by newbie-02 (newbie-02@….de) on Jul 18, 2024 at 7:35 UTC (comment 1.3)

Replying to zaytsev:

hello @ zaytsev,

thanks for caring, looks like we had 'mid-air collision' in commenting, pls. change status if necessary ( I'm not used to it ).

As you see in my comment:2 it's somehow a 'mc internal issue', haven't yet investigated about 'mc.macros', would appreciate if someone with knowledge about could add info here.

best regards,

newbie-02

@mc-butler
Copy link
Author

Changed by zaytsev (@zyv) on Jul 18, 2024 at 9:19 UTC (comment 4)

  • Milestone Future Releases deleted

You probably inadvertently triggered the "record macro" feature and then pasted something that got recorded as a macro.

@mc-butler
Copy link
Author

Changed by newbie-02 (newbie-02@….de) on Jul 18, 2024 at 11:00 UTC (comment 5)

hmmmmmmmmm ...

yes and no, I somtimes use Ctrl-r to re-read directory contents, when in editor that triggers macro recording, can imagine having made a mistake there.

But don't understand how that steps into pasting. IMHO macros shall be executed by Ctrl-a and a letter assigned to them. Is there some intention to run them in the process of pasting? And if so why doesn't it hit with short texts, why does it hit / start at different positions?

_Assume_ there is something weak in code, can't say what and why, but consider it inconvenient for users, 'not what they want' ... will try to reopen.

@mc-butler
Copy link
Author

Changed by newbie-02 (newbie-02@….de) on Jul 18, 2024 at 11:01 UTC (comment 6)

  • Resolution invalid deleted
  • Status changed from closed to reopened

@mc-butler
Copy link
Author

Changed by zaytsev (@zyv) on Jul 18, 2024 at 16:14 UTC (comment 7)

  • Resolution set to invalid
  • Status changed from reopened to closed

Please don't reopen unless you have a reproducer. As I said, my working assumption is that you accidentally pressed CTRL+R in the editor at some point and the pasted your stuff which got recorded as a huge macro, so there is nothing to fix here.

@mc-butler
Copy link
Author

Changed by newbie-02 (newbie-02@….de) on Jul 18, 2024 at 19:33 UTC (comment 8)

hello @ zaytsev,

reproducer: tried to boil it down.
Naturally I can't say how the macro creeped in or which start character it's assigned to.
I tried the following:

  1. starting mc without a
    mc.macro
    

    file in

    ~/.local/share/mc
    
  2. copying the website from 'Comparing Floating Point Numbers, 2012 Edition' up to 'What do you mean 'correct, or 2a. use the text I'll try to attach as 'mc_paste_test01.txt',
  3. make a new document,
  4. open it with F4,
  5. paste by Shift-Ctrl-V,

produces a copy of the file as expected.

  1. place the attached shortened macro file
    mc.macros
    

    in

    ~/.local/share/mc
    
  2. repeat steps 2. to 5. from above,

produces a file like the attached 'mc_paste_test02.txt', which has some injected parts

(xn) −
f((x))
f

think that isn't wanted?

@mc-butler
Copy link
Author

Changed by newbie-02 (newbie-02@….de) on Jul 18, 2024 at 19:34 UTC

Testfile which changes in pasting,

@mc-butler
Copy link
Author

Changed by newbie-02 (newbie-02@….de) on Jul 18, 2024 at 19:35 UTC

mc.macros file which triggers the change,

@mc-butler
Copy link
Author

Changed by newbie-02 (newbie-02@….de) on Jul 18, 2024 at 19:36 UTC

File changed in pasting

@mc-butler
Copy link
Author

Changed by zaytsev (@zyv) on Jul 19, 2024 at 7:48 UTC (comment 9)

Naturally I can't say how the macro creeped in or which start character it's assigned to.

This is the only thing that matters though.

think that isn't wanted?

With the macro that's to be expected.

@mc-butler
Copy link
Author

Changed by ossi (@ossilator) on Jul 19, 2024 at 9:59 UTC (comment 10)

With the macro that's to be expected.

no, it's not. notably, the editor already contains code that suppresses auto-indentation on pasted text. it obviously should suppress macro expansion as well.

@mc-butler
Copy link
Author

Changed by newbie-02 (newbie-02@….de) on Jul 19, 2024 at 12:10 UTC (comment 11)

Maybe I'm thinking too simple there, which is always an option, but if I want to paste something, I usually want to paste >that<, not changes.
If there is an option for combined pasting and 'prettifying' or similar, I would expect it to be documented and that I would be asked before the action.

@ zaytsev:

Naturally I can't say how the macro creeped in or which start character it's assigned to.

This is the only thing that matters though.

don't think so, consider it an accident in typing, but should somehow be communicated and / or accessible to the user that he now has a macro and in which situations it will strike.

think that isn't wanted?

With the macro that's to be expected.

can I find somewhere which type / content of macro would take which action in which situation? All I found yet was how to record, assign a key and run. Nothing about any automated action and how to design for that.

@ ossi:
agree that pasting shouldn't be changing, if it's implemented / good / wanted for special purposes it should be documented and optional.

@mc-butler
Copy link
Author

Changed by zaytsev (@zyv) on Jul 19, 2024 at 15:45 UTC (comment 12)

Well, detecting pasting is not possible reliably in a general way. We might try to use paste mode, if the terminal supports it, but I'm not sure whether we do have any code in the editor right now for that. If you're saying there is something there to suppress autoindent, then it can be extended to support macros, if it doesn't do this already. If you want to make a patch, we might try to take it in.

@mc-butler
Copy link
Author

Changed by newbie-02 (newbie-02@….de) on Jul 20, 2024 at 0:57 UTC (comment 13)

Am I right in understanding that there is something like 'automatic macro expansion' which in mc's internal editor automatically executes a replacement of the character assigned as macro start character by the steps defined for the macro?
E.g. having

[editor]
u=InsertChar:88;
x=InsertChar:85;

in mc.macros will replace every typed 'u' by a 'X' and 'x' by 'U'?
And same for contents of pasting?
Without option to switch off?

That's quite different to starting macros by Ctrl-a [assigned character].
Think that renders either macros or the internal editor useless /
only useable for special purposes.

Workaround tip for others affected: avoid macros or use another editor.

@mc-butler
Copy link
Author

Changed by ossi (@ossilator) on Jul 20, 2024 at 5:55 UTC (comment 14)

yes, the macro functionality is extremely aggressive. while it can be very convenient to assign shortcuts like ctrl-a (though the really interesting ones like ctrl-f1 don't seem to work), it seems a tad over the top that printable characters can be mapped as well.

note that the text from your webpage testcase actually contains markup; you can verify that by pasting it into libreoffice. apparently, your terminal interprets this by inserting kpenter (whatever that might be) into the stream.

@mc-butler
Copy link
Author

Changed by newbie-02 (newbie-02@….de) on Jul 20, 2024 at 8:06 UTC (comment 15)

short:

  • make automatic replacement optional by one or two switches / options for typing and pasting.
  • evaluate external changes to mc.macros on the fly.
  • split in macros vs. replacement patterns.
  • implement Ctrl-C and Ctrl-V in mc-editor for active push / pull clipboard?

IMHO defining macros as something the user can activate by a special typed! key sequence ( Ctrl-A and the assigned start character ) and then evaluating it at totally different situations ( pasting ) and in a totally different way ( replace the assigned character on its own ) is 'questionable'.

Agreeing that it's convenient and powerful for some use case to have 'automatic macro expansion' or 'automatic replacement' without needing manual activation, as well for typing as for pasting, but it's unwanted in other - most? - cases.

Thus I'd think it should have some switch to toggle between 'automatic macro expansion' or 'pure typing / pasting'. Two switches for replace on typing and replace on paste? While not implemented users may toggle by removing or renaming? 'mc.macros'.

Where it's inconvenient / inconsistent that adding macros is accounted on the fly, while changing or deleting them or even removing the mc.macros file needs restart of mc to take effect.

Evtl. it could also be meaningful to have different definitions for macros vs. automatic replacements?

AFAIK the clipboard contains different versions ( e.g. STRING, UTF8_STRING, TEXT, text/html ... ) of the copied text. Versions can be checked and partly steered by 'xclip', get: 'sudo apt install xclip', show targets: 'xclip -selection clipboard -o -t TARGETS', clear except UTF8_STRING: 'xclip -selection c -o | xclip -selection c'.

Don't know which one is selected or how it can be steered which to paste by Ctrl-Shift-V. But simple character replacement in mc hits on any version.

Would say the effect that pasting short texts with my obscure macro works clean, and after some length / content replacing steps in, replacing also in positions prior to the length / content needed to activate, is ... hard to understand.

Don't know if shell ( zsh in my case ) or terminal ( debian / XFCE Terminal Emulator in my case ) handles Shift-Ctrl-V, just learned at some point that that's the easy way to get content into mc editor as it doesn't accept Ctrl-V.

Don't know anything about 'kpenter', but think mc knows as it's evaluating it in some way?

Would say the general assigning of shortcuts / hotkeys on different system layers and in different programs becomes questionable at least when Ctrl-C is sometimes 'break' and sometimes 'copy'. But as widespread as Ctrl-C and Ctrl-V are in use for copy and paste I wouldn't bother if mc-editor would adopt that to push / pull to from clipboard instead of waiting for terminal to push something in it ... but that's for experienced people to decide what's possible / meaningful.

@mc-butler
Copy link
Author

Changed by zaytsev (@zyv) on Jul 20, 2024 at 9:39 UTC (comment 16)

kpenter is "Keypad Enter"

@mc-butler
Copy link
Author

Changed by ossi (@ossilator) on Jul 20, 2024 at 12:06 UTC (comment 17)

yes. from my side the question was what the actual meaning/code is in a terminal, as i couldn't produce it in konsole without doing research first.

xclip can be used by mc; it's just a mess and requires configuration (see #4458 & #3083).

@mc-butler
Copy link
Author

Changed by newbie-02 (newbie-02@….de) on Jul 20, 2024 at 13:56 UTC (comment 18)

hmmmmm ...

so similar had been thought in the past.

I didn't want to propose using xclip if that's any difficult, just mentioned as user tool to check the clipboard.

But having usual reaction on Ctrl-C and Ctrl-V which users intuitively expect acc. 'common user interface' would be good. Is mc restricted to terminal functions to communicate with the clipboard? Or are there other paths's available?

As some level is capable of working with 'Shift-Ctrl-C' and 'Shift-Ctrl-V' it could be possible to use Ctrl-C and Ctrl-V as hotkeys and activate / trigger the code users would start by Shift-Ctrl...?

Similar it could be 'nice' to work on 'normal selection' like Ctrl-F does, rather than to have to mark text by 'Shift-mousing'.

If someone reworks that area: placing '<------>' which mc shows in editor instead of double-TAB or eight spaces is normally unwanted by users, as well as filling rows with spaces to full screen width when marking with shift-mouse and then copying with 'Shift-Ctrl-C'. Can't tell if terminal or mc issue, just in case ...

'kpenter' if that's something special and distinguished from 'Enter' it probably shouldn't trigger in pasting?

BTW, I tried to use another editor, unsetting 'Options - Configuration - use internal edit' fell back on vim, none of 'select-editor' or manually changing ~/.selected_editor with or without new start of mc could switch to nano, only exporting 'export EDITOR=nano' before starting mc changes. 'export EDITOR=mousepad' works too, could become my favorite.

@mc-butler
Copy link
Author

Changed by andrew_b (@aborodin) on Jul 20, 2024 at 16:56 UTC (comment 19)

index 56d381df2..8f5fe8bfe 100644
--- a/src/editor/editwidget.c
+++ b/src/editor/editwidget.c
@@ -959,7 +959,7 @@ edit_callback (Widget *w, Widget *sender, widget_msg_t msg, int parm, void *data
             cb_ret_t ret = MSG_NOT_HANDLED;
 
             /* The user may override the access-keys for the menu bar. */
-            if (macro_index == -1 && edit_execute_macro (e, parm))
+            if (macro_index == -1 && !bracketed_pasting_in_progress && edit_execute_macro (e, parm))
             {
                 edit_update_screen (e);
                 ret = MSG_HANDLED;

@mc-butler
Copy link
Author

Changed by zaytsev (@zyv) on Jul 21, 2024 at 11:27 UTC (comment 20)

Do you want to commit? I approve.

@mc-butler
Copy link
Author

Changed by andrew_b (@aborodin) on Jul 21, 2024 at 16:58 UTC (comment 21)

  • Summary changed from pasting long block from clipboard inserts wrong content for line breaks [ solved ] to pasting long block from clipboard inserts wrong content for line breaks
  • Status changed from closed to reopened
  • Milestone set to 4.8.32
  • Branch state changed from no branch to on review
  • Resolution invalid deleted

Branch: 4562_mcedit_macros_paste
[e9258fc]

@mc-butler
Copy link
Author

Changed by andrew_b (@aborodin) on Jul 21, 2024 at 16:59 UTC (comment 22)

  • Status changed from reopened to accepted
  • Owner set to andrew_b

@mc-butler
Copy link
Author

Changed by zaytsev (@zyv) on Jul 21, 2024 at 18:44 UTC (comment 23)

  • Branch state changed from on review to approved
  • Votes set to zaytsev

@mc-butler
Copy link
Author

Changed by andrew_b (@aborodin) on Jul 22, 2024 at 18:37 UTC (comment 24)

  • Resolution set to fixed
  • Branch state changed from approved to merged
  • Status changed from accepted to testing
  • Votes changed from zaytsev to committed-master

Merged to master: [45c28d9].

@mc-butler
Copy link
Author

Changed by andrew_b (@aborodin) on Jul 22, 2024 at 18:39 UTC (comment 25)

  • Status changed from testing to closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: mcedit mcedit, the built-in text editor prio: low Minor problem or easily worked around
Development

No branches or pull requests

2 participants