[patch] Screen and input corruption under xterm [non-UTF] #1668
Comments
The problem exist when current panel translation (c-x t) is set to no translation (my default) or any ISO. I don't see any connection of translating file names and displaying file contents, but apparently something here depends on something that it shouldn't... |
|
This error manifest themself only when using slang library. |
..and only when compiled with --enable-charset. |
OK, as apparently noone even took a look at this ticket so here's what I've found on my own:
2a. util.c/strip_ctrl_codes mentions http://www.xfree86.org/current/ctlseqs.html for skipping xterm's OSC (Operating System Command), where
ESC Z Return Terminal ID (DECID is 0x9a). Obsolete form of CSI c (DA).
but this stripping doesn't occour in viewer.
2b. there are is*_printable functions in util.c which use xterm_flag to exclude characters 128-154,156-159 as '"Full 8 bits output" doesn't work on xterm'. None of these functions are used either in viewer now, while one was used in view.c/display before 4.7.0pre.
and defines constant full_eight_bits (1) if charset support is enabled (it can't be disabled via full_eight_bits=0 in ini file).
There are two solutions:
The difference is that a) display dots where b) displays mostly carets - the latter is some kind of mistake, because in real string like <8D> is printed, but cursor position moves just 1 character forward, so any next overwrites second one ('8' here) and so on. |
|
|
Oops, tty_display_8bit is called from setup_pre and it's too early to use xterm_flag (always equals 0 then). Updated patch to recall when needed in init_xterm_support. |
|
Created branch 1668_io_corruption_xterm_nonutf
Initial changeset: [efe0b37d6c164fd962ae55f34847b1b0a96cc973]
I tested on 'mcview' attachment and I don't see massive screen corruption.
This branch for testing only. As I tested under koi8-r - all seems ok, but need to ask 'koi8'-native users :) |
At first just do:
cat mcview
you should end up with:
$ cat mcview
As for my patches, only slang-8bit_xterm.patch could influence somehow koi8-r users, viewer-printable_only.patch is fully safe (but I'm not sure how 'proper' in terms of mc code). And both of them address the same issue, so there's no need to use them together (unless both would be considered as proper).
BTW there is -k8/allowC1Printable xterm option since: http://cseweb.ucsd.edu/~mbtaylor/xterm.log.html#xterm_175 which addresses the problem for KOI8-R users and so 'fixes' mc behaviour - make sure you don't have it enabled for testing. |
For me this problem is temporarily fixed if I choose Options -> Display Bits and then select "OK". I don't actually have to change anything, just open that dialog and then "OK" is enough. From there it does not have the problem for the remainder of the session. So I don't know if this "OK" initializes something differently than what mc does on startup?
I use ISO 8859-1 and do not use "Full 8 bits support".
GNU Midnight Commander 4.7.0.2 |
Created 1668_screen_corruption branch.
In this changeset, the call of 'Display bits' dialog is emulated. |
Guys, it seems to me that you are quite crazy :) To me [97967701ff7f7a776b47d571c7db42fc646991f0] looks like a horrible kludge. Is there any particular reason why you can't just follow gotar's approach with is_printable() checks, he did a great job at figuring out the true reason for this breakage... |
Replying to zaytsev:
Ok, forget this. :) In any case, this bug is not reproducable for me at all. |
Of course you cannot reproduce this bug! You said that you are using KOI8-R as a system locale:
Replying to gotar:
Before or after applying the patch you will not notice anything at all if the patch is correct. To reproduce this we need someone using single-byte locale where these symbols are unprintable.
I think we need to investigate branch 1668_io_corruption_xterm_nonutf (changeset: [efe0b37d6c164fd962ae55f34847b1b0a96cc973]) which seems to contain a correct fix. However what has to be tested is whether it breaks anything in UTF-8 mode. That's it. |
Gotar, your patches are S-Lang-related only. Does this bug not happen in NCurses-based MC. Or you didn't check that? |
If I build with ncurses it works fine for me and I can't reproduce this issue. |
Replying to andrew_b:
The bug doesn't occour when using ncurses library - see 4. for explanation. This is purely either wrong Slang setting (SLsmg_Display_Eight_Bit) OR missing simple pre-filtering out 154-char. Slang approach filters every 128-160 which is almost identical to 128-154,156-159 filtered out by is*_printable (see 2b), so unless there is some legitimate character in this range there's no difference which solution is chosen.
In short: ncurses handles 154 properly, while Slang requires SLsmg_Display_Eight_Bit>=154 (OR manual filtering if xterm display detected - but checking TERM would fail under screen, OR running xterm with allowC1Printable). |
OK, since Paul's workaround actually works, we probably should investigate the SLsmg_Display_Eight_Bit route? For me, actually, it only fails under screen so filtering does work for xterm, it's just that the terminals that have different TERM from xterm are failing. |
|
|
|
I'm not sure if it is directly related to this bug, or it's something completely different.
Midnight Commander, Slackware build (all versions built in distro slack-11.0 at least).
Sometimes repeated presses of ESC key gets around the problem. |
1) mc -V |
Replying to andrixnet:
Just check test file (the 2-bytes one) or workarounds or proposed patches. It looks you have hit the same problem, except for:
it can't happen on local console. |
Replying to angel_il:
There is no real hangup, one just can't exit from search dialog as it pops back over and over again every time he presses esc or enter.
please
By any means. Thank you. |
andrixnet: |
gotar: can you reproduce issue with 0x9A on current master or 4.7.5 ? |
Replying to angel_il:
Yes, I've just compiled 4.7.5.1 tarball (./configure with no options except paths) and the problem still exists. The only difference I've noticed is that it happens after F9 (ViewToggleNroffMode) is switched to the state where keybar shows '9 Unform' (i.e. to the place where my patch shows carets). |
Oh, my. I am ashamed I was unaware of this : the mc build I use most often is from aug 2008. No version number though. |
|
branch: 1668_io_corruption_xterm_nonutf (parent) |
|
ok, i don't know how to reproduce this, my vote here... |
Well, seems we initialize S-Lang incorrectly. As John E. Davis wrote in http://mailman.jedsoft.org/pipermail/slang-users-l/2011/000724.html, SLutf8_enable has to be called first before any of the other functions.
Gotar, would you try the following patch: |
I did (over 4.5.7.1 tarball) and it doesn't fix the problem (and doesn't break anything neither as far as I can see). I doubt this one could be solved via anything at terminfo level, so unless SLang gets native xterm (and screen?) output fiter it needs to be done like [fe32d549dfd711e37eeeb4402c2a7cb6ae4ec681] (please verify if second chunk doesn't break KOI8-R output). |
Replying to gotar:
It doesn't. |
|
|
|
Merged to master. |
Hi guys!
Please confirm that everything works for you on latest master and nothing is broken as a consequence of this patch. I'd love to cherry-pick it to stable, but I'm the only from the dev team who is able to reproduce it, so it's holding us back... |
cherry-picked in stable:
|
Important
This issue was migrated from Trac:
gotar(gotar@….pl)arekm@….pl(@arekm),paul.hartman+gentoo@gmail.com(paul.hartman+gentoo@….com),zaytsev(@zyv),andrixnet@….comLANG=
LC_CTYPE=pl_PL
LC_COLLATE=pl_PL
LC_[*]="POSIX"
LC_ALL=
Input / display codepage (i.e. display_codepage): ISO-8859-2
source_codepage (set via 'Fix it'): ISO-8859-2
However the same happens with any display_codepage different from 7-bit ASCII or UTF-8 and any (non-UTF at least) locale setting.
The problem:
viewing binary files leads to massive screen corruption and Search dialog pops up with 1;2c search string (multiple times depending on actual screen contents). So it looks like the file 'presses' F7 or / and shift-right_arrow for every specified character combination occurrence. In case of bigger files it's impossible to exit from such viewer, as search dialog keeps popping up after closing.
No such behaviour on linux console with any circumstances.
I've minimized the problem to 2-byte file: 0x9A 0x14 (attached).
GNU Midnight Commander 4.7.0-pre3
Virtual File System: tarfs, extfs, cpiofs, ftpfs, fish, mcfs
With builtin Editor
Using system-installed S-Lang library with terminfo database
With subshell support as default
With support for background operations
With mouse support on xterm and Linux console
With support for X11 events
With internationalization support
With multiple codepages support
Data types: char 8 int 32 long 32 void * 32 off_t 64 ecs_char 8
Note
Original attachments:
gotar(gotar@….pl) onOct 4, 2009 at 11:06 UTCgotar(gotar@….pl) onNov 7, 2009 at 15:37 UTCgotar(gotar@….pl) onNov 7, 2009 at 16:11 UTCThe text was updated successfully, but these errors were encountered: