New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test_filter_commands() fails when $LANG=C.UTF-8 #3577

Closed
scolby33 opened this Issue Oct 30, 2018 · 7 comments

Comments

Projects
None yet
4 participants
@scolby33

scolby33 commented Oct 30, 2018

I am attempting to compile Vim 8.1.0500 on a Raspberry Pi running Raspbian Jessie. Running make test fails with:

Found errors in Test_filter_commands():
function RunTheTest[40]..Test_filter_commands line 18: Pattern '^\\s*helplang=\\w*$' does not match '   helplang=C.'

A git bisect was performed and found that this failure was introduced in 8.1.0495: f86db78#diff-93660a689d3188bef809a36937a9ba2bR109.

In Vim on this machine:

  • :set helplang returns helplang=C. (note the trailing period)
  • :language messages gives Current messages language: "C.UTF-8"
  • :language gives Current language: "LC_CTYPE=C.UTF-8;LC_NUMERIC=C;LC_TIME=C.UTF-8;LC_COLLATE=C.UTF-8;LC_MONETARY=C.UTF-8;LC_MESSAGES=C.UTF-8;LC_PAPER=C.UTF-8;LC_NAME=C.UTF-8;LC_ADDRESS=C.UTF-8;LC_TELEPHONE=C.UTF-8;LC_MEASUREMENT=C.UTF-8;LC_IDENTIFICATION= C.UTF-8"
  • $ echo $LANG prints C.UTF-8

From the help documentation, it seems that helplang comes from the messages language. On another machine with $LANG=en_US.UTF-8, helplang=en as expected.

I am not very familiar with the internals of Vim, but it seems that a $LANG without an underscore is being parsed improperly. I don't think this is a problem with the filter() functionality from patch 495 but a different bug revealed by the new test.

@deoxys314 ran the bisect and helped to diagnose this issue with me

@tonymec

This comment has been minimized.

tonymec commented Oct 30, 2018

I was surprised by that locale, but indeed, on my system (openSUSE 15.0) locale -a |grep ^C (where ^C is a spacing circumflex followed by a capital C) answers

C
C.utf8

(with utf8 in lowercase and with no dash, however)

Best regards,
Tony.

@scolby33 scolby33 closed this Oct 30, 2018

@scolby33 scolby33 reopened this Oct 30, 2018

@chrisbra

This comment has been minimized.

Member

chrisbra commented Oct 30, 2018

I guess we need this patch:

diff --git a/src/option.c b/src/option.c
index d1f1d1382..7934ee6ed 100644
--- a/src/option.c
+++ b/src/option.c
@@ -4256,12 +4256,15 @@ set_helplang_default(char_u *lang)
            p_hlg = empty_option;
        else
        {
-           /* zh_CN becomes "cn", zh_TW becomes "tw". */
+           // zh_CN becomes "cn", zh_TW becomes "tw".
            if (STRNICMP(p_hlg, "zh_", 3) == 0 && STRLEN(p_hlg) >= 5)
            {
                p_hlg[0] = TOLOWER_ASC(p_hlg[3]);
                p_hlg[1] = TOLOWER_ASC(p_hlg[4]);
            }
+           // C.UTF-8 becomes C
+           else if (STRLEN(p_hlg) >=2 && STRNICMP(p_hlg, "C.", 2) == 0)
+               p_hlg[1] = NUL;
            p_hlg[2] = NUL;
        }
        options[idx].flags |= P_ALLOCED;
@brammool

This comment has been minimized.

Contributor

brammool commented Oct 30, 2018

@scolby33

This comment has been minimized.

scolby33 commented Oct 30, 2018

It seems to have fixed the first problem I had, but there's a few places where helplang=C isn't accepted:

Test results:


From test_mksession.vim:
Found errors in Test_mksession_arglist():
Caught exception in Test_mksession_arglist(): Vim(set):E474: Invalid argument: helplang=C @ /usr/local/src/vim/src/testdir/Xtest_mks.out, line 6
Found errors in Test_mksession_blank_tabs():
Caught exception in Test_mksession_blank_tabs(): Vim(set):E474: Invalid argument: helplang=C @ /usr/local/src/vim/src/testdir/Xtest_mks.out, line 6
Found errors in Test_mksession_blank_windows():
Caught exception in Test_mksession_blank_windows(): Vim(set):E474: Invalid argument: helplang=C @ /usr/local/src/vim/src/testdir/Xtest_mks.out, line 6
Found errors in Test_mksession_large_winheight():
Caught exception in Test_mksession_large_winheight(): Vim(set):E474: Invalid argument: helplang=C @ /usr/local/src/vim/src/testdir/Xtest_mks_winheight.out, line 6
Found errors in Test_mksession_lcd_multiple_tabs():
Caught exception in Test_mksession_lcd_multiple_tabs(): Vim(set):E474: Invalid argument: helplang=C @ /usr/local/src/vim/src/testdir/Xtest_mks.out, line 6
Found errors in Test_mksession_winheight():
Caught exception in Test_mksession_winheight(): Vim(set):E474: Invalid argument: helplang=C @ /usr/local/src/vim/src/testdir/Xtest_mks.out, line 6
TEST FAILURE
@chrisbra

This comment has been minimized.

Member

chrisbra commented Oct 31, 2018

Ah, did_set_stringoption does not allow to have 'helplang' being set to 'C', so it complains when reading in a session file with that value. Try this patch in addition:

diff --git a/src/option.c b/src/option.c
index b58613a0c..50d10712c 100644
--- a/src/option.c
+++ b/src/option.c
@@ -6232,9 +6232,11 @@ did_set_string_option(
     /* 'helplang' */
     else if (varp == &p_hlg)
     {
-       /* Check for "", "ab", "ab,cd", etc. */
+       // Check for "", "ab", "ab,cd", etc., but allow 'C'
        for (s = p_hlg; *s != NUL; s += 3)
        {
+           if (s[0] == 'C' && s[1] == NUL)
+               break;
            if (s[1] == NUL || ((s[2] != ',' || s[3] == NUL) && s[2] != NUL))
            {
                errmsg = e_invarg;
@chrisbra

This comment has been minimized.

Member

chrisbra commented Nov 1, 2018

Hm, perhaps it is better to special case locale C to en. then we do not need to handle helplang=C in all other places where helplang is used. So I think this patch is better:

diff --git a/src/option.c b/src/option.c
index bfeeee9c2..cf4d12da3 100644
--- a/src/option.c
+++ b/src/option.c
@@ -4256,12 +4256,18 @@ set_helplang_default(char_u *lang)
            p_hlg = empty_option;
        else
        {
-           /* zh_CN becomes "cn", zh_TW becomes "tw". */
+           // zh_CN becomes "cn", zh_TW becomes "tw".
            if (STRNICMP(p_hlg, "zh_", 3) == 0 && STRLEN(p_hlg) >= 5)
            {
                p_hlg[0] = TOLOWER_ASC(p_hlg[3]);
                p_hlg[1] = TOLOWER_ASC(p_hlg[4]);
            }
+           // any C like setting becomes "en"
+           else if (STRLEN(p_hlg) >=1 && STRNICMP(p_hlg, "C", 1) == 0)
+           {
+               p_hlg[0] = 'e';
+               p_hlg[1] = 'n';
+           }
            p_hlg[2] = NUL;
        }
        options[idx].flags |= P_ALLOCED;

@chrisbra chrisbra added the patch label Nov 1, 2018

chrisbra added a commit to chrisbra/vim that referenced this issue Nov 2, 2018

Fix helplang vim#3577
Set helplang=en if locale is C

@brammool brammool closed this in dcd71cb Nov 4, 2018

chrisbra added a commit to chrisbra/vim that referenced this issue Nov 4, 2018

Fix helplang vim#3577
Set helplang=en if locale is C
@scolby33

This comment has been minimized.

scolby33 commented Nov 4, 2018

Sorry for the delay in response; I lost track of this for a few days. I see the patch has already been merged, but I can confirm Vim with patches 1-510 now builds and tests on my system with the C.UTF-8 locale. Thanks!

janlazo added a commit to janlazo/neovim that referenced this issue Nov 6, 2018

vim-patch:8.1.0510: filter test fails when $LANG is C.UTF-8
Problem:    Filter test fails when $LANG is C.UTF-8.
Solution:   Set 'helplang' to "en" for any C language. (Christian Brabandt,
            closes vim/vim#3577)
vim/vim@dcd71cb
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment