Tests: fix init.lua tests hanging #13744

mjlbach · 2021-01-13T00:00:03Z

Without this workaround, the test suite hangs for me on macOS, linux, and NixOS.

mjlbach · 2021-01-13T00:01:05Z

@dm1try do you know why this is necessary? These tests seem awfully slow.

dm1try · 2021-01-13T00:14:37Z

@mjlbach I was not able to debug this properly. I remember the test was hanging and waiting for "any" input, so I left the comment.

mjlbach · 2021-01-13T00:15:30Z

Is there any harm in adding the workaround for these tests too?

dm1try · 2021-01-13T00:19:21Z

Is there any harm in adding the workaround for these tests too?

nope (maybe a "mental" harm only for a code reader :))
though, let me re-check why this is needed in the first place.

mjlbach · 2021-01-13T00:20:19Z

What's super weird is this only started giving me issues (very) recently), and doesn't seem to affect CI.

dm1try · 2021-01-13T00:43:31Z

What's super weird is this only started giving me issues (very) recently), and doesn't seem to affect CI.

yeah, I was not able to reproduce this on my side(osx) on current master

and I realized why the "workaround" was needed in the first place: the screen on test nvim istance is small enough so the message "Conflicting configs ..." takes multiple lines and shows "Press Enter or type any command.." at the end.

so this not a workaround anymore but an expected step :)

mjlbach · 2021-01-13T00:44:51Z

Now it all makes sense. Thanks :)

mjlbach · 2021-01-13T00:46:42Z

Maybe I should remove the "why is this needed" in the tests haha. I think it's noted somewhere you need CR for small multiline messages

dm1try · 2021-01-13T00:49:33Z

Maybe I should remove the "why is this needed" in the tests haha. I think it's noted somewhere you need CR for small multiline messages

oops, I've already opened a PR :)

mjlbach · 2021-01-13T00:50:22Z

I'm going to do it in this one anyways since we need to add the additional not to fail local tests

dm1try · 2021-01-13T01:33:09Z

test/functional/core/startup_spec.lua

@@ -458,7 +458,7 @@ describe('user config init', function()

  it('loads init.lua from XDG config home by default', function()
    clear{ args_rm={'-u' }, env={ XDG_CONFIG_HOME=xconfig }}
-
+    feed('<cr>') -- confirm "Conflicting config ..." message


hmm, in this test, such message should not exist on clear setup(init.vim is not created in this context).

how did you run the tests? does test Xhome directory get removed between test runs?

I just ran with make test. Let me see if adding this back causes the test to hang. When I tested before, I had to disable this test in order for the test suite to not hang.

If I remove that one, it hangs.

I was unable to reproduce it, try to debug on your side.
for example, put some "test" expectation after feed

feed('<cr>') matches('check it!', meths.exec('messages', true))

Not sure if helpful, but this test was hanging for me as well. I tried the debugging suggestion and got the following output:

[ RUN ] user config init loads init.lua from XDG config home by default: 1020.91 ms ERR test/helpers.lua:100: Pattern does not match. Pattern: check it! Actual: E484: Can't open file $HOME/bin/share/nvim/syntax/syntax.vim stack traceback: test/helpers.lua:100: in function 'matches' test/functional/core/startup_spec.lua:463: in function <test/functional/core/startup_spec.lua:459>

I was surprised to see it was referencing $HOME/bin. I had run make CMAKE_INSTALL_PREFIX=$HOME/bin/ to build but hadn't run make install yet. Running make install fixed the hanging test. Unsure if this is working as intended.

yeah, there is certainly a problem with setup of testing environment(maybe XDG_CONFIG_HOME somehow leaked)
I'm going to take a look at this one more time.

@dm1try any news ? I am hit by this too. Error is triggered when neovim sees both an init.lua and init.vim but I haven't looked into why it finds both (leftover from previous state ? )

any news ?

@teto I checked the assumption above(about XDG_CONFIG_HOME) but didn't find anything suspicious.

Error is triggered when neovim sees both an init.lua and init.vim but I haven't looked into why it finds both (leftover from previous state ? )

if this is still reproducible , could you re-check if XDG_CONFIG_HOME points to "Xhome/Xconfig"
and is this directory removed on each test run(maybe there is some os-specific behavior and this setup does not work as expected)

also, maybe the tests are run in parallel somehow?)

Having some trouble reproducing the hangups

mjlbach · 2021-02-18T18:48:52Z

@dm1try I did some additional digging. The issue seems to be the runtime path getting polluted with entries from $HOME, and missing syntax.vim (as also noted by @fkm3)

See with my standard neovim configuration
And where I deleted ~/.local/share/nvim

Note the runtime/packpath. Note, just deleting local packpath is not sufficient to prevent the test from hanging, I believe the hang is due to this error about missing syntax.vim in runtime path.

The following modification fixes the test without feeding <cr>, but these aren't the correct directories.

  it('loads init.lua from XDG config home by default', function()
    clear{args_rm={'-u' }, env={ XDG_CONFIG_HOME=xconfig ; XDG_DATA_HOME=xconfig; VIMRUNTIME=xconfig}}

    eq(1, eval('g:lua_rc'))
    eq(init_lua_path, eval('$MYVIMRC'))
  end)

mjlbach · 2021-02-18T18:50:11Z

Sidenote, I'm also mildly concerned about the complaints about the "Failed to start server: address in use" warning.

mjlbach · 2021-02-18T19:04:28Z

test/functional/core/startup_spec.lua

@@ -441,6 +441,7 @@ describe('user config init', function()

  before_each(function()
    rmdir(xhome)
+    rmdir(xconfig)


Removing xconfig isn't necessary to fix the tests, but this seems like something we should do for consistency? @teto

why? it's a nested directory.

mjlbach · 2021-02-18T19:05:57Z

test/functional/core/startup_spec.lua

@@ -472,7 +473,7 @@ describe('user config init', function()
    end)

    it('loads custom lua config and does not set $MYVIMRC', function()
-      clear{ args={'-u', custom_lua_path }, env={ XDG_CONFIG_HOME=xconfig }}
+      clear{ args={'-u', custom_lua_path }, env={ XDG_CONFIG_HOME=xconfig ; XDG_DATA_HOME=xconfig; VIMRUNTIME=xconfig}}


What's weird is XDG_DATA_HOME should be set on line 12 in cmake set(ENV{XDG_DATA_HOME} ${BUILD_DIR}/Xtest_xdg/share)

I guess this points at INFO 2021-02-18T11:07:10.488 1012134 emsg_multiline:681: E484: Can't open file Xhome/Xconfig/syntax/syntax.vim

mjlbach · 2021-02-18T19:48:11Z

This actually seems to be an issue with not running installPhase. Using the nix flake

cmakeConfigurePhase
buildPhase
TEST_FILE="test/functional/core/startup_spec.lua" make functionaltest # hangs
installPhase
TEST_FILE="test/functional/core/startup_spec.lua" make functionaltest # works

dm1try · 2021-02-19T08:55:50Z

@dm1try I did some additional digging. The issue seems to be the runtime path getting polluted with entries from $HOME, and missing syntax.vim (as also noted by @fkm3)

good investigation
not sure about the nix issue described above, but it looks like runtimepath should be properly stubbed in test environment for excluding side-effects provided by sharing those directories with other environments/instances.

mjlbach · 2021-02-19T09:04:13Z

Closing as we know what the issue is (seems like a nix thing)

teto · 2021-02-19T21:16:19Z

I dont think it's a nix issue, the tests are looking into the install folder (I beleive CMAKE_INSTALL_PREFIX) but one should be able to run the tests without installation. We may need to better control the runtimepath in the tests.

mjlbach · 2021-02-19T21:17:56Z

Sorry, that was a bit dismissive on my part. I should have said, that the issue is exposed by nix, and we should probably address what you and I had discussed on IRC, but that isn't really about fixing the test, but fixing CMAKE_INSTALL_PREFIX so we can run tests without installing the runtimepath files into build

mjlbach changed the title ~~Tests: fix init.lua tests hanging~~ [RDY] Tests: fix init.lua tests hanging Jan 13, 2021

dm1try mentioned this pull request Jan 13, 2021

clarify the comment #13745

Closed

dm1try reviewed Jan 13, 2021

View reviewed changes

mjlbach changed the title ~~[RDY] Tests: fix init.lua tests hanging~~ Tests: fix init.lua tests hanging Jan 13, 2021

mjlbach marked this pull request as draft January 13, 2021 16:45

teto added this to the 0.5 milestone Feb 18, 2021

mjlbach commented Feb 18, 2021

View reviewed changes

Tests: fix init.lua tests hanging

21fabb3

mjlbach closed this Feb 19, 2021

teto mentioned this pull request Feb 22, 2021

pass functionaltests on nixos nix-community/neovim-nightly-overlay#95

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Tests: fix init.lua tests hanging #13744

Tests: fix init.lua tests hanging #13744

mjlbach commented Jan 13, 2021

mjlbach commented Jan 13, 2021

dm1try commented Jan 13, 2021

mjlbach commented Jan 13, 2021

dm1try commented Jan 13, 2021 •

edited

mjlbach commented Jan 13, 2021

dm1try commented Jan 13, 2021

mjlbach commented Jan 13, 2021

mjlbach commented Jan 13, 2021

dm1try commented Jan 13, 2021

mjlbach commented Jan 13, 2021

dm1try Jan 13, 2021 •

edited

dm1try Jan 13, 2021

mjlbach Jan 13, 2021

mjlbach Jan 14, 2021

dm1try Jan 14, 2021

fkm3 Feb 12, 2021

dm1try Feb 12, 2021

teto Feb 17, 2021

dm1try Feb 17, 2021

mjlbach Feb 18, 2021

mjlbach commented Feb 18, 2021 •

edited

mjlbach commented Feb 18, 2021

mjlbach Feb 18, 2021

dm1try Feb 19, 2021

mjlbach Feb 18, 2021

mjlbach Feb 18, 2021

mjlbach commented Feb 18, 2021 •

edited

dm1try commented Feb 19, 2021

mjlbach commented Feb 19, 2021

teto commented Feb 19, 2021

mjlbach commented Feb 19, 2021 •

edited

Tests: fix init.lua tests hanging #13744

Tests: fix init.lua tests hanging #13744

Conversation

mjlbach commented Jan 13, 2021

mjlbach commented Jan 13, 2021

dm1try commented Jan 13, 2021

mjlbach commented Jan 13, 2021

dm1try commented Jan 13, 2021 • edited

mjlbach commented Jan 13, 2021

dm1try commented Jan 13, 2021

mjlbach commented Jan 13, 2021

mjlbach commented Jan 13, 2021

dm1try commented Jan 13, 2021

mjlbach commented Jan 13, 2021

dm1try Jan 13, 2021 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mjlbach commented Feb 18, 2021 • edited

mjlbach commented Feb 18, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mjlbach commented Feb 18, 2021 • edited

dm1try commented Feb 19, 2021

mjlbach commented Feb 19, 2021

teto commented Feb 19, 2021

mjlbach commented Feb 19, 2021 • edited

dm1try commented Jan 13, 2021 •

edited

dm1try Jan 13, 2021 •

edited

mjlbach commented Feb 18, 2021 •

edited

mjlbach commented Feb 18, 2021 •

edited

mjlbach commented Feb 19, 2021 •

edited