We don't have a reliable way to tell when commands have finished executing

[Soares]

This single problem leads to a host of other issues. Mostly, it forces us to use arbitrary delays between commands (instead of having some way of knowing when commands are finished). This slows down the tests considerably.

Furthermore, it means that we can't reliably know whether messages were dropped or whether we just haven't waited long enough yet. Which sucks.

[AndrewRadev]

I'm the author of vimrunner, a project that also tries to drive Vim using the --remote interface.

My solution is to just always use --remote-expr for anything that requires confirmation, since it waits to return output. You can run normal-mode mappings by triggering --remote-expr with either feedkeys() or normal!, and you can run commands like :write by defining a vim-level function that executes the command.

The relevant code for commands is here. The VimrunnerEvaluateCommandOutput function is defined in Vimscript and also tries to capture any output from the command. I've had various issues with that and silent!, but that may not be interesting to you.

[dbarnett]

Awesome, thanks! We were evaluating that approach, but hit some snags with special key codes, continued lines, and error handling and haven't finished getting something working yet. But glad to know it's feasible.

[AndrewRadev]

Yeah, escaping can be annoying when you pass things around a bunch of strings that are then given to arguments to functions that are then sent through the remote interface. I would suggest you browse through the relevant Vimrunner code, since it seems to be working well in practice (even so, I'd appreciate reports of any bugs/inconsistencies you notice in there :)).

[AndrewRadev]

Ah, also, just noticed you mentioned error handling. I may sit down and write a lengthy explanation of what I remember of my own problems/solutions in that area, but I'll see if I find the time.

[dbarnett]

One special case I noticed is it seems to have trouble with executing shell commands in the foreground (with :!) that make it take slightly longer than our default neovim delay (example: https://travis-ci.org/github/bazelbuild/vim-bazel/jobs/756778325).