Make -s - read from stdin #8276

b-r-o-c-k · 2018-04-14T20:24:55Z

This is a continuation of #6299 with a fix I added for Windows.

Check whether `repeated_read_cmd` returned nil and did not actually run nvim or it did, but still returned nil for whatever reason.

This variant uses `fdopen()` which is not standard, but it fixes problem on my system. In next commit `scriptin` will use `FileDescriptor*` from os/fileio in place of `FILE*`.

Problem: as fileio is cached and reads blocks this is going to wait until either EOF or reading enough characters to fill rbuffer. This is not good when reading user input from stdin as script.

Assume something with system() if second test hangs as well. Assume something with reading stdin if not.

* Reading from stdin on Windows is fixed in the same way as it was in neovim#8267. * The file_read function was returning without filling the destination buffer when it was called with a non-blocking file descriptor.

justinmk · 2018-04-17T00:02:21Z

test/functional/core/main_spec.lua

+      eq(#('100500\n'), attrs.size)
+    end)
+    it('does not crash after reading from stdin in non-headless mode', function()
+      if helpers.pending_win32(pending) then return end


does this still need to be skipped on windows?

The test fails even though :cq works.

test\functional\ui\screen.lua:307: Row 2 did not match. Expected: |^ | |*{1:~ }| |{1:~ }| |{1:~ }| |{1:~ }| |{2:[No Name] 0,0-1 All}| | | | | Actual: |^ | |* | | | | | | | | | | | | |

justinmk · 2018-04-17T00:03:40Z

src/nvim/os/fileio.c

-    if (fp->eof) {
+    if (fp->eof
+        // Allow only at most one os_read[v] call.
+        || (called_read && fp->non_blocking)) {


@ZyX-I ok with this change?

janlazo · 2018-04-17T03:13:09Z

test/functional/core/main_spec.lua

+      ]], {
+        [1] = {foreground = 4210943, special = Screen.colors.Grey0},
+        [2] = {special = Screen.colors.Grey0, bold = true, reverse = true}
+      })


Why is this necessary? This won't work on Windows until the TUI highlighting works on cmd.exe or powershell .

FEATURES: 3cc7ebf #7234 built-in VimL expression parser 6a7c904 #4419 implement <Cmd> key to invoke command in any mode b836328 #7679 'startup: treat stdin as text instead of commands' 58b210e :digraphs : highlight with hl-SpecialKey #2690 7a13611 #8276 'startup: Let `-s -` read from stdin' 1e71978 events: VimSuspend, VimResume #8280 1e7d5e8 #6272 'stdpath()' f96d99a #8247 server: introduce --listen e8c39f7 #8226 insert-mode: interpret unmapped META as ESC 98e7112 msg: do not scroll entire screen (#8088) f72630b #8055 let negative 'writedelay' show all redraws 5d2dd2e win: has("wsl") on Windows Subsystem for Linux #7330 a4f6cec cmdline: CmdlineEnter and CmdlineLeave autocommands (#7422) 207b7ca #6844 channels: support buffered output and bytes sockets/stdio API: f85cbea #7917 API: buffer updates 418abfc #6743 API: list information about all channels/jobs. 36b2e3f #8375 API: nvim_get_commands 273d2cd #8329 API: Make nvim_set_option() update `:verbose set …` 8d40b36 #8371 API: more reliable/descriptive VimL errors ebb1acb #8353 API: nvim_call_dict_function 9f994bb #8004 API: nvim_list_uis 3405704 #7520 API/UI: forward option updates to UIs 911b1e4 #7821 API: improve nvim_command_output WINDOWS OS: 9cefd83 #8084, #8516 build/win: support MSVC ee4e1fd win: Fix reading content from stdin (#8267) TUI: ffb8904 #8309 TUI: add support for mouse release events in urxvt 8d5a46e #8081 TUI: implement "standout" attribute 6071637 TUI: support TERM=konsole-256color 67848c0 #7653 TUI: report TUI info with -V3 ('verbose' >= 3) 3d0ee17 TUI/rxvt: enable focus-reporting d109f56 #7640 TUI: 'term' option: reflect effective terminal behavior FIXES: ed6a113 #8273 'job-control: avoid kill-timer race' 4e02f1a #8107 'jobs: separate process-group' 451c48a terminal: flush vterm output buffer on pty output #8486 5d6732f :checkhealth fixes #8335 53f11dc #8218 'Fix errors reported by PVS' d05712f inccommand: pause :terminal redraws (#8307) 51af911 inccommand: do not execute trailing commands #8256 84359a4 terminal: resize to the max dimensions (#8249) d49c1dd #8228 Make vim_fgets() return the same values as in Vim 60e96a4 screen: winhl=Normal:Background should not override syntax (#8093) 0c59ac1 #5908 'shada: Also save numbered marks' ba87a2c cscope: ignore EINTR while reading the prompt (#8079) b1412dc #7971 ':terminal Enter/Leave should not increment jumplist' 3a5721e TUI: libtermkey: force CSI driver for mouse input #7948 6ff13d7 #7720 TUI: faster startup 1c6e956 #7862 TUI: fix resize-related segfaults a58c909 #7676 TUI: always hide cursor when flushing, never flush buffers during unibilium output 303e1df #7624 TUI: disable BCE almost always 249bdb0 #7761 mark: Make sure that jumplist item will not have zero lnum 6f41ce0 #7704 macOS: Set $LANG based on the system locale a043899 #7633 'Retry fgets on EINTR' CHANGES: ad60927 #8304 default to 'nofsync' f3f1970 #8035 defaults: 'fillchars' a6052c7 #7984 defaults: sidescroll=1 b69fa86 #7888 defaults: enable cscopeverbose 7c4bb23 defaults: do :filetype stuff unless explicitly "off" 2aa308c #5658 'Apply :lmap in macros' 8ce6393 terminal: Leave 'relativenumber' alone (#8360) e46534b #4486 refactor: Remove maxmem, maxmemtot options 131aad9 win: defaults: 'shellcmdflag', 'shellxquote' #7343 c57d315 #8031 jobwait(): return -2 on interrupt also with timeout 6452831 clipboard: macOS: fallback to tmux if pbcopy is broken #7940 300d365 #7919 Make 'langnoremap' apply directly after a map ada1956 #7880 'lua/executor: Remove lightuserdata' INTERNAL: de0a954 #7806 internal statistics for list impl dee78a4 #7708 rewrite internal list impl

ZyX-I and others added 23 commits March 19, 2017 14:13

main: Make -s - read from stdin

65c41e6

functests: Check that -s works as expected

0320a58

tests: Fix CI failures

0e9286a

tests: Fix repeated_popen_r usage, rename the function

3cd7bf3

functests: Alter the order of checks

1ea7541

Check whether `repeated_read_cmd` returned nil and did not actually run nvim or it did, but still returned nil for whatever reason.

functests: Make sure that line ending is LF and not CRLF

38687ee

functests: Use Neovim instance and system() in place of lua io.popen

d4639ea

functests: Fix testlint errors

29654cf

functests: Do not check stdout, it is different on Windows

d2268d5

main: Temporary fix assertion error

fdfa1ed

This variant uses `fdopen()` which is not standard, but it fixes problem on my system. In next commit `scriptin` will use `FileDescriptor*` from os/fileio in place of `FILE*`.

getchar: Use fileio instead of fdopen

bd798a3

Problem: as fileio is cached and reads blocks this is going to wait until either EOF or reading enough characters to fill rbuffer. This is not good when reading user input from stdin as script.

fileio,main: Do not restart syscall at EAGAIN when reading for -s

e78e75d

*: Fix linter errors

88e1a17

os/fileio: Fix QB failure

83b39a4

functests: Test -s errors

7df4fc8

unittests: Add tests for new fileio functions

ae4fae9

main: Translate full -s error message, not part of it

ca116df

functests: Do not run termopen test on Windows

99b4f25

functests: Disable system(-s -) test on Windows

62108c3

Assume something with system() if second test hangs as well. Assume something with reading stdin if not.

Merge branch 'master' into s-dash-stdin

c49e22d

getchar: Move REMAP_… values into a enum

78bc52e

Merge branch 'master' into s-dash-stdin

ad999ea

win: Fix reading from stdin

387fbcd

* Reading from stdin on Windows is fixed in the same way as it was in neovim#8267. * The file_read function was returning without filling the destination buffer when it was called with a non-blocking file descriptor.

marvim added the WIP label Apr 14, 2018

justinmk changed the title ~~[WIP] Make -s - read from stdin~~ Make -s - read from stdin Apr 16, 2018

justinmk approved these changes Apr 17, 2018

View reviewed changes

justinmk reviewed Apr 17, 2018

View reviewed changes

janlazo reviewed Apr 17, 2018

View reviewed changes

justinmk merged commit 7a13611 into neovim:master Apr 17, 2018

justinmk removed the WIP label Apr 17, 2018

b-r-o-c-k deleted the s-dash-stdin-zyx-i branch April 17, 2018 20:38

justinmk mentioned this pull request Apr 22, 2018

"dd if=/dev/zero | nvim -s -" hangs #6088

Open

justinmk added the gsoc community: Google Summer of Code project label Apr 24, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make -s - read from stdin #8276

Make -s - read from stdin #8276

b-r-o-c-k commented Apr 14, 2018

justinmk Apr 17, 2018

b-r-o-c-k Apr 17, 2018

justinmk Apr 17, 2018

ZyX-I Apr 17, 2018

janlazo Apr 17, 2018 •

edited

Loading

Make -s - read from stdin #8276

Make -s - read from stdin #8276

Conversation

b-r-o-c-k commented Apr 14, 2018

justinmk Apr 17, 2018

Choose a reason for hiding this comment

b-r-o-c-k Apr 17, 2018

Choose a reason for hiding this comment

justinmk Apr 17, 2018

Choose a reason for hiding this comment

ZyX-I Apr 17, 2018

Choose a reason for hiding this comment

janlazo Apr 17, 2018 • edited Loading

Choose a reason for hiding this comment

janlazo Apr 17, 2018 •

edited

Loading