fixed hang issue with --headless and -r option specified (#12209)

* fixed hang issue with --headless and -r option specified

Calling the do_more_prompt function in headless mode will freeze neovim because it is eventally in the input-accepting state (the same as waiting for --more--).

* fixed "Press ENTER or type command to continue" to be suppressed

If in headless mode, we need to exit at this point. If we continue, we will enter the normal mode and the message "Press ENTER or type command to continue" will be displayed and we will be in the input waiting state.

* fixed functional ex_cmds tests

* Revert "fixed "Press ENTER or type command to continue" to be suppressed"

This reverts commit a02dc40.

* Revert "fixed functional ex_cmds tests"

This reverts commit 3bdb8da.

* fixed conditional again

* added test for fixed hang issue with --headless (#11386)

(cherry picked from commit c6dc397)

src/nvim/message.c

@@ -2570,10 +2570,15 @@ static int do_more_prompt(int typed_char)
   msgchunk_T  *mp;
   int i;
 
+  // If headless mode is enabled and no input is required, this variable
+  // will be true. However If server mode is enabled, the message "--more--"
+  // should be displayed.
+  bool no_need_more = headless_mode && !embedded_mode;
+
   // We get called recursively when a timer callback outputs a message. In
   // that case don't show another prompt. Also when at the hit-Enter prompt
   // and nothing was typed.
-  if (entered || (State == HITRETURN && typed_char == 0)) {
+  if (no_need_more || entered || (State == HITRETURN && typed_char == 0)) {
     return false;
   }
   entered = true;

test/functional/core/startup_spec.lua

@@ -277,6 +277,24 @@ describe('startup', function()
         [4] = {bold = true, foreground = Screen.colors.Blue1},
     }})
   end)
+
+  it('fixed hang issue with --headless (#11386)', function()
+    local expected = ''
+    local period = 100
+    for i = 1, period - 1 do
+      expected = expected .. i .. '\r\n'
+    end
+    expected = expected .. period
+    eq(
+      expected,
+      -- FIXME(codehex): We should really set a timeout for the system function.
+      -- If this test fails, there will be a waiting input state.
+      funcs.system({nvim_prog, '-u', 'NONE', '-c',
+        'for i in range(1, 100) | echo i | endfor | quit',
+        '--headless'
+      })
+    )
+  end)
 end)
 
 describe('sysinit', function()