tool_doswin: Restore original console settings on CTRL signal

[jay]

• Move Windows terminal init code from tool_main to tool_doswin.

• Restore the original console settings on CTRL+C and CTRL+BREAK.

Background: On Windows the curl tool changes the console settings to
enable virtual terminal processing (eg color output) if supported
(ie Win 10). The original settings are restored on exit but prior to
this change were not restored in the case of the CTRL signals.

Closes #xxxx

---

@bitcrazed can you take a look at this please. It was my understanding from #3008 that if VT was set on the console then it needed to be restored on exit. Currently that doesn't happen on CTRL+C / CTRL+BREAK signals, so I made some changes in this PR to address that.

However.. after that I did some empirical testing in Windows 10 20H2 that makes me wonder if restoring the console's original vt setting is necessary in any case, since the system appears to do that already. Take for example the C program below that was run twice from the same command prompt console and turns on VT without shutting it off again:

foo\test
VT processing is off
VT processing is on

foo\test
VT processing is off
VT processing is on

#ifndef ENABLE_VIRTUAL_TERMINAL_PROCESSING
#define ENABLE_VIRTUAL_TERMINAL_PROCESSING 0x0004
#endif

HANDLE hStdOut = INVALID_HANDLE_VALUE;
DWORD dwOutputMode = 0;

#define ABORT(x) do { \
  printf("#x failed.\n"); \
  return 1; \
} while(0)

#define SHOWVT() do { \
  printf("VT processing is %s\n", \
    (dwOutputMode & ENABLE_VIRTUAL_TERMINAL_PROCESSING) ? "on" : "off"); \
} while(0)

int main(int argc, char *argv[])
{

  hStdOut = GetStdHandle(STD_OUTPUT_HANDLE);
  if(hStdOut == INVALID_HANDLE_VALUE)
    ABORT("GetStdHandle");

  if(!GetConsoleMode(hStdOut,&dwOutputMode))
    ABORT("GetConsoleMode");
  
  SHOWVT();

  if(!SetConsoleMode(hStdOut, (dwOutputMode |
                               ENABLE_VIRTUAL_TERMINAL_PROCESSING)))
    ABORT("SetConsoleMode");

  if(!GetConsoleMode(hStdOut,&dwOutputMode))
    ABORT("GetConsoleMode");
  
  SHOWVT();

  return 0;
}

Ref: https://docs.microsoft.com/en-us/windows/console/ctrl-c-and-ctrl-break-signals

[bitcrazed]

Perhaps I am missing something, but I am interested in why InterlockedExchange() is being used here (and below) when setting the value for a struct which (as far as I can see) will only be called from the main thread, once, during startup.

[jay]

restore_terminal can be called from the signal handler which the system calls from a separate thread

[bitcrazed]

Ah, got it. Thanks. Didn't realize that.

[bitcrazed]

See question above re. use of InterlockedExchange() here too,

[jay]

@bitcrazed could you please address whether it's necessary to restore the console's vt setting on process termination, as mentioned in the first post

[bitcrazed]

Apologies for my delayed reply. It's been a little nuts at work getting all the balls rolling after Xmas. I'll double-check and get back to you all.

[bitcrazed]

Okay, I've confirmed with the team:

Console modes are currently global state. While the team is aiming to eventually make modes per-process, that work has yet to land.

So command-line apps should save/restore Console modes on entry/exit.

[jay]

Console modes are currently global state.

Thanks. If the VT state global why is it resetting on process termination (as seen in the first post)?

[bitcrazed]

For back-compat reasons, Console and Terminal handle modes differently. And depending on what shell you're using, you may see VT mode being enabled or disabled by default.

For example, if you run your ConsoleMode sample in PowerShell, Windows PowerShell, or Cmd in the (legacy) Console, VT is off by default and is rest on process exit. But when using Terminal, VT mode is on by default. Other shell/terminal combo's may choose different options.

So, the best approach is to be a good command-line citizen and save/restore mode state on entry/exit so that your app knows what state it's in and considerately cleans up after itself.

[jay]

So, the best approach is to be a good command-line citizen and save/restore mode state on entry/exit so that your app knows what state it's in and considerately cleans up after itself.

Got it. Thanks again.

[bitcrazed]

No problem - thanks for making these improvements - let me know if you need any info/assistance 😁