ANSI/VT \e[7m and \e[27m (reverse/negative) do not work correctly after 24-bit color has been used

[PhMajerus]

Concerns Redstone 3 RTW / Windows 10 ver. 1709, tested on build 16299.19. Affects all WSL distros and more broadly, all Win32 CUI apps using VT.

VT escape control sequences "\x1B[7m" and "\x1B[27m" are used to turn reverse video ("negative", but not technically negative) on and off.
They work fine until the console processed a 24-bit color "\x1B[38;2;r;g;bm" or "\x1B[48;2;r;g;bm". At that point, negative off (27) has no effect or even acts erratically, even if the color has been reset ("\x1B[39m", set back to a base 16 color (\x1B[3(0-9)m"), or if SGR has been reset ("\x1B[m") in the meantime.

echo -e "\e[7m Reversed \e[27m Normal \e[38;2;250;20;40m 24-bit \e[36m 16-colors \e[7m Reversed \e[27m Normal \e[39m Default \e[7m Reversed \e[27m Normal \e[m Reset \e[7m Reversed \e[27m Normal"

This is a bug in conhost that affects all Windows CUI apps that uses VT, as the following repro code shows:

#include <windows.h>

#ifndef ENABLE_VIRTUAL_TERMINAL_PROCESSING
#define ENABLE_VIRTUAL_TERMINAL_PROCESSING 0x0004
#endif

int main()
{
	HANDLE hOutput = GetStdHandle(STD_OUTPUT_HANDLE);
	
	DWORD dwMode = 0;
	GetConsoleMode(hOutput, &dwMode);
	dwMode |= ENABLE_VIRTUAL_TERMINAL_PROCESSING;
	SetConsoleMode(hOutput, dwMode);
	
	LPTSTR pszTestString = TEXT("\x1B[7m Reversed \x1B[27m Normal \x1B[38;2;128;5;255m Set 24-bit foreColor \x1B[7m Reversed \x1B[27m Normal \r\n");
	WriteConsole(hOutput, pszTestString, _tcslen(pszTestString), NULL, NULL);
	
	getchar();
	return 0;
}

Calling SetConsoleScreenBufferInfoEx or showing and OK'ing the console's properties dialog seems to reset the issue until next time 24-bit color is used.

[zadjii-msft]

Fix is in code review now.

@PhMajerus Thanks for the great writeup :) It's always so much easier to fix things when the problem is so well isolated with excellent repro steps.

[PhMajerus]

@zadjii-msft Thanks for looking into it and fixing it.
Do you know when the fix might appear in insider build? I'm still seeing the issue in 17093 and would hate for something to go wrong and get unnoticed until it's too late for RS4.

[bitcrazed]

This issue was moved to microsoft/terminal#78