Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

win,tty: add support for ANSI codes in win10 v1511 #889

Closed
wants to merge 1 commit into from

Conversation

imran-iq
Copy link
Contributor

@imran-iq imran-iq commented May 27, 2016

Use windows 10's v1511 built in support for ANSI codes instead of emulating

@imran-iq
Copy link
Contributor Author

As discussed in #721

@imran-iq
Copy link
Contributor Author

Not sure who to ping on windows stuff
/cc @libuv/collaborators thoughts?

@saghul
Copy link
Member

saghul commented Jun 15, 2016

I'll take a look as soon as I can.

@joshgav
Copy link

joshgav commented Jun 16, 2016

/cc @miniksa

@miniksa
Copy link

miniksa commented Jun 16, 2016

Hey there! I was the one to actually implement this in Windows 10 v1511. :) There's some documentation available here that might help: https://msdn.microsoft.com/en-us/library/windows/desktop/mt638032(v=vs.85).aspx

However, it looks like you've got a handle on it already with this pull request. I would just recommend against checking the OS version using kernel32 that way because it is possible for someone to be on v1511+ and be using the Legacy console which doesn't support these sequences.

The preferred way that we've been suggesting to detect whether the support is available is to do probe-and-set (you can see an example in the contribution to Docker client here: https://github.com/docker/docker/blob/master/pkg/term/term_windows.go). Basically, try to get the current console mode, perform binary or with the additional virtual terminal flag onto it, and set. If it rejects it with STATUS_INVALID_PARAMETER, then you're on an old version of the OS or the user is on a compatibility mode console that doesn't support it. If it goes through with return code 0 (STATUS_SUCCESSFUL), you're golden. :)

The support has been made much more robust for the Anniversary update coming this summer, so if you're finding missing functionality, it might already be on the way!

If you have any particular questions or concerns on the implementation, let me know and I'll see if I can help answer them! You can also prod my ANSI sequence partner in crime, Mike (@zadjii-msft).

Thanks,
--Michael @ Microsoft

@saghul
Copy link
Member

saghul commented Jun 16, 2016

Thanks for chiming in! Much appreciated.

@imran-iq imran-iq force-pushed the win10ansi branch 3 times, most recently from 34cddca to 8a570fa Compare June 23, 2016 21:55
@imran-iq
Copy link
Contributor Author

So updated the code as per comments (for testing purposes put a printf in my code to let me know we are using my code, its not actually in the commit I hope!)

Heres a console using legacy mode running my modified test(our emulated implementation):

Legacy Mode

And non legacy mode(new code that uses built in support):
Non-LegacyMode

@miniksa
Copy link

miniksa commented Jun 27, 2016

Looks good to me!

@imran-iq
Copy link
Contributor Author

imran-iq commented Jul 4, 2016

If I can get one more LGTM (preferably from a libuv member) ill go ahead and land this and close #721

@saghul
Copy link
Member

saghul commented Jul 5, 2016

I'll review it this week.

@avih
Copy link

avih commented Jul 6, 2016

FWIW, I tested this technique on windows 10, and GetConsoleMode(h) works, as well as SetConsoleMode(h, ENABLE_VIRTUAL_TERMINAL_PROCESSING) (the latter only on Windows 10, and returning ERROR_INVALID_PARAMETER on windows 8.1), however, it seems buggy to me.

While colors do work on win10, I couldn't get bold to work, eventhough in their example at https://msdn.microsoft.com/en-us/library/windows/desktop/mt638032(v=vs.85).aspx it does seem to work.

Maybe it was just sensitive to the way the escape sequence was constructed (e.g. maybe they expect bold to be before/after the color, or maybe just one SGR code per escape.. didn't check deeper - but according to the docs the sequence looks good - just simple color and bold), but the same sequence works on linux on every terminal I tested (VTE based, KDE terminal and st.suckless.org).

FYI.

@miniksa
Copy link

miniksa commented Jul 6, 2016

@avih Thanks for the information, but we're going to need to know exactly which sequence was emitted when we didn't react appropriately if we're going to fix it. Can you provide that detail?

cc: @zadjii-msft

@avih
Copy link

avih commented Jul 6, 2016

Not right now. I was experimenting with the win10 console support on another project, and I didn't yet dig deep (and didn't try it at all with libuv).

For now I'd just suggest to experiment with few simple SGR sequences which include bold and a one or two colors (e.g. 1;31 or 30;41;1 etc) and just make sure it works.

If I'll have more concrete findings, I'll report back.

@miniksa
Copy link

miniksa commented Jul 6, 2016

@avih I'll wait to hear your concrete findings.

@@ -121,6 +121,18 @@ static char uv_tty_default_fg_bright = 0;
static char uv_tty_default_bg_bright = 0;
static char uv_tty_default_inverse = 0;

#ifndef ENABLE_VIRTUAL_TERMINAL_PROCESSING
#define ENABLE_VIRTUAL_TERMINAL_PROCESSING 0x0004
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you make this # define ... (we do this in other places too, it helps readability)

@saghul
Copy link
Member

saghul commented Jul 7, 2016

@avih @miniksa I do see color and bold working together nicely: http://imgur.com/GilDcMB

@iwuzhere great work! left some comments.

@miniksa
Copy link

miniksa commented Jul 8, 2016

@saghul That looks like you're using ConEmu or some derivative, not the default Windows Console Host. It has a different rendering pipeline than ours which is known to be able to do ANSI color even before we added it to Windows 10, so that's not really verification. :(

@imran-iq imran-iq force-pushed the win10ansi branch 2 times, most recently from 727ca2a to 567e2d2 Compare August 24, 2016 01:56
@imran-iq
Copy link
Contributor Author

Updated once again.
Left a comment about using uv_once. We can't pass the handle to uv__determine_vterm_state by calling it through uv_once without using a global. Using a global to store a tty handle that is just used once during initializing doesn't sit right with me.

}

if (!MultiByteToWideChar(CP_UTF8, 0, buf.base, buf.len,
utf16_buf, utf16_buf_used)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you line these up vertically?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to clarify, do you mean like this? (for the calls to MultiByteToWideChar)

MultiByteToWideChar(CP_UTF8,
                    0,
                    buf.base,
                    buf.len,
                    utf16_buf,
                    utf16_buf_used)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep!

@saghul
Copy link
Member

saghul commented Aug 24, 2016

LGTM with a tiny style nit.

@imran-iq
Copy link
Contributor Author

Updated!

@saghul
Copy link
Member

saghul commented Aug 24, 2016

Rubber-stamp LGTM! 🎉

@saghul
Copy link
Member

saghul commented Aug 25, 2016

@iwuzhere can you land this one? I want to make a WIP libuv upgrade PR to Node to see if smoke comes out ;-)

imran-iq added a commit that referenced this pull request Aug 26, 2016
PR-URL: #889
Reviewed-By: Saúl Ibarra Corretgé <saghul@gmail.com>
@imran-iq
Copy link
Contributor Author

Landed 58ccfd4


FLUSH_TEXT();
continue;
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The indentation is off here.

bnoordhuis added a commit to bnoordhuis/libuv that referenced this pull request Nov 13, 2016
Causes regressions on Windows 10 in applications that use ANSI codes.

Also revert one-liner commit 0895ccf ("win: fix typo in type name").

This reverts commit 0895ccf.
This reverts commit 58ccfd4.

Refs: nodejs/node#9542
Refs: libuv#889
Refs: libuv#1135
bnoordhuis pushed a commit that referenced this pull request Nov 15, 2016
In uv_tty_write_bufs, if the console supports Virtual Terminal sequences,
we try to convert the passed in utf8 buffer to utf16. However, we need
to check if the buffer is of non-zero length- otherwise,
MultiByteToWideChar returns an error.

Fixes: #1135
Fixes: nodejs/node#9542
PR-URL: #1139
Refs: #889
Reviewed-By: Bartosz Sosnowski <bartosz@janeasystems.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Imran Iqbal <imran@imraniqbal.org>
bnoordhuis added a commit to bnoordhuis/libuv that referenced this pull request Nov 15, 2016
Causes regressions on Windows 10 in applications that use ANSI codes.

Also reverts commit 0895ccf ("win: fix typo in type name") and
commit d0c2641 ("win, tty: handle empty buffer in uv_tty_write_bufs".)

This reverts commit d0c2641.
This reverts commit 0895ccf.
This reverts commit 58ccfd4.

Refs: nodejs/node#9542
Refs: libuv#889
Refs: libuv#1135
bnoordhuis added a commit to bnoordhuis/libuv that referenced this pull request Nov 15, 2016
Causes regressions on Windows 10 in applications that use ANSI codes.

Also reverts commit 0895ccf ("win: fix typo in type name") and
commit d0c2641 ("win, tty: handle empty buffer in uv_tty_write_bufs".)

This reverts commit d0c2641.
This reverts commit 0895ccf.
This reverts commit 58ccfd4.

PR-URL: libuv#1138
Refs: libuv#1135
Refs: libuv#889
Refs: nodejs/node#9542
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Imran Iqbal <imran@imraniqbal.org>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
santigimeno added a commit to santigimeno/libuv that referenced this pull request Nov 29, 2016
To bring back support for ANSI codes in win10 v1511.

This reverts commit 8cbabaa.

PR-URL: libuv#1143
Refs: libuv#1138
Refs: libuv#889
Reviewed-By: Saúl Ibarra Corretgé <saghul@gmail.com>
Reviewed-By: Imran Iqbal <imran@imraniqbal.org>
santigimeno added a commit to santigimeno/libuv that referenced this pull request Nov 29, 2016
Make sure there's enough room in the output buffer by dynamically
allocating memory in case the size of the buffer needs to be greater
than 8192 characters.

PR-URL: libuv#1143
Refs: libuv#1138
Refs: libuv#889
Reviewed-By: Saúl Ibarra Corretgé <saghul@gmail.com>
Reviewed-By: Imran Iqbal <imran@imraniqbal.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants