Support ITU's T.416 - ODA colour sequences

[tusharsnx]

Description

This is generally covered in #4321. But, since that will closed by #15648, we need a separate issue to track ITU T.416 - ODA Color sequence support.

Brief explanation¹:

The ITU's T.416 Information technology - Open Document Architecture (ODA) and interchange format: Character content architectures uses ":" as separator characters instead:

Color 8-bit:

• Foreground color: ESC[38:5:⟨n⟩m
• Background color: ESC[48:5:⟨n⟩m

where n is the index from the xterm's color table

Color 24-bit:

This is a bit too extensive with CS tolerance. I've removed it for simplicity:

• Foreground color: ESC[38:2::⟨r⟩:⟨g⟩:⟨b⟩m
• Background color: ESC[48:2::⟨r⟩:⟨g⟩:⟨b⟩m

The two colons in the middle signifies ignored color-space-id, which is not supported in most cases. The ITU-RGB variation is supported by xterm, with the colorspace ID and tolerance parameters ignored. The parameters after the '2' (r, g, and b) are optional and can be left empty.

Per the ITU's T.416 - ODA documentation

The parameter values 38 and 48 are followed by a parameter substring used to select either the character foreground “colour value” or the character background “colour value”.

A parameter substring for values 38 or 48 may be divided by one or more separators (03/10) into parameter elements,
denoted as Pe. The format of such a parameter sub-string is indicated as:
Pe : P ...

Each parameter element consists of zero, one or more bit combinations from 03/00 to 03/09, representing the digits
0 to 9. An empty parameter element represents a default value for this parameter element. Empty parameter elements at
the end of the parameter substring need not be included.

The first parameter element indicates a choice between:
0 implementation defined (only applicable for the character foreground colour)
1 transparent;
2 direct colour in RGB space;
3 direct colour in CMY space;
4 direct colour in CMYK space;
5 indexed colour.

• If the first parameter has the value 0 or 1, there are no additional parameter elements.
• If the first parameter element has the value 5, then there is a second parameter element specifying the index into the
  colour table given by the attribute “content colour table” applying to the object with which the content is associated.
• If the first parameter element has the value 2, 3, or 4, the second parameter element specifies a colour space identifier
  referring to a colour space definition in the document profile.
• If the first parameter element has the value 2, the parameter elements 3, 4, and 5, are three integers for red, green, and
  blue colour components. Parameter 6 has no meaning.
• If the first parameter has the value 3, the parameter elements 3, 4, and 5 and three integers for cyan, magenta, and yellow
  colour components. Parameter 6 has no meaning.
• If the first parameter has the value 4, the parameter elements 3, 4, 5, and 6, are four integers for cyan, magenta, yellow,
  and black colour components.
• If the first parameter element has the value 2, 3, or 4, the parameter element 7 may be used to specify a tolerance value
  (an integer) and the parameter element 8 may be used to specify a colour space associated with the tolerance (0 for
  CIELUV, 1 for CIELAB).

Tasks

Beta Give feedback

1. Support 8-bit color
2. Support 24-bit color
3. Add support for sending the right sequences in vtrenderer: We use semi-colons to separate the SGR color sequence while forwarding it to ConPTY clients. This is to remain compatible with older clients.

Footnotes

1. It is mostly cited from Wikipedia as it is short and clear. ↩

[zadjii-msft]

I'm not sure that we need to do the vtrenderer bits honestly, but everything else looks straightforward. Unless there was a meaningful difference in implementation between 256/RGB colors being output with ; vs :, then ultimately it shouldn't matter, yea?

[tusharsnx]

Unless there was a meaningful difference in implementation between 256/RGB colors being output with ; vs :, then ultimately it shouldn't matter, yea?

The final (on-screen) output remains same. But, I guess we should ensure that if the application had sent us (ConPTY) \e[38:2::r:g:b, then we should forward the same. Currently, we don't have a way to indicate that the colors applied to the attr is done using SubParams. vtrenderer will produce \e[38;2;r;g;b in the output buffer.

[j4james]

I guess we should ensure that if the application had sent us (ConPTY) \e[38:2::r:g:b, then we should forward the same.

Not all conpty clients will support the colon format, though, so it's best we always forward ODA colors using semicolons. If/when we do the extended underline attributes, though, those will have to use colons.

[DHowett]

Yeah, that works for me. We should accept as much as we can and begrudgingly emit the most compatible thing we can (which for better or for worse was a big accident that is more broadly-supported than the right thing ;))

[tusharsnx]

Not all conpty clients will support the colon format, though, so it's best we always forward ODA colors using semicolons. If/when we do the extended underline attributes, though, those will have to use colons.

That seems easy. I already have something locally that works.

I might have concern regarding clients that feature test colon syntax by first sending the sequence and then querying ConPTY if the colour was applied. Wouldn't it confuse the client app if we send semi-colons instead of colons?

Querying the terminal for colour support: https://github.com//termstandard/colors#querying-the-terminal

On Windows Terminal (if were to return semi-colon):

$ (echo -e '\e[48:2:1:2:3m\eP$qm\e\\' ; xxd)

^[P1$r0;48;2;2;3;0m^[\
00000000: 1b50 3124 7230 3b34 383b 323b 323b 333b  .P1$r0;48;2;2;3;

It might be less of a concern for us, if we've other ways to show that we also support colon format.

[j4james]

I might have concern regarding clients that feature test colon syntax by first sending the sequence and then querying ConPTY if the colour was applied. Wouldn't it confuse the client app if we send semi-colons instead of colons?

That query is handled by conhost, and we definitely should be replying to that with colons once you've added the ODA support. So the app that's doing the feature testing will assume we support colons, and send us ODA colors using colons. But we should always pass that on to the conpty client with semicolons, since we have no idea what they support.

It's theoretically possible they don't support ODA colors at all, in which case the feature testing isn't going to work correctly. But that's just an unfortunate limitation of conpty, and hopefully one day it could be resolved with the "passthrough" mode.

[DHowett]

This might be wishful thinking, but does 38:1 (where 1 is "transparent" (???)) suggest that we might have a way to specify (on the Terminal side) a foreground color that you can see through to the background? Similar to the request in #7014?

[j4james]

does 38:1 (where 1 is "transparent" (???)) suggest that we might have a way to specify (on the Terminal side) a foreground color that you can see through to the background?

I'm not sure, but I don't think so. My impression was that this was more or less the equivalent of SGR 8 (the "concealed" attribute), because that's the one SGR attribute that isn't listed in the ODA docs.

And bear in mind that ODA is a document format - most of the functionality is geared towards printed output. Invisible text is easy enough to print (technically not print), but you're unlikely to be printing text that makes a hole in the paper so you can see through the background!

It's probably worth checking to see what other terminals are doing with that option though. I think I might have looked into this in the past, but I can't find anything in my notes - possibly because nobody actually supported it. But I'll double check when I get a chance.