[css-text] Control characters

[upsuper]

There was a long thread in www-style discussing about how control characters should be rendered, and that resulted in the following text in css-text-3:

Control characters (Unicode category Cc) other than tab (U+0009), line feed (U+000A), form feed (U+000C), and carriage return (U+000D) must be rendered as a visible glyph and otherwise treated as any other character of the Other Symbols (So) general category and Common script.

It seems to me that no browser is currently shipping this behavior, and IIRC we wanted to coordinate shipping this behavior together so that websites know it is an intentional behavior change for the platform.

I'd like to ask are we still want to do so? Firefox has implemented a switch for the new behavior since long ago, and we have it enabled by default for Nightly and early Beta. We are able to ship this change at any time, but we don't want to ship it without intent from other vendors. (And although it is not a big deal, the switch itself still adds some burden to maintaining, especially given that this switch wasn't supposed to stay for long.)

@gregwhitworth, @tabatkins, @litherum have Edge / Blink / WebKit implemented this switch? Can we start coordinating shipping the new behavior again?

[kojiishi]

Blink has shipped this change in M56, the tracking bug at crbug.com/530342.

[upsuper]

@kojiishi It seems that on Blink, the characters are only visible on Windows by default.

Referring @jfkthame's comment from Gecko bug:

I think what you're seeing may be that they are simply passing the characters through to the rendering system and drawing whatever the current font offers, which on Windows turns out to be ".notdef" glyph boxes, but could be zero-width invisible glyphs or blank spaces or whatever happens to be in the font. That doesn't seem a very satisfactory implementation to me.

[kojiishi]

Thanks, it's our misunderstanding if that's the original intention.

We have crbug.com/689599 to render something when font doesn't have glyphs, but we thought it's a different topic.

@gregwhitworth can you help us to understand which was the intention?

[upsuper]

The spec says:

Control characters (Unicode category Cc) other than ... must be rendered as a visible glyph

which seems to indicate that UA should probably not pass through to font, as font can make the character invisible (and some of them actually do so).

[kojiishi]

Ah, that's one interpretation I can agree now with your explanation, but I can still read it differently, since a "glyph" for me is the one from a font, and I didn't imagine a synthesized graphics is a "glyph". Now I understand the text is a bit ambiguous when the font doesn't have the glyph, or have a glyph that doesn't have a ink. The latter is even more complex, I guess it involves heuristic? I'm curious how Gecko implemented it.

This was a request from @gregwhitworth to make browsers interoperable IIUC (correct?), I'm interested to hear which was his intention.

[jfkthame]

For control characters (category Cc) gecko never attempts to draw a glyph from the font; it draws these (if the pref to make them visible is enabled) with the same "hexbox" code it uses as a last-resort rendering for printable characters that are not supported by any available font: a simple box with the character code in tiny hexadecimal digits inside.

[gregwhitworth]

@kojiishi No this was not a change that I wanted, more that I wanted the breaking change to ship in more than 1 UA so they don't get "bugs". We do similar to what to @jfkthame has stated, but have yet to turn it on by default. But yes, if the font doesn't have it you should render something as showing nothing kind of defeats the purpose and results in you being in the same state as before where authors can inadvertently have control characters that mess up other peripherals that depend on those CC without meaning to.

[kojiishi]

Thank you for the clarification, @jfkthame and @gregwhitworth, I understand how it is supposed to work now. This is a feature that ignores glyphs in the font even if it has for Cc code points, and renders synthesized graphics instead. /cc @eaenet

Then we haven't implemented this yet.

[kojiishi]

Probably related, we're seeing the L-SEP glyph (in the font) rendered, since our change only in Chrome (stackoverflow). This is Zl category and isn't default-ignorable as per the Unicode data.

Can I understand that, when other browsers enabled this, Chrome will match to other browsers?

[kojiishi]

...never mind, sorry for the noise, I remember we only care Cc category.

[css-meeting-bot]

The Working Group just discussed Shipping visible Cc, and agreed to the following resolutions:

• RESOLVED: clarify spec to say these characters should be always visible

The full IRC log of that discussion

<fantasai> Topic: Shipping visible Cc
<fantasai> https://drafts.csswg.org/css-text-3/issues-lc-2013#issue-161
<dael> github: https://github.com//issues/1990
<dael> astearns: This time we'll syncronize.
<dael> myles: didn't have a date and it failed. Maybe we just shouldn't do it- show them visible.
<dael> fantasai: We need it all impl behind a flag and then all flip on.
<dael> myles: That's fair, but has always been true.
<dael> fantasai: Didn't some people release?
<dael> florian: They impl something...not what we speced?
<myles> s/Maybe we just shouldn't do it- show them visible./We shouldn't show them visibly/
<dael> fantasai: Comment from me was we wanted to coordinate shipping this together so that authors know it's an intentional thing for the platform.
<fantasai> s/from me/from xidorn/
<dael> eae: For us it's font dependant. If the font has a glyph it shows.
<dael> myles: I think that's what we do too.
<dael> astearns: So people sync shipping things...
<astearns> s/things/thiiinnngs
<dael> fantasai: Previous spec said control characters are invisible. People wanted it visible, I changed the spec, we were going to coordinate and it shipped.
<dael> eae: WE had a flag and there was a rough time peroid.
<dael> fremy: We were supposed to do it and we did not.
<dael> fantasai: I think goal was ship on a particular day. Anyone who shipped on proposed schedule had no one with them.
<dael> astearns: However this happened. What's before us right now is how to deal with spec. Do we leave it in that we want control characters visible? Change spec to match what's interop?
<dael> florian: We no longer have full interop. Some browsers are visible or not depending on font.
<dael> astearns: Any browsers always visible?
<dael> florian: I think that's behind FF flag.
<dael> astearns: Edge has not shipped it. Safari?
<dael> myles: [missed]
<dael> florian: Blink basic font for windows has control characters so it's there but on iOS it doesn't so they're not.
<dael> Rossen: We shipped when we said it would.
<dael> fremy: Did we?
<dael> fantasai: We have 2 interop impl on windows.
<fantasai> I didn't say anything
<dael> dbaron: Based on what I'm reading FF nightly has had it on for a while.
<dael> astearns: This being in the spec is not gated on the simultaious shipping.
<dbaron> (though I'm less sure than usual that I'm reading correctly since somebody just rewrote a bunch of pref stuff...)
<dael> florian: Clarify spec that we mean visible visible.
<dael> ??: What character do you use
<dael> fantasai: Your default.
<fantasai> s/Your default/missing glyph replacement/
<dael> florian: You put a visible thing so people fix it.
<dael> astearns: So this issue is basically close and clarify that it should be visible. But the coordination convo is not relelvent
<dael> astearns: Objections to clarify spec to say these characters should be always visible ?
<xidorn> (dbaron: I just had a look at the code and I don't think we enable it right now on nightly)
<dael> eae: First browser will look broken. It'll be worse if we force to always. I'm not opposed but a little unpleasent.
<dael> astearns: Still better if coordinate
<dael> RESOLVED: clarify spec to say these characters should be always visible
<dbaron> xidorn, layout.css.control-characters.visible seems to be enabled