HTML entities

[mathiasbynens]

E.g. © → © or &#00000000000000000000000000000169; → ©

You could use he’s he.decode() (combined with he.escape(), perhaps) for this. It implements the character reference decoding algorithm described in the HTML spec and support the special “attribute value mode” as well.

[kangax]

Hm, is this safe to do? For all document encodings/lack of encoding?

[mathiasbynens]

For pages with <meta charset=utf-8> or similar that are saved using UTF-8 encoding this would be safe.

[kangax]

Then we can have it as a non-default option to see how it behaves in other cases

[alexlamsl]

So I'll need some pointers here - what are the valid (enough) non-ASCII characters that does not need to be escaped in a HTML document?

I am thinking of tackling this alongside all those weird and wonderful JS/CSS attributes which has escaped characters beyond quotation marks (which we already handle today).

As for encoding, I think given the constraints of JavaScript we are probably better off just treating all input as Unicode. I haven't had to use the "encoding" section of my web browser for years now.

[mathiasbynens]

Must remain escaped: <, & in some cases (see below), " and ' in attribute values. And the backtick character too, if you care about old IE. > doesn’t need to be escaped unless it’s part of an unquoted attribute value.

See https://mathiasbynens.be/notes/ambiguous-ampersands for some hardcore optimizations you could enable.