Use RFC 7230 for token definition

[dyladan]

Resolves the issue raised by w3ctag/design-reviews#650 (comment)

Changes the definition of token to refer to RFC 7230 instead of RFC 2616. RFC 7230 is already used for the definition of OWS and for header concatenation rules. IMO the new definition is more clear and easily readable.

Note: The definitions are written differently. The old 2616 definition defines token in terms of what characters ARE NOT allowed (separators). The new 7230 definition defines token in terms of which characters ARE allowed. The definitions appear to be identical as far as I can tell. There are 72 allowed characters in the US ASCII range (0-127).

Old token definition from 2616:

       CHAR           = <any US-ASCII character (octets 0 - 127)>
       CTL            = <any US-ASCII control character
                        (octets 0 - 31) and DEL (127)>

       token          = 1*<any CHAR except CTLs or separators>
       separators     = "(" | ")" | "<" | ">" | "@"
                      | "," | ";" | ":" | "\" | <">
                      | "/" | "[" | "]" | "?" | "="
                      | "{" | "}" | SP | HT

New token definition from 7230:

     VCHAR defined as any visible [USASCII] character

     token          = 1*tchar

     tchar          = "!" / "#" / "$" / "%" / "&" / "'" / "*"
                    / "+" / "-" / "." / "^" / "_" / "`" / "|" / "~"
                    / DIGIT / ALPHA
                    ; any VCHAR, except delimiters

---

Preview | Diff

[dyladan]

The repository manager does not recognize me as a contributor, and when I click "revalidate" it gives me the following error:

Could there be some sort of outage?