JFV + ECMA-262

[igrigorik]

Based on the various discussion at IETF, HTTP Workshop, and mailing lists, JFV may not be the right long-term "replacement" header encoding format. However, while sketches are being pushed around, it's not clear if and when such a format will come either… In the meantime, I believe we can shape JFV (with minor modifications) into a format that is immediately useful and addresses many of the current pain points for new header fields. Specifically:

• UA's want a reusable and tested parser. Today we end up hand-rolling parsers for most every field (due to ABNF quirks, past implementation mistakes, etc), which leads to interop problems, security issues, and so on. Also, this is not a Chrome specific problem, all major UA's have the same pain points, and we all need to do better.
• At the same time, all UA's have well-tested JSON parsers. Reusing existing code helps cut down on attack surface area, helps with interop, etc. As such, JFV format is a strong plus.
• JSON parsers are easy to come by on the server side and all the same benefits and motivation is just as applicable.

That said, as others have already pointed out, we do need additional restrictions on JFV beyond what is currently in the spec. Also, it just so happens that browsers already follow the ECMA-262 spec, which addresses most of the raised concerns:

• Specify how to deal with repeated fields:
  • InternalizeJSONProperty(holder,
    name):
    "In the case where there are duplicate name Strings within an object,
    lexically preceding values for the same key shall be overwritten." All
    browsers already do this.
• Specify number encoding format
  • 4.3.20 Number
    value:
    Number value is primitive value corresponding to a double-precision 64-bit
    binary format IEEE 754-2008 value.
  • 11.8.3 Numeric
    Literals
• Specify date encoding format
  • 20.3.1.16 Date Time String
    Format,
    Date.parse()
    • We can require ISO 8601 Extended format.
• Specify character encoding
  • UTF-8 [RFC3629]

Are there other issues that were brought up that are not addressed above?

My proposal is that we update JFV to spec above ECMA-262 restrictions and requirements. Doing so would enable UA's to adopt JFV with minimal work.

p.s. in progress specs and browser features that plan on leveraging JFV: Clear-Site-Data, Feature Policy, Reporting, Network Error Logging, … and I'm sure we'll see more soon.

/cc @reschke @mnot @mcmanus

[mnot]

Hmm. The number issues are around parsing, not generation, AIUI. So, your solution is good for response headers, but not request headers -- because the browsers haven't yet taken over the server side implementations.

[clelland]

The pointer to 11.8.3 was intended to tackle parsing; 11.8.3.1 defines the parsing algorithm for numeric literals (precisely, down to potentially rounding in the 20th significant digit). The examples where actually significant digits get ignored would indicate a bug in parsing, according to the ES spec.

[mnot]

Right. The pushback against doing something like this that I've heard is that no matter what we say in the spec, people are going to just point the JSON parser they have at hand at the header field value, and trust what it gives them.

As such, using JSON is a bit of an attractive nuisance.

It may be that we can specify a profile of JSON that is safe to use, and create a set of easy-to-use libraries and get them into the relevant server-side platforms, but that's a substantial amount of effort (any volunteers?), and the natural lag of many distros/platforms means that they might not be available for some time (leading people to use plain JSON parsers).

The most serious issue is undoubtedly the number format. Perhaps cautioning against minting headers that use non-integer numbers would be a workable stopgap (although the temptation might be quite strong in some use cases).

[igrigorik]

Well, the alternatives are: hand-rolled ABNF parsers with buggy substring matches (because that seems to be preferred "optimized" way of doing things); or completely new parses for each platform for some yet-to-be-defined format... shrug, pick your poison.

UA's can enforce the JSON subset ("strict mode") we define here, which will nudge all the other parties in the right direction.

[mnot]

I do agree that not inventing something new is pretty attractive. How you feel about disallowing (or just strongly warning) against non-integer numbers in request headers?

[igrigorik]

Strongly warning seems reasonable. I'm not aware of any existing efforts that are planning to use non-integer values + JFV, but making that a requirement doesn't feel right either.

[rousskov]

A JSON profile, strict modes, wrapper API suggestions, and/or strong warnings simply document attack vectors without solving the underlying problem. If JSON is the answer, then it comes with a "fuzzy numbers" problem that does not have reliable solutions. You can sugar-coat it or hope that it will go away in 20 years, but you cannot solve it.

[kazuho]

@igrigorik @mnot
If we are to going to use a modified variant of JSON, I strongly support disallowing non-integer numbers.

Using ECMA262's definition of number would not help us, neither in terms of the definition or the ease of implementation.

Regarding the definition, my understanding is that ECMA262 merely specifies how the numbers should look like as strings, but does not specify how the strings should be converted to IEEE754 numbers. Lack of a formal definition of such procedure is what leads us to disagreements between the implementation (that in turn becomes a cradle of vulnerabilities).

Regarding the ease of implementation, ECMA262 is a superset of JSON in terms of how numbers can be represented (i.e. binary, octal and hexadecimal representations), which seems unnecessary complex to me.

@mnot

It may be that we can specify a profile of JSON that is safe to use, and create a set of easy-to-use libraries and get them into the relevant server-side platforms, but that's a substantial amount of effort (any volunteers?)

It might be possible to filter out JSON containing floating point representations before applying a JSON parser, and in that regard, using json2.js (that uses eval to parse JSON after filtering out dangerous strings) as a basis might be worth considering. I anticipate that just modifying the number representation in this regexp would be sufficient for the matter.

[igrigorik]

Parallel thread: https://lists.w3.org/Archives/Public/ietf-http-wg/2016JulSep/thread.html#msg436

@rousskov I appreciate where you're coming from, but (personally, at least) I think that paints an overly black and white prognosis. I think we could make reasonable progress here with a restricted set + browsers enforcement.

@kazuho @mnot I can't shake the feeling that restricting to integer values will come back to bite us, but if you think otherwise.. shrug, we can give it a shot.

[clelland]

Presumably, we'll also have to restrict the range of integers to Number.MIN_SAFE_INTEGER <= n <= Number.MAX_SAFE_INTEGER, to be sure that the value parsed is the same as the number presented in the JSON.

[rousskov]

@igrigorik I agree -- reasonable progress is very possible, for some definition of reasonable, especially if your primary focus is "browsers". That reasonable progress will not solve the problem -- fuzzy JSON numbers will cause CVEs. If reasonable progress [with browsers] is good enough, and a few design-triggered CVEs are not important enough, then this JSON problem is not a big deal at all.

[reschke]

FWIW, another plausible thing to do would be to require conformance to the I-JSON profile (see https://tools.ietf.org/html/rfc7493).

@igrigorik -- is that something you would consider implementing (essentially adding a stricter mode to the existing JSON parser?)

[igrigorik]

FWIW, another plausible thing to do would be to require conformance to the I-JSON profile (see https://tools.ietf.org/html/rfc7493).

@reschke that's effectively what we proposed above.. ECMA-262 and I-JSON dictate same rules for number encoding?

---

@clelland @esprehn fyi... phk put together a rough draft of his (alternative) proposal:

• https://lists.w3.org/Archives/Public/ietf-http-wg/2016OctDec/0099.html
• https://datatracker.ietf.org/doc/draft-kamp-httpbis-structure/

[reschke]

@igrigorik it's different with respect to duplicate keys...

[igrigorik]

@reschke true, but what's the benefit of I-JSON over the existing ECMA-262 behavior in browsers?

[reschke]

easier to specify from an IETF spec :-).

At the end of the day, we'll forbid sending duplicate keys anyway, right? The difference is just how the recipient handles those (complain vs last wins)

[igrigorik]

At the end of the day, we'll forbid sending duplicate keys anyway, right? The difference is just how the recipient handles those (complain vs last wins)

Right. I got internal pushback earlier on special-casing this in our JSON parser (to ignore+complain).. last-wins is simple and consistent with the rest of the (browser) platform.

[esprehn]

Yeah we don't want to have two versions of the JSON parser. So whatever this spec says should agree with 262.

[reschke]

@igrigorik - would something like 9928ebd work for you?

[igrigorik]

@reschke yes, I think that looks reasonable.

Furthermore, ordering of object members is not significant and can not be relied upon.

Paging @clelland @esprehn for sanity check..

[mnot]

Closing, since the WG has abandoned JFV in favour of PHK's proposal.