Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Structure limits #578

Merged
merged 8 commits into from May 21, 2018
Merged

Structure limits #578

Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
cabf1ca
Remove size limits, except numbers and identifiers
mnot Apr 12, 2018
eb3aa40
Add security consideration about header sizes
mnot Apr 12, 2018
840d6bb
Add minimum limits for parsers.
mnot Apr 12, 2018
6a9ad12
Merge branch 'master' into structure-limits
mnot May 15, 2018
7bf7ff8
Merge branch 'master' into structure-limits
mnot May 15, 2018
0d77829
talk about header size limits
mnot May 15, 2018
1745c45
Remove length constraint on identifiers
mnot May 16, 2018
2c06d04
not 1*
mnot May 16, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
87 changes: 46 additions & 41 deletions draft-ietf-httpbis-header-structure.md
View file
Open in desktop
Expand Up @@ -153,6 +153,8 @@ Section 4.2), it MUST be resolved ([RFC3986], Section 5) before being
used.
~~~

This specification defines minimums for the length or number of various structures supported by Structured Headers implementations. It does not specify maximum sizes in most cases, but header authors should be aware that HTTP implementations do impose various limits on the size of individual header fields, the total number of fields, and/or the size of the entire header block.

Note that specifications using Structured Headers do not re-specify its ABNF or parsing algorithms; instead, they should be specified in terms of its abstract data structures.

Also, empty header field values are not allowed, and therefore parsing for them will fail.
Expand Down Expand Up @@ -189,12 +191,12 @@ This section defines the abstract value types that can be composed into Structur

## Dictionaries {#dictionary}

Dictionaries are unordered maps of key-value pairs, where the keys are identifiers ({{identifier}}) and the values are items ({{item}}). There can be from 1 to 1024 members, and keys are required to be unique.
Dictionaries are unordered maps of key-value pairs, where the keys are identifiers ({{identifier}}) and the values are items ({{item}}). There can be one or more members, and keys are required to be unique.

In the textual HTTP serialisation, keys and values are separated by "=" (without whitespace), and key/value pairs are separated by a comma with optional whitespace. Duplicate keys MUST cause parsing to fail.

~~~ abnf
dictionary = dict-member 0*1023( OWS "," OWS dict-member )
dictionary = dict-member *( OWS "," OWS dict-member )
dict-member = identifier "=" item
~~~

Expand All @@ -206,6 +208,8 @@ Example-DictHeader: foo=1.23, en="Applepie", da=*w4ZibGV0w6ZydGUK=*

Typically, a header field specification will define the semantics of individual keys, as well as whether their presence is required or optional. Recipients MUST ignore keys that are undefined or unknown, unless the header field's specification specifically disallows them.

Parsers MUST support dictionaries containing at least 1024 key/value pairs.


### Parsing a Dictionary from Text {#parse-dictionary}

Expand All @@ -218,23 +222,22 @@ Given an ASCII string input_string, return a mapping of (identifier, item). inpu
3. Consume a "=" from input_string; if none is present, fail parsing.
4. Let this_value be the result of running Parse Item from Text ({{parse-item}}) with input_string.
5. Add key this_key with value this_value to dictionary.
6. If dictionary has more than 1024 members, fail parsing.
7. Discard any leading OWS from input_string.
8. If input_string is empty, return dictionary.
9. Consume a COMMA from input_string; if no comma is present, fail parsing.
0. Discard any leading OWS from input_string.
1. If input_string is empty, fail parsing.
6. Discard any leading OWS from input_string.
7. If input_string is empty, return dictionary.
8. Consume a COMMA from input_string; if no comma is present, fail parsing.
9. Discard any leading OWS from input_string.
0. If input_string is empty, fail parsing.
3. No structured data has been found; fail parsing.


## Lists {#list}

Lists are arrays of items ({{item}}) with one to 1024 members.
Lists are arrays of items ({{item}}) with one or more members.

In the textual HTTP serialisation, each member is separated by a comma and optional whitespace.

~~~ abnf
list = list-member 0*1023( OWS "," OWS list-member )
list = list-member *( OWS "," OWS list-member )
list-member = item
~~~

Expand All @@ -244,6 +247,8 @@ For example, a header field whose value is defined as a list of strings could lo
Example-StrListHeader: "foo", "bar", "It was the best of times."
~~~

Parsers MUST support lists containing at least 1024 members.


### Parsing a List from Text {#parse-list}

Expand All @@ -253,26 +258,25 @@ Given an ASCII string input_string, return a list of items. input_string is modi
2. While input_string is not empty:
1. Let item be the result of running Parse Item from Text ({{parse-item}}) with input_string.
2. Append item to items.
3. If items has more than 1024 members, fail parsing.
4. Discard any leading OWS from input_string.
5. If input_string is empty, return items.
6. Consume a COMMA from input_string; if no comma is present, fail parsing.
7. Discard any leading OWS from input_string.
8. If input_string is empty, fail parsing.
3. Discard any leading OWS from input_string.
4. If input_string is empty, return items.
5. Consume a COMMA from input_string; if no comma is present, fail parsing.
6. Discard any leading OWS from input_string.
7. If input_string is empty, fail parsing.
3. No structured data has been found; fail parsing.


## Parameterised Lists {#param}

Parameterised Lists are arrays of a parameterised identifiers with 1 to 256 members.
Parameterised Lists are arrays of a parameterised identifiers.

A parameterised identifier is an identifier ({{identifier}}) with up to 256 parameters, each parameter having a identifier and an optional value that is an item ({{item}}). Ordering between parameters is not significant, and duplicate parameters MUST cause parsing to fail.
A parameterised identifier is an identifier ({{identifier}}) with an optional set of parameters, each parameter having a identifier and an optional value that is an item ({{item}}). Ordering between parameters is not significant, and duplicate parameters MUST cause parsing to fail.

In the textual HTTP serialisation, each parameterised identifier is separated by a comma and optional whitespace. Parameters are delimited from each other using semicolons (";"), and equals ("=") delimits the parameter name from its value.

~~~ abnf
param-list = param-id 0*255( OWS "," OWS param-id )
param-id = identifier 0*256( OWS ";" OWS identifier [ "=" item ] )
param-list = param-id *( OWS "," OWS param-id )
param-id = identifier *( OWS ";" OWS identifier [ "=" item ] )
~~~

For example,
Expand All @@ -281,6 +285,8 @@ For example,
Example-ParamListHeader: abc_123;a=1;b=2; c, def_456, ghi;q="19";r=foo
~~~

Parsers MUST support parameterised lists containing at least 1024 members, and support members with at least 256 parameters.


### Parsing a Parameterised List from Text {#parse-param-list}

Expand All @@ -290,12 +296,11 @@ Given an ASCII string input_string, return a list of parameterised identifiers.
2. While input_string is not empty:
1. Let item be the result of running Parse Parameterised Identifier from Text ({{parse-param-id}}) with input_string.
2. Append item to items.
3. If items has more than 256 members, fail parsing.
4. Discard any leading OWS from input_string.
5. If input_string is empty, return items.
6. Consume a COMMA from input_string; if no comma is present, fail parsing.
7. Discard any leading OWS from input_string.
8. If input_string is empty, fail parsing.
3. Discard any leading OWS from input_string.
4. If input_string is empty, return items.
5. Consume a COMMA from input_string; if no comma is present, fail parsing.
6. Discard any leading OWS from input_string.
7. If input_string is empty, fail parsing.
3. No structured data has been found; fail parsing.


Expand All @@ -316,8 +321,7 @@ Given an ASCII string input_string, return a identifier with an mapping of param
8. If the first character of input_string is "=":
1. Consume the "=" character at the beginning of input_string.
2. Let param_value be the result of Parsing an Item from Text ({{parse-item}}) from input_string.
9. If parameters has more than 255 members, fail parsing.
0. Insert (param_name, param_value) into parameters.
9. Insert (param_name, param_value) into parameters.
4. Return the tuple (primary_identifier, parameters).


Expand Down Expand Up @@ -419,12 +423,12 @@ See {{parse-number}} for the parsing algorithm for floats.

## Strings {#string}

Abstractly, strings are up to 1024 printable ASCII {{!RFC0020}} characters (i.e., the range 0x20 to 0x7E). Note that this excludes tabs, newlines, carriage returns, etc.
Abstractly, strings are zero or more printable ASCII {{!RFC0020}} characters (i.e., the range 0x20 to 0x7E). Note that this excludes tabs, newlines, carriage returns, etc.

The textual HTTP serialisation of strings uses a backslash ("\\") to escape double quotes and backslashes in strings.

~~~ abnf
string = DQUOTE 0*1024(chr) DQUOTE
string = DQUOTE *(chr) DQUOTE
chr = unescaped / escaped
unescaped = %x20-21 / %x23-5B / %x5D-7E
escaped = "\" ( DQUOTE / "\" )
Expand All @@ -442,6 +446,8 @@ Unicode is not directly supported in this document, because it causes a number o

When it is necessary for a field value to convey non-ASCII string content, binary content ({{binary}}) SHOULD be specified, along with a character encoding (preferably, UTF-8).

Parsers MUST support strings with at least 1024 characters.


### Parsing a String from Text {#parse-string}

Expand All @@ -460,16 +466,15 @@ Given an ASCII string input_string, return an unquoted string. input_string is m
3. Append next_char to output_string.
3. Else, if char is DQUOTE, return output_string.
4. Else, append char to output_string.
5. If output_string contains more than 1024 characters, fail parsing.
6. Otherwise, fail parsing.


## Identifiers {#identifier}

Identifiers are short (up to 256 characters) textual identifiers; their abstract model is identical to their expression in the textual HTTP serialisation.
Identifiers are short textual identifiers; their abstract model is identical to their expression in the textual HTTP serialisation. Parsers MUST support identifiers with at least 64 characters.

~~~ abnf
identifier = lcalpha 0*255( lcalpha / DIGIT / "_" / "-"/ "*" / "/" )
identifier = lcalpha *( lcalpha / DIGIT / "_" / "-"/ "*" / "/" )
lcalpha = %x61-7A ; a-z
~~~

Expand All @@ -489,13 +494,12 @@ Given an ASCII string input_string, return a identifier. input_string is modifie
1. Prepend char to input_string.
2. Return output_string.
3. Append char to output_string.
1. If output_string contains more than 256 characters, fail parsing.
4. Return output_string.


## Binary Content {#binary}

Arbitrary binary content up to 16384 bytes in size can be conveyed in Structured Headers.
Arbitrary binary content can be conveyed in Structured Headers.

The textual HTTP serialisation encodes the data using Base 64 Encoding {{!RFC4648}}, Section 4, and surrounds it with a pair of asterisks ("\*") to delimit from other content.

Expand All @@ -510,7 +514,7 @@ not be possible with some base64 implementations.
This specification does not relax the requirements in {{!RFC4648}}, Section 3.1 and 3.3; therefore, parsers MUST fail on characters outside the base64 alphabet, and on line feeds in encoded data.

~~~ abnf
binary = "*" 0*21846(base64) "*"
binary = "*" *(base64) "*"
base64 = ALPHA / DIGIT / "+" / "/" / "="
~~~

Expand All @@ -520,6 +524,8 @@ For example, a header whose value is defined as binary content could look like:
Example-BinaryHeader: *cHJldGVuZCB0aGlzIGlzIGJpbmFyeSBjb250ZW50Lg==*
~~~

Parsers MUST support binary content with at least 16384 octets after decoding.


### Parsing Binary Content from Text {#parse-binary}

Expand All @@ -529,9 +535,8 @@ Given an ASCII string input_string, return binary content. input_string is modif
2. Discard the first character of input_string.
3. Let b64_content be the result of removing content of input_string up to but not including the first instance of the character "\*". If there is not a "\*" character before the end of input_string, fail parsing.
4. Consume the "\*" character at the beginning of input_string.
5. If b64_content is has more than 21846 characters, fail parsing.
6. Let binary_content be the result of Base 64 Decoding {{!RFC4648}} b64_content, synthesising padding if necessary (note the requirements about recipient behaviour in {{binary}}).
7. Return binary_content.
5. Let binary_content be the result of Base 64 Decoding {{!RFC4648}} b64_content, synthesising padding if necessary (note the requirements about recipient behaviour in {{binary}}).
6. Return binary_content.



Expand All @@ -542,8 +547,7 @@ This draft has no actions for IANA.

# Security Considerations

TBD

The size of most types defined by Structured Headers is not limited; as a result, extremely large header fields could be an attack vector (e.g., for resource consumption). Most HTTP implementations limit the sizes of size of individual header fields as well as the overall header block size to mitigate such attacks.

It is possible for parties with the ability to inject new HTTP header fields to change the meaning
of a Structured Headers. In some circumstances, this will cause parsing to fail, but it is not possible to reliably fail in all such circumstances.
Expand All @@ -557,6 +561,7 @@ of a Structured Headers. In some circumstances, this will cause parsing to fail,
## Since draft-ietf-httpbis-header-structure-04

* Remove identifiers from item.
* Remove most limits on sizes.


## Since draft-ietf-httpbis-header-structure-03
Expand Down