Semantics organisation

[mnot]

I find that I struggle with the current organisation of Semantics every time I open it. When you're looking for how a given mechanism (e.g., conneg, partial content) works, it's very confusing to have to bounce between parts of s 6, s 8, s 10 and sometimes s 9.

I think the document would read far more easily and logically (especially in the text-only rendering) if we added four new top-level sections, with the content from the listed sections inserted (likely with some rearrangement):

• Partial Content
  • 8.3 Range
  • 6.3.4 Content-Range
  • 6.3.5 Media Type multipart/bytesranges
  • 10.4.1 Accept-Ranges
• Content Negotiation
  • 6.4 Content Negotation
  • 8.4 Negotiation
  • 6.2.1 Content-Type
  • 6.2.2 Content-Encoding
  • 6.2.3 Content-Language
• Conditional Requests
  • 8.2 Preconditions
  • 10.2 Validators
• Authentication
  • 10.3 Authentication Challenges
  • 8.5 Authentication Credentials

I haven't suggested moving the status codes, as that would be more disruptive, and I think there isn't much value in doing so.

[reschke]

I think I agree.

It also helps with fields thare both request and response fields. Right now, Accept-Encoding is in Secion 8, although it also defines the response variant.

[royfielding]

The current organization of Semantics is split by request/response, whereas Conditionals, Range, Auth, Conneg, and Caching are better explained in proximity. That is, of course, why we split them for 7231.

I have been thinking of going back to the reference style of 2616, where the major text sections describe what can be done by reference to the detailed sections per element sorted alphabetically.

I can try that after the Apache board meeting today. Did I mention that I am now chairman of the ASF, again?

[reschke]

FWIW, if we do a major re-org, we should avoid to have that mixed with other changes (in the same Internet Draft).

[royfielding]

Yes, we are long overdue for a new draft set. I think we need to stop waiting for meetings to generate them.

[reschke]

Will submit one today.

[royfielding]

Unfortunately, any depth-first approach to organizing the document doesn't work because each protocol element is important for two to four different actions one might want to take in HTTP. We keep getting lost in the details.

For example, placing Content-Type under Content Negotiation doesn't work because negotiation is only relevant to some 5% of resources, whereas everyone needs the self-descriptive aspect of a Content-Type. Range is easier to describe separately, but it has an impact on what any response message is saying (i.e., it fundamentally alters the message semantics such that all of the other cases where we describe "what is in a payload" become wrong if we don't mention 206 and Content-Range as the exception). Likewise, conditional requests depend on understanding methods, header fields, the selected representation, and validators, and Range. Auth is the only one that isn't looped into everything, for better or worse.

What I think we can do is extract the summaries into sections that overview the major protocol features and define them mostly by reference to the individual methods, codes, fields, etc. That would replace/transform the Architecture section to include resources, URIs, representations, and message-based interaction first, followed by the features of content negotiation, conditional requests, range requests, and authentication. We could then follow that with detailed requirements for each method, status code, and header field listed alphabetically.

Meanwhile, I am still trying to find the right balance between the Notation sections and the Conformance section. I am thinking of merging them into a section below the Intro.

Something like:

1.   Introduction
    1.1.   Purpose
    1.2.   Evolution
    1.3.   Semantics
    1.4.   Obsoletes
2.   Conformance
    2.1.   Syntax Notation
    2.2.   Requirements Notation
    2.3.   Role-based Requirements
    2.4.   Length Requirements
    2.5.   Error Handling
3.  Interaction
    3.1.  Resources
            3.1.1. URI
            3.1.2. References
            3.1.3. Target URI
    3.2.  Representations
    3.3.  Components
            3.3.1.  Client/Server
            3.3.2.  Intermediaries
            3.3.3.  Caches
            3.3.4.  Implementation Diversity
    3.3.  Messages
            3.3.1.  Framing
            3.3.2.  Fields
            3.3.3.  Payload
4.  Identifier Schemes
     4.1.  http URI Scheme
     4.2.  https URI Scheme
     4.3.  http and https URI Normalization and Comparison
     4.4.  Deprecated userinfo
     4.5.  Fragment Identifiers on http(s) URI References
     4.6.  Schemes that alias http(s)
5.  Routing Messages
     5.1.  Target URI
     5.2.  Origin
     5.3.  Authoritative Responses
6.  Requests
     6.1.  Methods
             6.1.1.  Safe
             6.1.2.  Idempotent
             6.1.3.  Impact on Caching
     6.2.  Request Fields
     6.3.  Request Payload
7.  Responses
     7.1.  Status Codes
     7.2.  Response Fields
     7.3.  Response Payload
8.  Features
     8.1.  Authentication
     8.2.  Content Negotiation
     8.3.  Preconditions
     8.4.  Range Requests
9.  Method Definitions
10.  Status Code Definitions
11.  Field Definitions
12.  Security Considerations
13.  IANA Considerations
14.  References

[mnot]

That looks like a good start. Based upon the ToC above, a few comments/suggestions:

Where does "extending and versioning HTTP" (current 4) go?

Re: 3 - 'Interaction' isn't quite the right classifier; perhaps 'Concepts'? Back to 'Architecture'? Hm. I'm not sure why these things are bucketed together, and why other things aren't included.I'm tempted to say that they should be promoted to top-level sections (Resources, Representations, Components and Messages) -- that would also help avoid some very deep headers (e.g., in fields).

Overall, it would be good if the section titles were a bit more descriptive.

Features should be higher; e.g., before Requests. I'd also consider making each of its items a top-level section.

[reschke]

/me not sure about "Syntax Notation" below "Conformance".

[royfielding]

So, promote subsections of Interaction up, move Features closer to front, consider discussing basic method features just before that.

[royfielding]

and move Routing closer to URIs

[royfielding]

My reorg proposal is pretty much complete, since I can't do much else without adding new text. It might be possible (and make more sense) to place IANA Considerations before Security Considerations, but that might upset someone.

The current outline of #446 looks like:

    1.   Introduction
        1.1.   Purpose
        1.2.   Evolution
        1.3.   Semantics
        1.4.   Obsoletes
    2.   Conformance
        2.1.   Syntax Notation
        2.2.   Requirements Notation
        2.3.   Length Requirements
        2.4.   Error Handling
    3.   Terminology
        3.1.   Resources
        3.2.   Connections
        3.3.   Messages
        3.4.   User Agent
        3.5.   Origin Server
        3.6.   Example Request and Response
        3.7.   Intermediaries
        3.8.   Caches
    4.   Identifiers
        4.1.   URI References
        4.2.   URI Schemes
            4.2.1.   http URI Scheme
            4.2.2.   https URI Scheme
            4.2.3.   http(s) Normalization and Comparison
            4.2.4.   http(s) Deprecated userinfo
            4.2.5.   http(s) References with Fragment Identifiers
        4.3.   Authoritative Access
            4.3.1.   URI Origin
            4.3.2.   http origins
            4.3.3.   https origins
            4.3.4.   https certificate verification
    5.   Message Abstraction
        5.1.   Protocol Version
        5.2.   Framing
        5.3.   Control Data
            5.3.1.   Request
            5.3.2.   Response
        5.4.   Header Fields
            5.4.1.   Field Ordering and Combination
            5.4.2.   Field Limits
            5.4.3.   Field Names
            5.4.4.   Field Values
        5.5.   Payload
            5.5.1.   Purpose
            5.5.2.   Identification
            5.5.3.   Payload Metadata
            5.5.4.   Payload Body
        5.6.   Trailer Fields
            5.6.1.   Purpose
            5.6.2.   Limitations
            5.6.3.   Processing
        5.7.   Common Rules for Defining Field Values
            5.7.1.   Lists (#rule ABNF Extension)
                5.7.1.1.   Sender Requirements
                5.7.1.2.   Recipient Requirements
            5.7.2.   Tokens
            5.7.3.   Whitespace
            5.7.4.   Quoted Strings
            5.7.5.   Comments
            5.7.6.   Parameters
            5.7.7.   Date/Time Formats
    6.   Routing
        6.1.   Target Resource
            6.1.1.   Request Target
            6.1.2.   Host
            6.1.3.   Reconstructing the Target URI
        6.2.   Routing Inbound
            6.2.1.   To a Cache
            6.2.2.   To a Proxy
            6.2.3.   To the Origin
        6.3.   Response Correlation
        6.4.   Message Forwarding
            6.4.1.   Connection
            6.4.2.   Max-Forwards
            6.4.3.   Via
        6.5.   Transformations
        6.6.   Upgrade
    7.   Representations
        7.1.   Selected Representation
        7.2.   Data
        7.3.   Metadata
        7.4.   Content-Type
            7.4.1.   Media Type
            7.4.2.   Charset
            7.4.3.   Canonicalization and Text Defaults
            7.4.4.   Multipart Types
        7.5.   Content-Encoding
            7.5.1.   Content Codings
                7.5.1.1.   Compress Coding
                7.5.1.2.   Deflate Coding
                7.5.1.3.   Gzip Coding
        7.6.   Content-Language
            7.6.1.   Language Tags
        7.7.   Content-Length
        7.8.   Content-Location
        7.9.   Validators
            7.9.1.   Weak versus Strong
            7.9.2.   Last-Modified
                7.9.2.1.   Generation
                7.9.2.2.   Comparison
            7.9.3.   ETag
                7.9.3.1.   Generation
                7.9.3.2.   Comparison
                7.9.3.3.   Example: Entity-Tags Varying on Content-Negotiated Resources
            7.9.4.   When to Use Entity-Tags and Last-Modified Dates
    8.   Methods
        8.1.   Overview
        8.2.   Common Method Properties
            8.2.1.   Safe Methods
            8.2.2.   Idempotent Methods
            8.2.3.   Methods and Caching
        8.3.   Method Definitions
            8.3.1.   GET
            8.3.2.   HEAD
            8.3.3.   POST
            8.3.4.   PUT
            8.3.5.   DELETE
            8.3.6.   CONNECT
            8.3.7.   OPTIONS
            8.3.8.   TRACE
    9.   Context
        9.1.   Request Context
            9.1.1.   Expect
            9.1.2.   From
            9.1.3.   Referer
            9.1.4.   TE
            9.1.5.   Trailer
            9.1.6.   User-Agent
        9.2.   Response Context
            9.2.1.   Allow
            9.2.2.   Date
            9.2.3.   Location
            9.2.4.   Retry-After
            9.2.5.   Server
    10.   Authentication
        10.1.   Authentication Scheme
        10.2.   Authentication Parameters
        10.3.   Challenge and Response
        10.4.   Credentials
        10.5.   Protection Space (Realm)
        10.6.   Authenticating User to Origin Server
            10.6.1.   WWW-Authenticate
            10.6.2.   Authorization
            10.6.3.   Authentication-Info
        10.7.   Authenticating Client to Proxy
            10.7.1.   Proxy-Authenticate
            10.7.2.   Proxy-Authorization
            10.7.3.   Proxy-Authentication-Info
    11.   Content Negotiation
        11.1.   Proactive Negotiation
            11.1.1.   Shared Negotiation Features
                11.1.1.1.   Absence
                11.1.1.2.   Quality Values
                11.1.1.3.   Wildcard Values
            11.1.2.   Accept
            11.1.3.   Accept-Charset
            11.1.4.   Accept-Encoding
            11.1.5.   Accept-Language
        11.2.   Reactive Negotiation
            11.2.1.   Vary
        11.3.   Request Payload Negotiation
    12.   Conditional Requests
        12.1.   Preconditions
            12.1.1.   If-Match
            12.1.2.   If-None-Match
            12.1.3.   If-Modified-Since
            12.1.4.   If-Unmodified-Since
            12.1.5.   If-Range
        12.2.   Evaluation
        12.3.   Precedence
    13.   Range Requests
        13.1.   Range Units
            13.1.1.   Range Specifiers
            13.1.2.   Byte Ranges
        13.2.   Range
        13.3.   Accept-Ranges
        13.4.   Content-Range
        13.5.   Media Type multipart/byteranges
    14.   Status Codes
        14.1.   Overview of Status Codes
        14.2.   Informational 1xx
            14.2.1.   100 Continue
            14.2.2.   101 Switching Protocols
        14.3.   Successful 2xx
            14.3.1.   200 OK
            14.3.2.   201 Created
            14.3.3.   202 Accepted
            14.3.4.   203 Non-Authoritative Information
            14.3.5.   204 No Content
            14.3.6.   205 Reset Content
            14.3.7.   206 Partial Content
                14.3.7.1.   Single Part
                14.3.7.2.   Multiple Parts
                14.3.7.3.   Combining Parts
        14.4.   Redirection 3xx
            14.4.1.   300 Multiple Choices
            14.4.2.   301 Moved Permanently
            14.4.3.   302 Found
            14.4.4.   303 See Other
            14.4.5.   304 Not Modified
            14.4.6.   305 Use Proxy
            14.4.7.   306 (Unused)
            14.4.8.   307 Temporary Redirect
            14.4.9.   308 Permanent Redirect
        14.5.   Client Error 4xx
            14.5.1.   400 Bad Request
            14.5.2.   401 Unauthorized
            14.5.3.   402 Payment Required
            14.5.4.   403 Forbidden
            14.5.5.   404 Not Found
            14.5.6.   405 Method Not Allowed
            14.5.7.   406 Not Acceptable
            14.5.8.   407 Proxy Authentication Required
            14.5.9.   408 Request Timeout
            14.5.10.   409 Conflict
            14.5.11.   410 Gone
            14.5.12.   411 Length Required
            14.5.13.   412 Precondition Failed
            14.5.14.   413 Payload Too Large
            14.5.15.   414 URI Too Long
            14.5.16.   415 Unsupported Media Type
            14.5.17.   416 Range Not Satisfiable
            14.5.18.   417 Expectation Failed
            14.5.19.   418 (Unused)
            14.5.20.   422 Unprocessable Payload
            14.5.21.   426 Upgrade Required
        14.6.   Server Error 5xx
            14.6.1.   500 Internal Server Error
            14.6.2.   501 Not Implemented
            14.6.3.   502 Bad Gateway
            14.6.4.   503 Service Unavailable
            14.6.5.   504 Gateway Timeout
            14.6.6.   505 HTTP Version Not Supported
    15.   Extending HTTP
        15.1.   Method Extensibility
            15.1.1.   Method Registry
            15.1.2.   Considerations for New Methods
        15.2.   Status Code Extensibility
            15.2.1.   Status Code Registry
            15.2.2.   Considerations for New Status Codes
        15.3.   Field Name Extensibility
            15.3.1.   Field Name Registry
            15.3.2.   Considerations for New Field Names
            15.3.3.   Considerations for New Field Values
        15.4.   Authentication Scheme Extensibility
            15.4.1.   Authentication Scheme Registry
            15.4.2.   Considerations for New Authentication Schemes
        15.5.   Range Unit Extensibility
            15.5.1.   Range Unit Registry
            15.5.2.   Considerations for New Range Units
        15.6.   Content Coding Extensibility
            15.6.1.   Content Coding Registry
            15.6.2.   Considerations for New Content Codings
        15.7.   Upgrade Token Registry
    16.   Security Considerations
        16.1.   Establishing Authority
        16.2.   Risks of Intermediaries
        16.3.   Attacks Based on File and Path Names
        16.4.   Attacks Based on Command, Code, or Query Injection
        16.5.   Attacks via Protocol Element Length
        16.6.   Attacks using Shared-dictionary Compression
        16.7.   Disclosure of Personal Information
        16.8.   Privacy of Server Log Information
        16.9.   Disclosure of Sensitive Information in URIs
        16.10.   Disclosure of Fragment after Redirects
        16.11.   Disclosure of Product Information
        16.12.   Browser Fingerprinting
        16.13.   Validator Retention
        16.14.   Denial-of-Service Attacks Using Range
        16.15.   Authentication Considerations
            16.15.1.   Confidentiality of Credentials
            16.15.2.   Credentials and Idle Clients
            16.15.3.   Protection Spaces
            16.15.4.   Additional Response Fields
    17.   IANA Considerations
        17.1.   URI Scheme Registration
        17.2.   Method Registration
        17.3.   Status Code Registration
        17.4.   HTTP Field Name Registration
        17.5.   Authentication Scheme Registration
        17.6.   Content Coding Registration
        17.7.   Range Unit Registration
        17.8.   Media Type Registration
        17.9.   Port Registration
        17.10.   Upgrade Token Registration
    18.   References
        18.1.   Normative References
        18.2.   Informative References

[mnot]

I say ship it and we'll go from there.

[reschke]

Thaniks, Roy.

I'm on the road today, but I'll do a sanity check and then submit drafts tomorrow!

[royfielding]

Merging .... Wheeeeeeeeee!