Cosmetic fixes/cleanups

CHANGELOG.rst

@@ -7,22 +7,22 @@ Unreleased
 Changed
 -------
 - Notice `1277`_ (obsolete 'X-' prefix) is now reported only once per message.
-- HTTPolice no longer attempts to process very long header lines (currently
-  16K; they will fail with notice `1006`_/`1009`_) and message bodies
-  (currently 1G; notice `1298`_).
+- When parsing TCP streams, HTTPolice no longer attempts to process very long
+  header lines (currently 16K; they will fail with notice `1006`_/`1009`_)	
+  and message bodies (currently 1G; notice `1298`_).
 - The syntax of `chunk extensions`_ is no longer checked.
 
 Added
 -----
-- HTTPolice can now use much less memory when parsing long TCP streams.
-- Similarly, large HTML reports need much less memory to produce.
 - Checks for the `Forwarded`_ header (notices `1296`_, `1297`_).
-- Minor speedup in case when request URLs often repeat.
 
 Fixed
 -----
+- HTTPolice can now use much less memory when parsing long TCP streams.
+- Similarly, large HTML reports need much less memory to produce.
 - Notice `1013`_ is no longer wrongly reported for some headers
   such as ``Vary``.
+- Minor speedup in case when request URLs often repeat.
 - Fixed a crash on some pathological values of ``charset`` in ``Content-Type``.
 
 .. _Forwarded: https://tools.ietf.org/html/rfc7239

README.rst

@@ -11,9 +11,9 @@ HTTPolice
 .. image:: https://travis-ci.org/vfaronov/httpolice.svg?branch=master
    :target: https://travis-ci.org/vfaronov/httpolice
 
-HTTPolice is a **validator for HTTP requests and responses**. It can spot bad
-header syntax, inappropriate status codes, and other interoperability problems
-in your HTTP server or client.
+HTTPolice is a **validator or “linter” for HTTP requests and responses**.
+It can spot bad header syntax, inappropriate status codes, and other potential
+problems in your HTTP server or client.
 
 See `example report`__.
 

doc/index.rst

@@ -1,9 +1,9 @@
 HTTPolice user manual
 =====================
 
-`HTTPolice`__ is a validator for HTTP requests and responses. It can spot bad
-header syntax, inappropriate status codes, and other interoperability problems
-in your HTTP server or client.
+`HTTPolice`__ is a validator or “linter” for HTTP requests and responses.
+It can spot bad header syntax, inappropriate status codes, and other potential
+problems in your HTTP server or client.
 
 __ https://github.com/vfaronov/httpolice
 

httpolice/framing1.py

@@ -40,10 +40,10 @@ def parse_streams(inbound, outbound, scheme=None):
     is unreliable, because response framing depends on the request.
 
     :param inbound:
-        The inbound (request) stream as a :class:`~httpolice.parse.Stream`,
+        The inbound (request) stream as a :class:`~httpolice.stream.Stream`,
         or `None`.
     :param outbound:
-        The outbound (response) stream as a :class:`~httpolice.parse.Stream`,
+        The outbound (response) stream as a :class:`~httpolice.stream.Stream`,
         or `None`.
     :param scheme:
         The scheme of the request URI, as a Unicode string,
@@ -242,7 +242,7 @@ def _parse_response_body(resp, stream):
 def parse_header_fields(stream):
     """Parse a block of HTTP/1.x header fields.
 
-    :param stream: The :class:`~httpolice.parse.Stream` from which to parse.
+    :param stream: The :class:`~httpolice.stream.Stream` from which to parse.
     :return: A list of :class:`HeaderEntry`.
     :raises: :class:`ParseError`
     """

httpolice/message.py

@@ -72,6 +72,7 @@ def rebuild_headers(self):
 
     @derived_property
     def decoded_body(self):
+        """The payload body with Content-Encoding removed."""
         r = self.body
         codings = self.headers.content_encoding.value[:]
         while codings and okay(r) and r:
@@ -116,13 +117,14 @@ def guessed_charset(self):
     def unicode_body(self):
         if not okay(self.decoded_body):
             return self.decoded_body
-        if self.guessed_charset is None:
+        if not okay(self.guessed_charset):
             return Unavailable(self.decoded_body)
         # pylint: disable=no-member
         return self.decoded_body.decode(self.guessed_charset)
 
     @derived_property
     def content_is_full(self):
+        """Does this message carry a complete instance of its Content-Type?"""
         return True
 
     @derived_property
@@ -139,7 +141,7 @@ def json_data(self):
                 if self.guessed_charset not in ['ascii', 'utf-8', 'utf-16',
                                                 'utf-16-le', 'utf-16-be',
                                                 'utf-32', 'utf-32-le',
-                                                'utf-32-be']:
+                                                'utf-32-be', None]:
                     self.complain(1281)
             return r
         else:
@@ -205,6 +207,11 @@ def url_encoded_data(self):
 
     @derived_property
     def displayable_body(self):
+        """
+        The payload body in a form that is appropriate for display in a message
+        preview, along with a list of phrases explaining which transformations
+        have been applied to arrive at that form.
+        """
         removing_te = [u'removing Transfer-Encoding'] \
             if self.headers.transfer_encoding else []
         removing_ce = [u'removing Content-Encoding'] \

httpolice/notices.xml

@@ -1242,7 +1242,10 @@ One non-obvious thing is how references work
 
   <debug id="1168">
     <title><h>Age</h> header implies response from cache</title>
-    <explain>This response has an <h>Age</h> header, which implies that it was served from a cache, without contacting the origin server.</explain>
+    <rfc num="7234" sect="5.1">
+      The presence of an <h>Age</h> header field implies that the response was not
+   generated or validated by the origin server for this request.
+    </rfc>
   </debug>
 
   <debug id="1169">
@@ -2016,7 +2019,7 @@ One non-obvious thing is how references work
   <error id="1274">
     <title><var ref="header"/> header without credentials</title>
     <explain>An <var ref="header"/> header usually consists of two parts: the name of the authentication scheme, and the actual credentials. But in this request, it only contains the scheme (“<var ref="header.value.item"/>”).</explain>
-    <explain>Did you mean something like “<var ref="header"/>: <auth>Bearer</auth> <var ref="header.value.item"/>”?</explain>
+    <explain>Was something like “<var ref="header"/>: <auth>Bearer</auth> <var ref="header.value.item"/>” intended?</explain>
     <explain>See also <rfc num="7235" sect="2"/>.</explain>
     <!-- This is an error, not a comment, because it's hard to imagine a conformant use of ``credentials`` consisting only of the ``auth-scheme`` (in particular, none of the schemes registered with IANA allow this), whereas forgetting the ``auth-scheme`` is a common mistake. -->
   </error>
@@ -2031,7 +2034,7 @@ One non-obvious thing is how references work
     <title><var ref="header"/>: <var ref="wildcard"/> is as good as <var ref="value"/></title>
     <explain>The <var ref="header"/> header is not an ordered list. Whether <var ref="wildcard"/> comes before or after <var ref="value"/>, does not affect its priority. Only explicit quality values (q=) count.</explain>
     <explain>In other words, this request says that any <var ref="wildcard"/> is as acceptable as <var ref="value"/>.</explain>
-    <explain>If you want to make <var ref="wildcard"/> a fallback, you should add a lower quality value to it, for example: <var ref="wildcard"/>;q=0.1</explain>
+    <explain>To make <var ref="wildcard"/> a fallback, it should have a lower quality value, for example: <var ref="wildcard"/>;q=0.1</explain>
     <explain>See <rfc num="7231" sect="5.3.1"/>.</explain>
   </error>
 
@@ -2117,6 +2120,7 @@ One non-obvious thing is how references work
   <error id="1288">
     <title><m>GET</m> request cannot <h>Prefer</h>: <prefer>return</prefer>=minimal</title>
     <explain>The whole point of a <m>GET</m> request is to retrieve a representation of the resource (as in “<prefer>return</prefer>=representation”). There is no “minimal” response that could satisfy this request.</explain>
+    <explain>Instead, the <h>Accept</h> header could be used to select the desired kind of representation.</explain>
   </error>
 
   <error id="1289">

httpolice/parse.py

@@ -32,7 +32,7 @@
 For example, in an HTML report,
 the ``text/xml`` in ``Accept: text/xml;q=0.9`` becomes a hyperlink to RFC,
 because it is parsed into a :class:`~httpolice.structure.MediaType` object.
-The list of classes to annotate must be passed to :meth:`Stream.parse`.
+The list of classes to annotate must be passed to :func:`parse`.
 Also, the object (in this case, ``MediaType(u'text/xml')``)
 must be the end result of a distinct :class:`Nonterminal`,
 **not** buried inside some :class:`Rule`.

httpolice/reports/common.py

@@ -69,7 +69,7 @@ def expand_parse_error(error):
         if option:
             para.extend([option] if i == 0 else [u'or ', option])
             if symbols:
-                para.append(u', as part of ')
+                para.append(u' as part of ')
         for j, symbol in enumerate(symbols or []):
             para.extend([symbol] if j == 0 else [u' or ', symbol])
         paras.append(para)

httpolice/response.py

@@ -101,6 +101,7 @@ def __repr__(self):
 
     @derived_property
     def content_is_full(self):
+        """Does this response carry a complete instance of its Content-Type?"""
         if self.status == st.not_modified:
             return False
         if self.status == st.partial_content and \