Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix numerous message parsing issues (v2) #3113

Merged
merged 16 commits into from
Dec 25, 2023
Merged

Commits on Dec 15, 2023

  1. pytest: raise on malformed test fixtures

    and unbreak test depending on backslash escape
    pajod committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    559caf9 View commit details
    Browse the repository at this point in the history
  2. RFC compliant header field+chunk validation

    * update HEADER_RE and HEADER_VALUE_RE to match the RFCs
    * update chunk length parsing to disallow 0x prefix and digit-separating underscores.
    kenballus authored and pajod committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    2dbe49d View commit details
    Browse the repository at this point in the history
  3. Disallow empty header names.

    kenballus authored and pajod committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    735e9e8 View commit details
    Browse the repository at this point in the history
  4. RFC compliant request line and header parsing

    - Unify HEADER_RE and METH_RE
    - Replace CRLF with SP during obs-fold processing (See RFC 9112 Section 5.2, last paragraph)
    - Stop stripping header names.
    - Remove HTAB in OWS in header values that use obs-fold (See RFC 9112 Section 5.2, last paragraph)
    - Use fullmatch instead of search, which has problems with empty strings. (See GHSA-68xg-gqqm-vgj8)
    - Split proxy protocol line on space only. (See proxy protocol Section 2.1, bullet 3)
    - Use fullmatch for method and version (Thank you to Paul Dorn for noticing this.)
    - Replace calls to str.strip() with str.strip(' \t')
    - Split request line on SP only.
    
    Co-authored-by: Paul Dorn <pajod@users.noreply.github.com>
    kenballus and pajod committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    72238fc View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    f0c91cc View commit details
    Browse the repository at this point in the history
  6. Create SECURITY.md

    pajod committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    13027ef View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    42dd419 View commit details
    Browse the repository at this point in the history
  8. strict: header field validation: stop casefolding

    * refusing lowercase and ASCII 0x23 (#) had been partially enforced before
    * do not casefold by default, HTTP methods are case sensitive
    pajod committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    b284678 View commit details
    Browse the repository at this point in the history
  9. silently drop or refuse header names w/ underscore

    Ambiguous mappings open a bottomless pit of "what is user input and what is proxy input" confusion.
    Default to what everyone else has been doing for years now, silently drop.
    
    see also https://nginx.org/r/underscores_in_headers
    pajod committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    72b8970 View commit details
    Browse the repository at this point in the history
  10. unconditionally log request error

    Somehow exception logging was conditional on successful request uri parsing.
    Add it back for the other branch.
    pajod committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    0b10cba View commit details
    Browse the repository at this point in the history
  11. fail-safe on unsupported request framing

    If we promise wsgi.input_terminated, we better get it right - or not at all.
    * chunked encoding on HTTP <= 1.1
    * chunked not last transfer coding
    * multiple chinked codings
    * any unknown codings (yes, this too! because we do not detect unusual syntax that is still chunked)
    * empty coding (plausibly harmless, but not see in real life anyway - refused, for the moment)
    pajod committed Dec 15, 2023
    2 Configuration menu
    Copy the full SHA
    ac29c9b View commit details
    Browse the repository at this point in the history
  12. Ignore secure_scheme_headers in Trailer section

    In common configuration unlikely a big security problem in itself
    you are just fooling the remote about https.
    However, it is offers an oracle for otherwise invisible proxy request headers,
    so it might help exploiting other vulnerabilities.
    pajod committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    fd67112 View commit details
    Browse the repository at this point in the history
  13. strict HTTP header field name validation

    Do the validation on the original, not the result from unicode case folding.
    
    Background:
    latin-1 0xDF is traditionally uppercased 0x53+0x53 which puts it back in ASCII
    pajod committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    f550111 View commit details
    Browse the repository at this point in the history
  14. strict HTTP version validation

    Note: This is unrelated to a reverse proxy potentially talking HTTP/3 to clients.
    This is about the HTTP protocol version spoken to Gunicorn, which is HTTP/1.0 or HTTP/1.1.
    
    Little legitimate need for processing HTTP 1 requests with ambiguous version numbers.
    Broadly refuse.
    
    Co-authored-by: Ben Kallus <benjamin.p.kallus.gr@dartmouth.edu>
    pajod and kenballus committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    7ebe442 View commit details
    Browse the repository at this point in the history
  15. briefly document security fixes in 2023 news

    further information to be published in security advisories, published out of tree on Github
    pajod committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    b6c7414 View commit details
    Browse the repository at this point in the history

Commits on Dec 17, 2023

  1. HTTP parser: stricter chunk-ext OBS handling

    chunk extensions are silently ignored before and after this change;
    its just the whitespace handling for the case without extensions that matters
    applying same strip(WS)->rstrip(BWS) replacement as already done in related cases
    
    half-way fix: could probably reject all BWS cases, rejecting only misplaced ones
    pajod committed Dec 17, 2023
    Configuration menu
    Copy the full SHA
    e710393 View commit details
    Browse the repository at this point in the history