Skip to content

RootWire 7.0.0

Latest

Choose a tag to compare

@EONRaider EONRaider released this 07 Nov 23:49
023d77f

RootWire adopts NETProtocols
2.9: the library now decodes some twenty more protocols, and RootWire
renders each of them on screen, adds --decode-as, pcapng replay and
conversation statistics, and hardens what reaches the terminal. It is a
major release for three breaking changes, each marked BREAKING
below: the NDJSON wire type field of ICMPv4/ICMPv6/IGMP/BGP/OSPF/PIM/
VRRP layers moves to type_; rootwire.pcap.read_pcap() is renamed
read_captures(); and DecodedFrame.payload no longer carries the
Ethernet padding or captured FCS past the frame's declared lengths,
which move to DecodedFrame.trailer. Existing CLI options and exit
codes are unchanged; --decode-as is new.

Security

  • LLDP's System Name, Port Description and System Description are now
    shown at most four times each, with any further duplicates counted
    as (+N more). Every duplicate used to get its own line, so one
    crafted LLDPDU of minimal TLVs (about 500 of them fit in a frame)
    could flood the terminal and bury real output.
  • Library diagnostics are sanitized before they reach the terminal.
    The [!] Malformed header: ... line and every [!] ... malformed
    line printed for a lazily parsed section now pass through the same
    control-character neutralization as payload text: NETProtocols'
    error messages can quote wire bytes, and an attacker-chosen byte
    string must never reach the analyst's terminal raw. The new [~]
    best-effort-miss line (below) is sanitized the same way.
  • The new DNS renderer (below) now neutralizes terminal control
    characters in question/answer names and RDATA, the same way -d's
    payload display already does — a DNS name is fully attacker-controlled
    (whatever a queried server answers), and rendering it unsanitized
    would let a crafted response inject ANSI escape sequences into the
    analyst's terminal. Caught by review before merge (#75).
  • DNS question names, answer names and RDATA are now also kept on one
    line: a label holding a newline used to split the record's line, so a
    crafted name could forge a RootWire [!] line under it (#94).
  • The IPv4 and TCP Options: lines now list at most 16 options and a
    (+N more) marker, so a header full of one-byte No-Operation options
    can no longer stretch the line without bound (#94).

Changed

  • NETProtocols floor raised to 2.9 (netprotocols>=2.9,<3; lock
    2.2.1 → 2.9.0).
    The library now decodes LLDP, CDP, STP, LLC/SNAP,
    MPLS, VXLAN, IP-in-IP, IPsec ESP/AH, OSPF, PIM, VRRP, SCTP, BGP, TLS,
    STUN, NTP, RADIUS, RIP, HSRP and TFTP on its default dispatch, so
    frames that used to end at Ethernet, IP or the transport now carry
    those layers, in full in --json and through a renderer of its own
    on screen (see Added). An LLDP frame (EtherType 0x88CC) is no longer
    "Ethernet plus payload".
  • Best-effort application decoders are no longer mistaken for
    malformed frames.
    2.3+ dispatches TLS (443), BGP (179), STUN (3478),
    NTP (123), RADIUS (1812/1813), HSRP (1985), TFTP (69), RIP (520) and
    VXLAN (4789) on the port alone, and calls the decoder on whatever the
    segment holds; every bare ACK and continuation segment on those ports
    then raised, which RootWire recorded as frame.error and counted as
    malformed. A stop inside one of those classes — derived from the
    library's registry at import, not hardcoded — is now recorded as the
    new DecodedFrame.l7_miss: no [!], not counted as malformed, the
    undecoded bytes left as payload, and a [~] Payload left undecoded
    note only when there are bytes to explain. DNS, DNS-over-TCP and DHCP
    are deliberately excluded: their malformations predate the upgrade and
    remain errors.
  • Layer cap raised from 16 to 32, the library's own default. BGP,
    STUN over TCP and TLS decode one layer per message or record, and a
    valid segment holds dozens; at 16, ordinary dense BGP/TLS traffic was
    diagnosed as malformed. An overflow where the next layer is another
    of the same port-dispatched class as the last is a best-effort miss;
    any other overflow (an extension-header chain, a VXLAN nesting bomb)
    is still an error, now reading "chain still going after 32 headers".
  • A lying length now stops the walk. The library bounds every layer
    by the declared IPv4 Total Length / UDP Length and rejects one
    shorter than its own header, so for an IPv4 total_length below
    ihl * 4 the IPv4 layer still renders but the layers inside it are
    no longer decoded (previously they were, off ihl), and
    frame.error is set alongside frame.malformed_length. The two are
    one fault: stats count it once and the screen prints the single
    specific line, now ending "upper layers not decoded". A UDP Length of
    1 to 7 is likewise an error at UDP. total_length == 0 (TSO) stays
    exempt. Restoring the inner layers for display is left for later.
  • BREAKING (library API): Ethernet padding and a captured FCS are split out as
    DecodedFrame.trailer
    (the library's packet.trailer) instead of
    being part of frame.payload, so -d no longer prints padding as
    payload. NDJSON reports its length as the new trailer_len. The
    split needs a declared length to cut at (IPv4 Total Length, UDP
    Length, an 802.3 Length), so behind an Ethernet II EtherType that
    nothing decodes, a captured FCS stays in payload.
  • NDJSON records gain l7_miss and trailer_len.
  • decoder.py now delegates its chain walk to netprotocols.decode_frame()
    instead of hand-rolling the same loop the library has shipped since
    2.0.
    No behavior change for well-formed frames. The layer-cap
    diagnostic's wording changed to match the library's own
    MaxDepthExceededError message (DecodedFrame.error now reads
    "chain still going after N headers (...)" instead of "decode chain
    exceeded N layers"); every other error message is unchanged, since
    each already names its own protocol in its text (#74).
  • Replay's non-Ethernet-linktype rejection (previously classic-pcap
    only) now also covers pcapng, checked against every declared
    Interface Description Block before any frame is decoded — an
    Enhanced Packet Block can reference any interface a preceding IDB
    declared, not just the first one, so checking only the first would
    miss a crafted file mixing an Ethernet IDB with a non-Ethernet one.
    Nothing else stops a wrong link type from silently decoding into
    nonsense, since Ethernet accepts any 14+ bytes structurally (#77).

Added

  • --decode-as TABLE=KEY:CLASS, repeatable: decode what a
    NETProtocols dispatch table maps KEY to as CLASS for this capture
    (tcp.port=8080:HTTP, udp.port=40000:TFTP, udp.port=5353:DNS),
    or as nothing with none (udp.port=1812:none). The overrides derive
    one registry for the whole capture. A class mapped to a port is
    best-effort like the library's own port decoders, so a bare ACK or a
    response body on a port told to be HTTP is a [~] miss, not a
    malformed frame; a class a type field also reaches (IPv4 on a port)
    is not. A bad value -- malformed, an unknown table or decoder, a key
    out of its field's range, a table and key given twice -- is a usage
    error (exit 2).
  • BREAKING (library API): -r/--read now replays pcapng captures, not just classic pcap.
    RootWire's own reader was classic-pcap-only (an explicit descope of
    the original pcap/replay work); the read path now delegates to
    NETProtocols' read_captures(), which auto-detects the format from
    its magic bytes. rootwire.pcap.read_pcap() is renamed
    read_captures() to match — a break only for code embedding
    RootWire as a library and importing that function by name, not for
    any CLI usage. -w/PcapWriter are unaffected: NETProtocols ships
    no writer, so -w still emits classic pcap only (#77).
  • --filter accepts sctp, IP protocol 132 in both stacks, exactly
    as tcpdump -dd sctp compiles it (pinned against tcpdump 4.99.4
    bytecode over the whole corpus). port still matches TCP and UDP
    only, as before.
  • Checksum verification rendering (closes the README's own
    long-standing Roadmap item). Every IPv4/ICMPv4/ICMPv6/TCP/UDP
    checksum is now verified against a recomputation via
    netprotocols.checksum.verify(), flagging a mismatch inline (e.g.
    Checksum: 0x0000 [!] mismatch (expected 0x1a2b)) instead of just
    displaying the on-wire value. Silent (no false positive) for a
    non-reassembled IP fragment — the wire checksum covers the
    reassembled datagram, which RootWire never rebuilds, so it can't be
    verified against a single fragment's bytes without lying (#78).
    IGMP and GRE checksums are verified the same way (GRE only when its
    Checksum-Present bit is set), and the renderers added below verify
    their own: SCTP's CRC32c, VRRP, PIM, OSPF and its LSAs, CDP, and the
    STUN FINGERPRINT.
  • Decoded IPv4 and TCP options, replacing the previous
    Options: N bytes byte count. Kinds the library understands
    (IPv4's Record Route, Timestamp, Router Alert; TCP's MSS, Window
    Scale, SACK, Timestamps) render their actual value; every other kind
    still shows its name with a raw byte count. Malformed options
    (a declared length running past the buffer) surface as a [!]
    diagnostic rather than raising (#79).
  • ICMPv6 Neighbor Discovery detail. A Router/Neighbor Solicitation
    or Advertisement (or Redirect) now shows its target address and
    every option TLV (source/target link-layer address rendered as a
    MAC, others as their raw byte count) instead of a bare Type/Code
    line — IPv6's ARP-equivalent was previously indistinguishable from
    any other ICMPv6 message on screen.
  • Screen renderers for DNS, DNS-over-TCP, and DHCP. These already
    decoded correctly (chain dispatch is registry-driven, not a RootWire
    whitelist) but printed as the generic "(no renderer)" placeholder
    on screen. DNS now shows the query/response direction, opcode/RCODE,
    and every question and answer (name, type, TTL, decoded RDATA); DHCP
    shows the message type, client MAC, the four address fields, and a
    parsed-option count. Both surface a [!] diagnostic instead of
    raising if their on-demand-parsed sections (DNS records, DHCP
    options) turn out malformed — these are parsed lazily on first
    access, unlike every previously-rendered protocol, which validates
    fully at decode time (#75).
  • Screen renderers for GRE and IGMP. Both already decoded correctly
    (chain dispatch is registry-driven, not a RootWire whitelist) but
    printed as the generic "(no renderer)" placeholder on screen. GRE
    now shows the encapsulated EtherType and which optional
    checksum/key/sequence fields are present; IGMP shows the message
    type, and either the multicast group (query/v1/v2 report/leave) or
    every group record (a v3 report), each with its record type and
    source-address list. IGMP's body-parsing accessors surface a [!]
    diagnostic instead of raising if malformed, matching the DNS/DHCP
    renderers' contract for the same reason: parsed on demand, not
    validated at decode time (#76).
  • Screen renderers for TLS, SCTP and NTP. TLS renders one line per
    record (content type, handshake message, record version, length) and,
    for a ClientHello, its version, SNI host name, cipher-suite count and
    extension list. SCTP shows its ports, verification tag, a verified
    CRC32c checksum and its chunk types with flags. NTP shows version,
    mode and stratum, leap/poll/precision/root delay and dispersion, the
    Reference ID or a stratum-0 kiss code, the transmit time, NTPv4
    extension fields and the MAC. Attacker-chosen text (the SNI, which the
    library only checks is ASCII — ESC is ASCII — and the Reference ID or
    kiss code) is sanitized for the terminal; on-demand sections (SCTP
    chunks, the NTP body) render a [!] line instead of raising when
    malformed, and a Handshake record that does not parse as the
    ClientHello its first byte announces renders an informational [~]
    line, since an encrypted TLS 1.2 Finished begins that way one time in
    256; and list lines cap at 16 items with "(+N
    more)", since one packet can carry hundreds of 4-byte SCTP chunks.
  • SNI of a ClientHello split across segments. A ClientHello too
    large for one segment (Chromium's) is a TLS best-effort miss, but its
    first segment usually holds the host name: the decoder now reads it
    with NETProtocols' peek_server_name() into the new
    DecodedFrame.sni_peek, rendered as [+] TLS ClientHello (continues in a later segment) SNI: ... and emitted as NDJSON sni_peek. Only a
    found name is kept: "not reached" (the name is in a later segment)
    is not "no SNI", and shows nothing.
  • Screen renderers for VRRP, HSRP, RIP, OSPF, PIM and BGP, the
    routing and first-hop-redundancy protocols an analyst triages a
    segment by. VRRP (v2/v3, IPv4/IPv6) shows the VRID, priority (0 a
    master resigning, 255 the address owner), virtual addresses,
    interval, a verified checksum and VRRPv2's authentication; HSRP the
    op code, group, state, priority, virtual IP and MAC, timers and the
    Authentication Data, flagged when it is the default cisco; RIP
    (v1/v2) a request for the whole table, or each route with its prefix,
    next hop and metric (16 shown as unreachable), and a RIPv2
    password or RFC 4822 keyed-digest entry; OSPFv2 the router and area,
    the authentication, the packet checksum (or that cryptographic
    authentication replaces it), and per message type the Hello's timers,
    DR/BDR and neighbors, a Database Description's MTU, flags and LSA
    headers, a request's LSAs, an update's LSAs with each LS checksum
    verified, and an acknowledgment's headers; PIM a verified checksum and
    the Hello options, a Register's inner flow (or Null-Register),
    Register-Stop, every Join/Prune group with its joined and pruned
    sources and their WC/RPT bits, and Assert metrics; BGP one section
    per message — an OPEN's version, AS, hold time, ID and parameters,
    an UPDATE's withdrawn routes, attributes and NLRI (or End-of-RIB), a
    NOTIFICATION's error and RFC 9003 shutdown communication. Clear-text
    passwords (VRRPv2 PASS, RIP, OSPF AuType 1), HSRP's Authentication
    Data and BGP's shutdown communication are sanitized for the terminal
    and kept on one line (a newline shows as \n, so a peer cannot forge
    a frame header or diagnostic); a raw BGP attribute value shows its
    first 32 bytes and how many more it holds;
    BGP and PIM bodies parse on demand and render a [!] line instead of
    raising when malformed; every list caps at 16 items. A PIM Register's
    inner datagram is read from its IP header alone, not walked, so the
    frame keeps a single layer budget.
  • Screen renderers for LLDP, CDP, STP, MPLS, LLC and SNAP, the
    layer-2 neighbor-discovery, bridging and label-switching headers.
    LLDP shows the Chassis ID (a MAC as a MAC), Port ID and TTL (0 a
    shutdown LLDPDU), flags an LLDPDU that does not open with them as
    802.1AB requires, and shows the System Name, Port Description and
    System Description, then every other TLV by name or, for an
    Organizationally Specific one, by OUI and subtype; CDP its version,
    TTL and verified checksum, the Device ID, Port ID, Platform and
    Software Version, the capabilities and addresses; STP (802.1D and
    RSTP) the root and bridge identifiers, root path cost, port, flags
    (an RST BPDU's port role and proposal/agreement handshake among
    them) and timers, or a TCN's type alone; MPLS each label of the
    stack with its traffic class and TTL, special-purpose labels by name
    and the bottom marked, and the payload the stack implies; LLC its
    DSAP and SSAP by name, group and command/response bits and control
    format, or Novell's raw 802.3; SNAP its OUI and protocol ID, named
    as an EtherType where the OUI makes it one. The neighbor-chosen text
    of LLDP and CDP is sanitized for the terminal, kept on one line and
    cut at 128 characters.
  • Screen renderers for VXLAN, ESP, AH, TFTP, RADIUS and STUN, the
    tunnels, IPsec headers and UDP services. VXLAN shows its VNI and
    flags, and notes reserved bits that are set; ESP its SPI, sequence
    number and the size of the protected rest (its Next Header is
    encrypted, so the walk stops there); AH, under IPv4 or IPv6, its SPI,
    sequence number and what it protects, tunnel mode named, and the
    ICV's size (never verified: that needs the SA's key). IP-in-IP needs
    no renderer of its own: the inner IPv4 or IPv6 header renders as one.
    TFTP shows a request's filename, mode and options, a DATA's or ACK's
    block, an ERROR's code, name and message, and an OACK's options;
    RADIUS its code, identifier, length and Authenticator, each text
    attribute (User-Name, Reply-Message, the station IDs,
    Acct-Session-Id) on its own line, and every other attribute by name
    with its address or number, a hidden password by size only; STUN its
    method and class, Transaction ID and verified FINGERPRINT (a
    misplaced one flagged), the mapped addresses, ERROR-CODE, SOFTWARE,
    UNKNOWN-ATTRIBUTES and whether MESSAGE-INTEGRITY is present. All
    sender-chosen text is sanitized for the terminal, kept on one line
    and cut at 128 characters, and a malformed RADIUS or STUN attribute
    list is reported as [!] ... malformed without hiding the header.
  • HTTP/1.x screen renderer, for HTTP reached through
    --decode-as (NETProtocols registers it on no port): the start line;
    Host, User-Agent, Referer, Content-Type, Server and Location, then
    every field by name (credentials by name only); the body's framing
    and size, never its bytes, and a chunked body's trailer count; the
    anomalies that make a message's end ambiguous -- Content-Length with
    Transfer-Encoding, a repeated or invalid Content-Length, a second
    Host, ... -- flagged [!] as request-smuggling signal, the rest
    noted [~]; and a count of the bytes after the message, since the
    walk does not chain pipelined messages. Every value of the head is
    sanitized for the terminal (read as latin-1, a lone 0x9B is a C1
    CSI), kept on one line and cut.
  • SCTP has its own bucket in the capture statistics.
  • Conversation statistics in the end-of-run summary: how many
    TCP/UDP/SCTP conversations the capture held and the ten largest by
    bytes, each with its frame and byte counts, both directions together
    (netprotocols.flow.flow_key). A tunnelled frame counts under the
    conversation inside the tunnel, VXLAN's underlay UDP never counts as
    one, and at most 65,536 conversations are tracked, the frames of any
    past that counted as untracked.
  • The corpus grows from 65 frames in 12 captures to 617 in 42,
    vendored from NETProtocols 2.9.0 (see tests/fixtures/MANIFEST.md),
    including the BGP, TLS and RFC 3489 STUN session captures upstream
    keeps outside its own corpus: their bare ACKs and split records are
    exactly what the best-effort rule above must not call malformed.
    Every frame is decoded, rendered and serialized, and no rendered
    checksum may read as a mismatch.
  • tests/fixtures/http/http.pcap, vendored byte-for-byte from
    NETProtocols 2.9.0: 98 frames of seven HTTP/1.1 connections, outside
    the corpus glob as upstream keeps it.

Fixed

  • The NDJSON encoder's last-resort str() fallback is no longer
    silent
    : it warns once per type on stderr, so a field type a future
    NETProtocols release adds is noticed instead of shipping as an opaque
    string.
  • --json crashed on LLDP and CDP frames ("Object of type bytes is
    not JSON serializable"): bytes were hex-encoded only at a layer's top
    level, but LLDP/CDP keep theirs inside TLV tuples, and LLDP is decoded
    by default. Encoding now goes through a json.dumps default= hook
    that handles bytes, enums, ipaddress objects and sets at any
    depth, and stringifies anything else rather than abort the capture.
  • BREAKING (NDJSON): NDJSON lost the layer name of ICMPv4, ICMPv6 and IGMP (and now
    BGP, OSPF, PIM, VRRP): spreading a layer's fields after its "type"
    discriminator let the layer's own type field overwrite it, so an
    echo request read "type": 8 instead of "type": "ICMPv4".
    "type" is now always the class name and the wire field is emitted
    as "type_". Underscored decode-context fields (BGP's _followed)
    are no longer emitted.
  • False checksum mismatches inside tunnels. TCP/UDP/ICMP checksums
    were verified against the outermost IP header, which is wrong for
    every tunnelled segment (VXLAN, GRE, IP-in-IP, AH tunnel mode):
    the corpus' inner UDP in VXLAN and TCP behind AH tunnel mode both
    read as mismatches. They now use the nearest enclosing IP header,
    as Packet.with_checksums() does.
  • False checksum mismatches behind an IPv6 Routing header. The
    pseudo-header now carries the route's final destination when a
    Routing header with segments left sits between the IP header and the
    transport (RFC 8200 §8.1), via verify(..., routing=).
  • False checksum mismatches on padded frames. A TCP/UDP/ICMPv6
    checksum input included Ethernet padding and any captured FCS (a
    54-byte bare ACK padded to 60 bytes summed 6 bytes too many); it now
    ends at that layer's own declared end, the tightest IPv4/IPv6/UDP/802.3
    length among the headers enclosing it. Per layer rather than the
    frame's trailer, so a VXLAN tunnel's outer UDP checksum still covers
    the inner frame's padding. ICMPv4 now gets its IP header too, so the
    library bounds its sum the same way.
  • Checksums were skipped, or wrongly checked, around tunnelled
    fragments.
    Fragmentation was judged frame-wide from the outermost
    IPv4 header and any IPv6 Fragment header anywhere, so an inner IPv4
    fragment got a false mismatch and an inner IPv6 fragment hid the
    outer UDP's. A checksum is now skipped only when a fragment header
    encloses its own layer.
  • Truncation was missed behind a VLAN tag. The truncation check put
    the IP header at a fixed offset of 14 bytes, so a capture short by up
    to a VLAN tag's (or QinQ pair's, MPLS stack's, LLC/SNAP header's)
    length read as complete. It now uses the outermost IP header's true
    offset.
  • A lying inner IPv4 length was not flagged as
    malformed_length (only the first IPv4 header was checked); every
    IPv4 header is checked now, and the screen line quotes the one that
    lied.