Repository navigation
PyPCAPKit v1.5.0
1.5.0 -- 2026-10-29
The largest release since 1.0, and the first recorded here as it happened rather than reconstructed. New extraction engines, ESP with payload decryption, SCTP and NGAP over SCTP, a library logger that no longer hijacks the consumer's, and a defect programme run through the issue tracker, citing issues and pull requests from #340 to #2257.
Preceded by 1.5.0a1 (2026-09-15), 1.5.0b1 to 1.5.0b3 (all 2026-09-19) and 1.5.0b4 to 1.5.0b14 (2026-09-20 to 2026-10-22), dated in UTC and all published as prereleases and so resolved only by pip install --pre. 1.5.0b1 half-shipped: the tag, the GitHub release and the Conda deployments landed, but PyPI rejected the wheel because twine check found a Sphinx-only :mod: role in README.rst, which pyproject.toml declares as the dynamic long description. 1.5.0b2 reshipped it -- an existing version cannot republish -- and 1.5.0b3 followed the CI change that stops a TestPyPI outage from costing a release its wheels (#497, #498).
pcapkit.const
Changed
- a breaking change to
pcapkit.const.reg.apptype.AppTypebecomes a memberless base over one registry per transport protocol --TCP,UDP,SCTPandDCCP-- becauseaenumrefuses to subclass an enumeration that has members.isinstance(TCP.http, AppType)still holds andAppType.get(port, proto=...)proxies to the registry, so nothing underpcapkit.protocolschanges. The rows IANA assigns no port, and those it assigns no transport protocol, stop being members; they become list-table docstrings on the registry that owns them.__members_proto__is now a per-registryMultiDictkeyed on port, so every service IANA registers on port 80 survives.get()answers with the canonical service, curated from/etc/servicesbecause IANA names no precedence, andget_all()reaches the aliases (#754). - the
%-formattedraise ValueError(...)calls in the generatedpcapkit.const.reg.apptype.apptype, left by #759, are f-strings (#792). The remainder followed: the_missing_guards inpcapkit.vendor.tcp.flags,.ftp.commandand.http.method,AppType's__new__/__repr__/__str__, and theextend_enumfallbacks inget()and in the span-handling tail. Theconsider-using-f-stringdisable drops fromtcp.flagsandreg.apptype.apptypeand stays onftp.commandandhttp.method, whose__repr__still used%. Edited in the vendor templates and applied to the generated const files by hand, since a crawl would fetch IANA's live registry and rewrite unrelated rows (#798). - the
__repr__methods #798 left on%(inpcapkit/const/ftp/command.pyandpcapkit/const/http/method.py) are f-strings, so the disable drops from both files and bothpcapkit/vendor/templates. The other bespoke templates ({const,vendor}/ftp/return_code.py,{const,vendor}/http/status_code.py,vendor/http/frame.py) andvendor/default.pystill carry it, so #804's claim that nothing inpcapkit/{const,vendor}/{ftp,http}/needs it holds for this pair only;repr()output is unchanged (#804). test_const_enum_no_mint.py'sis_manufactured()exemption for the hex-suffixed names #878 (below) preserves moves from a per-file to a per-name-shape one. The per-file exemption was shape-blind: rewriting a branch to'Xyplex_%d' % value-- exactly the value-derived shape the predicate exists to flag -- left the sweep green. The check now exempts only a'..._0x%s' % hex(value)[2:].upper().zfill(4)expression, and the exactexempt_hits == 53bound stays. Test-only. Closes #879 (#881).- a breaking change to
AppType.proto: it is the single transport of the registry the member lives in, not every transport IANA assigned the service, andregister_apptypetakes its transports as varargs (#806, #815). Aliases show inrepr()andstr(), and can be registered per transport (#807, #837). - a breaking change to
TransportProtocol: its base moves fromaenum.IntFlagtoaenum.IntEnum, so|between two members gives a plainintrather than a composite member (#808, #836). - a breaking change to
TLSKeyLabel: the hand-written class inpcapkit.protocols.misc.pcapngbecomes the generatedpcapkit.const.pcapng.tls_key_label.TLSKeyLabel(PCAPNG_TLSKeyLabelinpcapkit.const.pcapng), crawled from the IANA registry RFC 9850 creates (#886, #890), and the old path is removed (#1744, #1754). - a breaking change to
Method.get(): the method token is case-sensitive, per RFC 9110 Section 9.1, soMethod.get('get')no longer resolves toGET(#896, #907). It also looks a key up by value, soBASELINE-CONTROLandVERSION-CONTROLresolve (#908, #915). - a breaking change to
ExtensionHeader: it is generated from IANA's IPv6 Extension Header Types registry rather than the Protocol Numbers registry's flag, and the two disagree on 147 (BIT-EMU) (#925, #926). - the TCP and UDP
AppTyperegistries load on first use rather than at import (#1538, #1560). - a breaking change to
LLACode:Wilcardis spelledWildcardandMHisMN, the names RFC 5568 gives codes 0 and 2, with no alias for either old name (#1848, #1872). - a breaking change to
tcp.flags.Flags: each member is its wire mask in the flags field,FIN0x001up toAE0x100, rather than1 << offsetin IANA's most-significant-first bit numbering, which putFINat0x8000.int(tcp.connection)of a SYN/ACK is0x012, as the header carries it, not0x4800, andFlags(0x12)isSYN | ACK. The range guard narrows to the 12-bit flags field,0 <= value <= 0xFFF. Code that stored or compared the old integers has to change; membership tests and names are unaffected (#2310).
Fixed
- constant lookups that rejected a value the registry defines.
RouterAlert(0)is the only value RFC 2113 defines and the one IGMP, RSVP and MLD actually send, and it was discarded because the vendor crawler skipped a header row IANA's CSV does not have; IPXSocket(0)is that protocol's own default, sobytes(IPX(...))crashed on its own defaults; and the FTP_missing_overrides were plain methods rather than classmethods, so every unregistered value raisedTypeErrorinstead of extending the enumeration (#492, #503). - the string-keyed
get()inpcapkit.const.ftp.commandandpcapkit.const.http.methodtested membership with the raw key but registeredkey.upper(), so the first lowercase or mixed-case token raisedTypeError: 'RETR' already in use. Reachable from wire data:pcapkit.protocols.application.ftpmatches withre.Iand passes the match verbatim, and RFC 959 Section 5 makes FTP commands case-insensitive, soretr file.txtwas a valid request the library could not parse.get()and_missing_now look the key up under the canonical upper-case name, so every casing resolves to the existing member; a duplicateGETwould have lacked the real one'ssafeandidempotentattributes (#582, #583).http.methodno longer folds case: RFC 9110 Section 9.1 makes the method token case-sensitive, soMethod.get()matches it exactly (#896, above). get()'s documenteddefaultwas ignored on the integer path across the generatedpcapkit.consttree, because_missing_cannot see the caller'sdefault--Hardware.get(99999, 0)raisedValueErrorinstead of returning0. The integer path now consultsdefaultbefore letting the error escape.-1, the placeholder the generated signature already carried, still separates "no default" from "a default"; #859 replaced it with an identity sentinel once the shared base #858 (below) putget()behind one call site. The sweep covers the integer registries generated from the shared template, not only those #584 named;pcapngOptionTypeandregAppTypecarry a bespoke integer fallback and are deliberately left alone, since neither drops a default by raising. Not reachable from wire data, so a contract fix rather than a parse fix (#584)._missing_in the Mobility Header flag enumerations (BindingACKFlag,BindingUpdateFlag,HandoverACKFlag,HandoverInitiateFlag) ended inreturn cls(value), the constructor that had just failed, so every in-range value that is not already a member re-entered_missing_unbounded and raisedRecursionError--F(0), no flags set, and every composite such asBindingACKFlag(0x06). Defining_missing_at all caused it, by shadowing theaenumFlagmachinery that resolves exactly those values;pcapkit/const/tcp/flags.pyhad none and so never had the defect (it has one now, from #647 below, with the same tail). Each now ends inreturn super()._missing_(value), aspcapkit/vendor/default.pyemits and most const modules (#647's new ones included) already did, soF(0)is an empty flag andBindingACKFlag(0x06)isS|D. Minting throughextend_enumwas rejected for a flag type: naming0x06Unassigned_0x06would hide the composite and add a member per bit pattern. The range guard is untouched (#623).- constant registries rejected an invalid value differently from the built-in
enum, or did not reject it at all.Flags,ftp.command.CommandTypeandreg.apptype.TransportProtocolareIntFlagtypes with no_missing_, soaenumcomposed a pseudo-member for any integer:Flags(-1)returned65520(every TCP flag set, from a value no 16-bit field can hold) andFlags(-65536)none.ftp.command.Command,FEATCodeandhttp.method.MethodareStrEnumtypes whose_missing_calledvalue.upper()before checking the type, so an integer raisedAttributeError. These now raise a bareValueError, asenum.IntEnumdoes and the other modules already did. #647's proposal to replace those withpcapkit.utilities.exceptions.EnumErrorwas deliberately rejected:EnumErroris(BaseError, TypeError), not aValueError, so it would diverge from the built-in, walk past theexcept ValueErrorin every generatedget()body (silently undoing #584), and log CRITICAL fromBaseError.__init__once per discarded default. The one deliberate divergence is kept and pinned: mutable registries look up, miss, andextend_enumrather than raising, soMethod('FROBNICATE'),ProtectionAuthority(1 << 70)andAppType.get(65000, proto=tcp)still register; the #775 and #860 entries below record how each came to return an unregistered member instead. The flag guards end inreturn super()._missing_(value), so in-range composites still decompose and #623 stays fixed;Flagsaccepts0 <= value <= 0xFFFF. A newtests/const/test_const_enum_builtin_parity.pyasserts that every registry raises the same exception type as a stdlibenum.IntEnum; it keys onaenum.Enumbecause anIntFlagis not anIntEnumsubclass, which is how theIntFlagones escaped the existing sweeps (#647). - a breaking change to
AppType.get(): an out-of-range port is refused rather than minted. Itsexcept ValueErrorcaught the range rejection from_missing_and minted regardless, soAppType.get(-1, proto='tcp')returned an unreachablePORT_-1_tcpwhileTCP(-1)raised. It now tests_missing_forNone("no row for this port") and lets the rejection through; a valid but unassigned port still minted, until #874 (below) made it an unregistered member.Transport._make_port()passes an unvalidatedint, soTCP.make(srcport=99999)now raises where it minted junk; in 1.4.1,TCP(srcport=N)raisedAttributeErrorfor any bareint. Separately, the crawler rendered one_missing_branch per registry row, so the spans IANA registers on more than one transport emitted the same condition twice and every registry answered with the first:AppType.get(6010, proto='udp')gave a member whoseprotoreadtcp,6666/udpansweredircurather thanreserved, andproto='sctp'mintedx11where IANA assigns nothing. A branch naming a transport now testscls.__transport__; the branches naming none are unchanged (#764). - a breaking change to
AppType._dispatch(): it resolved aprotonaming several transports by its lowest set bit, andtcpis lowest, so every composite containing it dispatched into the TCP registry -- silently, and undetectably to a caller checking equality, sinceAppType.__eq__compares onportalone.TransportProtocolwas still anaenum.IntFlagand a member'sprotocarried the whole set IANA assigned the service, sotcp | udpwas an ordinary value to pass back in; theAppType.protoandTransportProtocolentries above record how #806 and #808 later retired both. Over every multi-transport member this misrouted port 888 (accessbuilderresolved to TCP'scddbp) and port 999 (puprouterresolved to TCP'sgarcon). It now raisesProtocolError, aValueErrorsubclass, so the documented contract holds. No library caller passes a composite (#759). get()'s string-key miss and every generated registry's_missing_bounded-range branch calledaenum.extend_enum()on an unrecognised value, so e.g.Hardware(40)minted a permanent member. Fixed at the shared sites inpcapkit/vendor/default.py, plus a newregister(value, name)base classmethod for the caller-named path that still mints. Every generated registry is updated; the non-mintingget()miss applies to all of them, but the_missing_fix changes behaviour only for those with a bounded-rangeextend_enumbranch.pcapkit/toolkit/scapy.pyandpcapkit/toolkit/pyshark.pyfedget()a string key with no default and now raiseMissingKeyErrorinstead of receiving a placeholder; the reader sites inhopopt.py/ipv6_opts.pyconvert theKeyErrorintoProtocolError. Riding along:pcapkit/toolkit/pyshark.pygains aFILTER_NAME_TO_LINKTYPEmap (ethtoETHERNET,trtoIEEE802_5), consulted ahead ofLinkType.getsince pyshark reports a filter name, not a DLT name. The other minting call sites in_missing_,getand the hand-writtenmh.pyandngap.pywere deferred; theregister/register_aliassites are not deferred but the by-design caller-named path (#838).- a breaking change to
pcapkit.const.ipx.socket.Socket's_missing_: archived range branches that were strict subsets of a range tested earlier --(0x0020, 0x003F)after(0x0001, 0x0BB8), and(0x4000, 0x4FFF)and(0x8000, 0xFFFF)after(0x0BB9, 0xFFFF)-- so they never fired. The generatorpcapkit/vendor/ipx/socket.pynow lists each subset range ahead of the range containing it. 36891 of the 65521 values that_missing_resolves (56.3%) change name: 32 fromRegistered by XeroxtoExperimental, 4095 fromDynamically AssignedtoDynamically Assigned Socket Numbersand 32764 fromDynamically AssignedtoStatically Assigned Socket Numbers. No defined member changes and every value still resolves (#847). LinkType(209).namereturned'IPMB_LINUX', which tcpdump's own table marks "Legacy names (do not use)", because a 2024-05-04 regeneration emitted that row above the currentI2C_LINUX = 209andaenumgives a value to the member defined first.pcapkit/vendor/reg/linktype.pysinks a row whose note mentions "legacy" after every current row, so the current name is defined first; in the registry crawled for this release that affects value 209 alone.IPMB_LINUXstays a reachable alias (LinkType['IPMB_LINUX']still resolves to 209). Not breaking: only the canonical name for the shared value moves (#848). The rule sinks only a row whose value another row also holds, counting values inside ranges, so a current row that merely mentions a legacy name stays first (#852, #854).- a breaking change to more const registries'
_missing_: applies the owner's mint/unmint ruling (settled on #847, confirmed on #775: a final, concrete assigned name mints, and a notation for readers does not) across the_missing_files still callingextend_enum. Most convert wholly, each minting a bare status word (Unassigned,Reserved [...],Reserved for Private/Experimental Use,Unspecified in the IANA registry,Deprecated), which the ruling treats as a notation rather than an assignment. Some are mixed:EtherType'sDEC Unassignedand "Old Xerox Experimental values. Invalid as an Ethertype since 1983." branches convert, while vendor-company names (Xyplex, Motorola, Walker Richer & Quinn and others) keep minting, since a proprietary protocol has no public name beyond the company's;IEEE802_3_Length_FieldandBerkeley_Trailer_encap_IPalso keep minting but are not company names.Socket(ipx)'s allocation-policy names (Experimental,Dynamically Assigned Socket Numbers,Statically Assigned Socket Numbers,Dynamically Assigned) convert, whileRegistered by Xeroxkeeps minting, a real ownership fact. Left alone areCGAType, whoseTag_<hex>mint is not an IANA-style range, and the files holding the classes #860 moves ontoEnumRegistry(theAppType,StatusCode,ftp.return_code,ftp.command,MethodandOptionTypemodules; #869 and #874, below). Each conversion changes both the vendor crawler and the generated file, per #838's and #858's precedent. The worked examples indocs/source/contributing/conventions/mint-criterion.rstkeepRegistered by Xeroxunder Mint (#861). - a breaking change to
EtherType's_missing_ordering: values0x0101-0x01FFnow resolve to their Old Xerox Experimental name instead of anIEEE802_3_Length_Field_0x0101-style one. The generated method tested0x0000 <= value <= 0x05DC(IEEE802.3 Length Field) first, and the Old Xerox range sits wholly inside it, so that branch was unreachable. Unlike #841's IPX fix, these bounds come from the live IANA CSV, so the fix is general:pcapkit/vendor/reg/ethertype.py'sprocess()collects range rows and a newEtherType._insert_rangeplaces each ahead of the first already-placed range that fully contains it, leaving non-overlapping pairs in CSV order; this pair is the only containment among the CSV's ranges. Closes #862 (#865). - a breaking change to
Method:__new__calledstr.__new__(cls)with no argument, so every registered member'sstrpayload was empty regardless of value --str(Method.GET) == ''andMethod.GET == 'GET'wasFalse;.valuewas always correct. #869 (above) had fixed the unregistered path only, leavingstr(Method('frob')) == 'frob'besidestr(Method.GET) == ''. Fixed tostr.__new__(cls, value), mirroringCommand.__new__, in the generator (pcapkit/vendor/http/method.py) and the generated module. No sibling hand-rolled__new__has the defect (OptionType/AppTypedeliberately store a display string). Consequences, all corrections:Method.GET == 'GET'isTrue;bool(Method.GET)flipsFalsetoTrue;json.dumps(Method.GET)emits"GET"not"";sorted()is lexicographic rather than input-order-stable; and{m: ... for m in Method}gains an entry per member where it had one, because every member previously hashed as''and compared equal. Nothing underpcapkit/protocolsrelied on the old behaviour. Closes #870 (#871). - a breaking change to
TransportProtocolandAppType: step 2 of #860, theAppTypehalf (the other bespoke registries were #869, above).AppTypeand its transport subclasses move ontoEnumRegistryand both mint sites --_missing_'s range branches andget()'s own mint (PORT_{port}_{transport}->'unknown') -- now build through a new_unregistered_memberoverride that reconstructssvc/port/proto, per the owner's ruling that only IANA-registered values are legitimate and thatget()lacks the information to create new ones. Every branch converts, including those naming a real service assigned to a whole port span (x11,active-net,satvid-datalnk,vrml-multi-use,ircu,swx,flex-lm); unlikeFEATCode's fix (#869, above) there is no import-time self-mutation to cure by declaring members statically, so these stay deliberately unregistered lookups.AppTypegains a workingregister(); the base's would have built a member withsvc='<null>'. The breaking part:TransportProtocolmoves from power-of-two values (tcp=1, udp=2, sctp=4, dccp=8) to sequentialauto()(sctp=3, dccp=4), per the owner's follow-up ruling once|-joined composites stopped being parsed.AppType.get()treats whateverintit is given as a whole, so the renumbering changes what specific integers mean: onAppType.get(80, proto=...), a bare4resolved toSCTP.httponmainand now silently resolves toDCCP.unknown, because a real member sits at4under both numberings; a bare8used to mintDCCP.PORT_80_dccpand now raisesValueError. The composed-value case is deliberately untested, per the owner's ruling that a breaking change already retired that path (#874). - closes the remaining scope of #775:
EtherTypeandSocket, the last registries whose_missing_minted a permanent member, now return a non-registeringcls._unregistered_member(value, name). The issue's AST census ofextend_enumin_missing_/getis down to the singleCGATypecarve-out (its mint is not an IANA-style range).EtherType(0x0888)used to add a member to__members__and return the identical object on a repeat; it now returns<EtherType.Xyplex_0x0888: 2184>with__members__unchanged, absent from_value2member_map_, equal but not identical across lookups -- likewiseSocket(0x0010). The regeneratedEtherTypewas not checked against the live IANA CSV, only against the committed fixture. Keeping each branch's hex-suffixed name reintroduces the value-derived shapetest_const_enum_no_mint.py'sis_manufactured()sweep forbids; rather than weaken the predicate, a narrow exemption covers exactly these files, since the collision the check guards against cannot occur once_unregistered_membernever registers (#878). OptionType.register()andCommand.register()built members thatget()could not find again, andOptionType.register_alias()raisednot a registered OptionTypefor every argument. Both registries now round-trip throughget(), in the generated files and the vendor templates (#1311, #1330, #1369).- the Quick-Start function registries rejected 9 to 15, though RFC 4782 gives the Function field four bits; the whole 4-bit range now resolves (#1350, #1353).
- every three-digit FTP reply and HTTP status code resolves, including those no registry row names (#1443, #1444).
- the
AppTyperegistry's dict union is defined only on interpreters whosedicthas one (#1694, #1703). - NGAP's
ProcedureCodeandProtocolIEare generated intopcapkit.const.ngapfrompycrate's compiled NGAP specification, with no network fetch. An unseen value gets a non-registering member, where every unknown key used to mint a member at the shared value-1and so aliased every other (#880, #899). FEATCode.get()matches case-insensitively, as RFC 5797 Section 2 defines FEAT codes; the registry-wide case-sensitivity audit found no other registry that needed to fold (#903, #913).registerrefuses whatgetrefuses.TCP.register(70000, 'x')andHardware.register(70000, 'x')minted a member thatgetthen returned, because no registry overrode the_validate_valuehookregistercalls. Each generated registry,AppTypeincluded, now overrides it with the check its_missing_makes, raisingEnumValueError.get(value, default)on a value the hook refuses still returnsdefaultand logs nothing (#1792, #1818, #1816, #1823; the quiet miss is #1845, #1873, underpcapkit.corekit).AppType.register()andOptionType.register()keep that rejection loud inside a quietget, asEnumRegistry.register()does (#1916, #1927).Command.get()refuses a key that is not astrwithEnumValueError, asMethod.get()does.get(b'\x01')minted a member keyed bybytes, andget(-1)raisedAttributeErrorwhatever itsdefault(#1824, #1826).- IANA's port spans that name no transport answer
AppType.get()with the asking registry's transport rather thanTransportProtocol.undefined, as itsunknownfallback for a port in no span already did:TCP.get(8012)isunassigned [tcp]. A port IANA lists on other transports only staysunknown: 8011 isreserved [tcp]andbier-echo [udp], andunknown [dccp]andunknown [sctp]. Lookups still mint nothing (#860) and an out-of-range port still raises (#2048, #2052). Method._missing_resolves an HTTP method whose registered name differs from its own value, soMethod('baseline-control')andMethod('version-control')giveBASELINE_CONTROLandVERSION_CONTROLrather than minting an unregistered member: the lookup falls back to the value map on the upper-cased value before it mints.Method.get()stays case-sensitive (#2284, #2351).
pcapkit.corekit
Added
tests/corekit/test_fields_numbers_width_repair.py, covering each octet boundary in its own method, since the defect is a pattern and a single case would pass against a fix that special-cased the reported width (#599).FieldBaseShortReadPaddingSideTests, covering both byte orders at 2, 4 and 8 octets and every truncation point, including the figures #604 reports, plus guard rails for a full read and for a read against an empty buffer, the #431 behaviour the option and list loops depend on (#604).EnumLookup, the base carryinggetandget_allthatEnumRegistryand the non-registry enumerations share (#877, #906, #921, #932). Itsgetraises pcapkit exceptions in the shape of the stdlibEnum: a name miss raisesEnumKeyError, aKeyError(#923, #927).MultiInfoandOrderedMultiInfo, immutableInfosubclasses that are also aMultiDictand anOrderedMultiDict, so a repeated key keeps every value. The data models' option lists use them (#1484, #1523).pcapkit.corekit.fields.byteorder, holdingpacket_byteorderandbyteorder_callback, the byte-order helpers the PCAP and PCAP-NG schemas share (#1528, #1711, #1715).can_seek(), whether a stream really seeks: on Windows a pipe answersseekable()withTruewhile itsseekdoes nothing. It trustsseekable()only for a regular file or a stream with no descriptor.Extractorwraps any other stream inSeekableReader, sopcapkit -reads a Windows pipe (#2097, #2104), andadvance()skips a record on one by reading (#2108, #2118,93a6173ae).
Changed
Probe,CipherSuiteandIntegritySuiteareInfosubclasses rather thantyping.NamedTuple. They are Mappings now, solen()and iteration yield field names rather than values.TCPSegmentinpcapkit.toolkit.pcapstays atyping.NamedTuple, since it is built for every TCP frame and anInfocosts more to construct.ModuleDescriptor.klassreads an already-imported module out ofsys.modulesrather than re-enteringimportlib.import_module, since next layer dispatch resolves a descriptor there on a per-frame path. Neither a registry hit holding aModuleDescriptornor a miss writes the resolved class back: a memoised hit went stale onimportlib.reload(#1559), and recording a miss in a class-levelcollections.defaultdictis the defect #421/#426 fixed at this layer, #425/#428 for the option, chunk and block registries, and #555/#560 at the schema layer. A resolution now costs ~117 ns rather than ~436 ns, which is not measurable inextract()wall clock. The change is taken for its shape:sys.modulesstays the only module cache, whereas a memo of the class would serve the pre-reload class after animportlib.reloadforever and failisinstanceagainst the live one. Proposed by@Ts-Boomin #563; the implementation differs because that one added a second, never-invalidated cache of resolved classes (#574).SeekableReader.truncate()raises instead of returning a size, and the misspelledwriteable()is nowwritable(). Both break external callers; nothing inside the package calls either.io.IOBasedocuments one gate over two methods -- "If False, write() and truncate() will raise OSError" -- and this reader'swritable()answers False, yettruncatereturned its new size, whereopen(path, 'rb').truncate()raises. The refusal isUnsupportedOperation('truncate'), the same onewriteraises; pcapkit's subclassesio.UnsupportedOperation. The old resizing is kept as the private_truncate_buffer: it resizes a private lookback window, not the stream, and it is the only route to the "position sits before the window" state thatseekand the buffered read paths must refuse, for which #643 and #644 landed tests.writeable()was never an override of anything, soio,shutiland third parties never saw it, and the value reported is unchanged (#645).- a breaking change to
@final, now enforced at runtime onInfoandSchema.info_finalandschema_finalboth endreturn final(cls), so every finalised class carriedtyping.final's__final__marker and nothing read it.@finalalone on anInfoorSchemasubclass -- without@info_finalor@schema_final-- now raisesInfoErrororSchemaErrorat first construction, the class being marked final but never finalised and so carrying no generated__init__.__init_subclass__cannot catch that, sincefinalapplies after class creation, so the check sits in the existing one-shotFinalisedState.NONEbranch and a finalised class pays nothing. Deriving from a finalised class raises too, from newInfo.__init_subclass__()andSchema.__init_subclass__()hooks (before, onlyEnumSchemahad one).SchemaErroris aValueError, so code catchingTypeErroraround a subclass declaration will not see it.@info_final @finaland@final @info_finalboth finalise silently; the re-entry check keys on__finalised__rather than__final__, because only the former records the decorator having run.@info_finaltwice warns and returns the class unchanged.pcapkit.utilities.compattakesfinalfromtyping_extensionsbelow 3.11 rather than 3.8, becausetyping.finalonly records__final__from 3.11. The guard cannot fire on the library's own classes (#778). - a breaking change to
Info.to_dict(): it returns anOrderedMultiDictat every level rather than adict, so a key that repeats, such as a second IPv6 extension header of one type, survives afrom_dict(to_dict())rebuild.type(d) is dictno longer holds (#1453, #1466, #1484, #1523). - a breaking change to
pcapkit.utilities.decorators: it moves topcapkit.corekit.decorators, with no re-export.seekset,beholderandprepareare no longer importable frompcapkit.utilities, and the module logs aspcapkit.corekit.decorators(#1735, #1737). - a breaking change to the sentinels: all live in
pcapkit.corekit.sentinels, which exports the objectsNULL,NO_VALUEandNO_DEFAULTand their common baseSentinel, but not their types, andNoValueis spelledNO_VALUE(#916, #911, #922, #937, #939). No moved name keeps a re-export, soNoValueTypehas only that home, not its 1.4.x one inpcapkit.corekit.fields.field(#1744, #1754).
Fixed
- stdlib exceptions leaking out where the library's own were promised: a malformed IP field value raised a bare
ValueErrorinstead ofFieldValueError(#465); abooladdress was silently packed as0.0.0.1or0.0.0.0,boolbeing anintsubclass (#491, #500); and@preparediscarded extra arguments silently and treated a declared zero length as end of stream, which is now distinguished from a genuinely exhausted one and raisesStreamEOFError(#454, #458). FieldBase.unpack()zero-padded up to a field's declaredlengthwithrjust()however little databufferheld; a ~40-octet PCAP-NG Decryption Secrets Block with a bogus inner length forced a multi-gigabyte allocation, sincelengthis often wire-derived. A declared length past 262144 octets -- libpcap'sMAXIMUM_SNAPLENand this package's defaultsnaplen-- that the buffer cannot back now raisesFieldValueError; the option and list loops' tolerance for a short read past a truncated area (#431) is far under that ceiling and untouched (#554).- that ceiling bounds one field, and a packet holds many, so the sum of a parse's zero padding was still unbounded: 200 minimal PCAP-NG Decryption Secrets Blocks (4,800 wire octets, each declaring
secrets_length262,142 against two supplied octets) retained 50.0 MiB, 10,922x per block, with the per-field guard never firing.FieldBase.unpack()now keeps a per-context ledger of octets supplied against octets synthesised, and raisesFieldValueErroronce the total of shortfalls past 65,536 octets passes262144 + 16 * supplied. The same 200 blocks retain 0.2 MiB (54.6x). A shortfall of 65,536 octets or fewer, the span of a 16-bit wire length, is padded unconditionally and charged to nothing, so no snapshot-truncated capture is subject to the budget; without that band the same legitimate frame parsed differently depending on what had been parsed before. What this does not close: a length declared by a 16-bit field can still be repeated without limit, and at this layer a crafted file is indistinguishable from a legitimate snapshot-truncated capture of offload-sized segments; separating them needs the frame's ownincl_len/orig_len. Nothing resets the ledger, so it bounds everything a context has parsed rather than one file (#573). - which exception a malformed TCP SACK option raised depended on unrelated process state: a clean interpreter raised
ProtocolErroras documented, but a process that had poppedpcapkit.corekit.fields.miscfromsys.modules(as the #439 ABC-cache regression tests do in everysetUp/tearDown) raisedFieldValueErrorfrom a different layer, before the documented check (#525).ListField.unpack()resolvedSchemaFieldthrough a function-local import re-run on every call; a module reimported mid-process is a second, distinct class, soisinstancemisclassified the field and billed each item by its declared length instead of what it consumed. Any caller relying on the previously-observedFieldValueErrornow getsProtocolError, deterministically, matching the docstring. - a
NumberFieldwhoselengthwas a callable could not pack or parse at any widthstructhas a native integer code for.lengthis-1until the callable resolves,-1has no native code, and the template builder raised_need_processfor it and never cleared it -- a latch. Resolving the real width rebuilt the template but left the latch set, sopre_processhandedbytesto a template that had become>Q(struct.error: required argument is not an integer), and parsing failed in mirror by callingint.from_byteson an integerstruct.unpackhad produced. The flag is now recomputed from the width in force, which also keeps a callable resolving to a width with no native code (3 octets) byte-packed. Every native width (1, 2, 4 and 8 octets) was affected, not only the 8 reported, onNumberFieldandEnumFieldalike. This made every extended 8-octet MPTCP DSS form unbuildable, those widths being chosen at runtime from the DSS flags; #585 worked around it in the TCP schema alone (#591). NumberField.pre_process()sized a value with floor division dressed as a ceiling. When a field is packed whilelengthis still the-1placeholder, the width came frommath.ceil(value.bit_length() // 8); the//had already floored, somath.ceildid nothing and the width was one octet short:256at one octet,65536at two,16777216at three. The reach is wider than "just past a boundary": every bit length that is not a multiple of eight is wrong, so1(bit length 1, floored to zero octets) and every value from 1 to 127 failed too. It is now the ceiling it was meant to be. The repair is reached only by packing a field the caller never resolved (a schema resolves every field first), which is why #591's suite, whose values all had byte-aligned bit lengths, missed it; #591's fix neither caused nor masked it. Deliberately left alone: a signed field is sized without room for its sign bit, so an unresolved signed field still cannot pack128. The same typo in the ILNP nonce option builders is a separate change (#599).FieldBase.unpack()padded a short read on the wrong side. When a buffer falls short of the declared length -- the accommodation that lets a snapshot-truncated capture parse (#431) -- the shortfall was zero-filled withrjust(), placing the zeros at the front, as if the octets never read were the leading ones; a short read has lost the trailing ones. It is nowljust(), correct for both byte orders: one octet of a four-octet little-endian120read as2013265920, and three octets of a four-octet big-endian0x01020304as0x10203. A full read is untouched (#604).SeekableReader.truncate()put its padding where the reader's bookkeeping says the content is, andreadthen returned one octet fewer than the buffer held. Same family as #604, but separate defects. The buffer keeps content at[0:_buffer_cur]with padding behind it, so a reduction must keep the octets it has and an extension must append at the tail;temp[-size:]sliced the buffer rather than the content, keeping trailing padding and discarding octets read, andtemp.rjust(size)prefixed the zeros. Separately,readcapped atmin(size, self._buffer_cur - 1), one short at the start and reaching into padding further in, and made up the shortfall from the stream past the skipped octet. The reported symptom needed both:read(4),truncate(8),seek(0),read(8)overb'abcde'returnedb'\x00\x00\x00e'and now returnsb'abcde'. Further contract faults are fixed with it: the position was reset to the buffer start rather than left alone; an omittedsizeresized to0rather than the current position; and_buffer_curwas left addressing octets a reduced buffer no longer had, so the next read raisedValueError: memoryview assignment: lvalue and rvalue have different structuresfrom_write_buffer.truncate(0)raised the same by a second route (buf[-self._buffer_size:]isbuf[-0:], the whole buffer), so_write_buffernow counts from the front; that state is reachable only throughtruncate. A reduction also advances_buffer_setpast the dropped octets, keeping_buffer_set + _buffer_curequal to how far the stream has been consumed, whichseekreads as its licence to fetch more; clamping_buffer_curalone made that sum under-report, and the next forwardseeksilently spliced in octets from the wrong offset (read(8),truncate(3),seek(6),read(1)overb'abcdefghijklmnop'returnedb'l'whereb'g'sits at offset 6). Latent rather than live: nothing here callstruncate, though it is public (#622).- the 16-bit band the #573 entry above records as deliberately left open: a length declared by a 16-bit wire field could still be repeated without limit. The crafted PCAP-NG of 2,000 Enhanced Packet Blocks, each with an option declaring 65,535 octets against four real ones, synthesised 1,637x its size in zeros and now synthesises none, with every frame and option still parsed. The bound is taken one layer up, where the missing information exists: a PCAP-NG block's Block Total Length is authoritative and cross-checked against its trailing copy, so the option area is that length less the fixed fields,
captured_lenand its padding, and an option declaring more payload than the area has left is malformed. A snapshot-truncated capture says so throughcaptured_lenand leaves its options whole, so it never trips this -- which the #571len(buffer) < lengthrejection could not distinguish, and why it was declined. A newbounded_optionclamps the payload to the octets the area has left and warns withSchemaWarning; it is applied to every variable-width option and record payload in the PCAP-NG schema, and deliberately not to the block-level payload fields, the 32-bit band the running ledger already budgets. A companionbounded_areaclamps a packet block's option area to the octets the block itself holds less the trailing Block Total Length: the area was otherwise sized from a declared length nothing checks against the file, so a block declaring 1,000,000 octets while holding 36 synthesised 65,535 zeros with no warning. Both are no-ops on a well-formed block. The non-packet blocks' option areas keep the framing assumption; the general fix for a declared length reaching a read is #678. Clamping rather than refusing keeps the #431 accommodation, since a PCAP-NG block read has no catch point aboveFieldBase.unpack()and one refusal would abort a whole extraction rather than one block; and the clamp reads only the block's own declared framing, never a running total, so identical input answers identically whatever preceded it. The skip for a remainder already past zero is load-bearing on the unpacking path, where clamping would give the field a negative template such as'-2s'. No clamp fires on the sample captures, and #678's uncaughtValueErroron truncated input is unchanged (#594). - a breaking change to
FieldBase.length: it calledstruct.calcsizeon a template built from a negative resolved length (e.g.'-5s', from alength=callback whose running counter had been overdrawn), raising a barestruct.errorthat ordinary caller code does not catch. The chokepoint every affected schema shares now raisesProtocolError, coveringapplication/{ftp,httpv1,httpv2,ngap}.py,internet/{hip,ipv6_route,mh}.py,link/ethernet.py,misc/pcapng.pyandtransport/sctp.pywith no other file changed; a non-negative length is unaffected. This closes the follow-on thehttpv2length-guard entry underpcapkit.protocolsfiled as out of scope (#805). - a breaking change to
EnumRegistry.register(): it now raisesValueErrornaming the existing member and pointing atregister_alias(), where it used to silently alias an already-registered value under the caller's new name --aenumtreats a taken value as an alias request, soregister(existing_value, 'TOTALLY_NEW_NAME')returned the existing member, minted nothing and raised nothing, contradicting its docstring.pcapkit.corekit.enum.EnumRegistryis a new mixin implementingget,get_all,register,register_alias,register_aliasesand_unregistered_memberonce, per the owner's ruling on #842; it first converts registries with no bespoke__new__(ipv6.extension_header.ExtensionHeader,tcp.flags.Flagsand themh.*_flagmodules), and servesIntEnum,IntFlagandStrEnumalike. The guard reaches only those: the other const registries still carry the old generator's bareextend_enum(cls, name, value), sopcapkit.constships contradictoryregistercontracts until #858 (below). These also gain the generated template'sKeyErrorcatching on a missing name, soFlags.get('NOPE', 2048)returnsFlags.ACKrather than raising; and a key that is neitherintnorstr, which raisedTypeErroron theIntFlagregistries (Flags.get(3.5)), now raisesValueError, tried as a value. OnExtensionHeaderit raisedKeyError, the shift #858's entry attributes to the other registries (#855). - a breaking change to
EnumRegistry.register(), extended: the const registries still on the generated template's bareextend_enumnow inherit the guard #855 (above) added, soTransType.register(6, 'TOTALLY_NEW_NAME')raisesValueErrornaming the value and the member that holds it and pointing atregister_alias(), where it used to mint and raise nothing.pcapkit/vendor/default.py's template now emitsclass {NAME}(EnumRegistry, IntEnum)and drops its ownget/register/_unregistered_member. The modules with a bespoke__new__(ftp/command,ftp/return_code,http/method,http/status_code,pcapng/option_typeandreg/apptypeamong them) are left alone, so every other class inheritsEnumRegistry(#855).get()'s dispatch shifts only for a key that is neitherintnorstr(KeyError, tried as a name; nowValueError, tried as a value), a key type nothing in the library passes.pcapkit/corekit/enums.pyis renamedenum.py, the owner's ruling (#858). EnumRegistry.get()'sstr-key branch tried only a name lookup (cls._member_map_[key]) and on a miss raised or returnedcls(default), never the value path, so on aStrEnumregistry a key that resolved through the constructor (_Str('known-value')) raisedKeyErrorthroughget(), and a supplieddefaultoverrode a good value match the code never attempted. It now falls back to a plain_value2member_map_lookup once the name lookup misses and before consultingdefault. Deliberately notcls(key):FEATCode._missing_()(pcapkit/const/ftp/command.py) mints viaextend_enumfor any unrecognised string, so routing through the constructor would let a failed name lookup mint a permanent member once a registry like it converts; a dict lookup never reaches_missing_. A name still wins when a string is both a name and a different member's value. Step 1 of #860; step 2, converting the bespoke registries, follows (#869, #874, below). The non-strpath is unchanged. Widens rather than breaks: a call that raisedKeyErrornow resolves, and no caller that got a correct result gets a different one (#863).- a breaking change to
EnumRegistry.get(): its docstring claimed "It never mints", but eachcls(default)call site reached_missing_for adefaultinside a still-minting registry's range, growing the registry as a side effect of resolvingdefaultrather thankey. Per the owner's ruling that onlyregister()mints, andget()only through the ranges_missing_mints, each site now resolvesdefaultthrough a plain_value2member_map_lookup;keyresolution is unchanged, soEtherType.get(0x0888)still mintsXyplex_0x0888through its own_missing_, as #775's deliberate exception permits. Adefaultnaming no registered member now falls through to the errorkeyitself would have raised, rather than a fresh error about the default -- the accepted cost of the ruling. The docstring's "It never mints" is qualified todefaultonly. SoHardware.get('Definitely-Not-A-Member', 40), with40inHardware's declared-but-unassigned range, now raisesKeyErrornaming the original key. Closes #864 (#868). ProtoChain.__add__()returned the left operand's cachedprotocolsandaliases: oncea.protocolshad been read,a + ProtoChain(IPv4)reportedchain == 'Ethernet:IPv4'butprotocols == (Ethernet,).copy.copy(self)carried the cache across; the result is now built fresh, asfrom_listbuilds one, and+=follows, since it falls back to__add__(#1099).- a
SchemaFieldwith no declared length handed its-1placeholder to the nested schema as that schema's length, so every item of such a list warnedpacket length < 0: a HIPLOCATOR_SETand an MH CGA Parameters item among them.SchemaField.unpack()now passes what is left in the buffer as a declared length -- notNone, which would turn a truncated item'sFieldValueErrorintoprepare()'s quietStreamEOFError. Fixed-length items are unchanged; the HTTP/2SETTINGSpair, whose size is fixed, is given it in its own schema instead (#1140, below) (#1148). - a signed
NumberFielddecoded as unsigned, a zero-lengthPayloadFieldmisread, and an emptyRawdid not rebuild (#1331). from_dict(to_dict())lost nested values, andlen()of anInfocounted its hidden keys (#1334).StringFieldpacks back exactly the octets it unpacked, where a guessed charset, aU+FFFDsubstitution or a character-count length used to change them; the decoded text is unchanged (#1293, #1336). A charset that rejects'replace'falls back to UTF-8 (#1706, #1728), and Python 3.8 unquotes throughpcapkit.utilities.compat.unquote(#1693, #1707).IPv4InterfaceFieldkeeps a mask that is not a netmask, and a deep copy of anOrderedMultiDictkeeps its order (#1346).- a breaking change to padding: non-zero padding octets are kept on rebuild rather than zeroed (#1223, #1436).
- fields keep the bits and octets they captured on rebuild: a
BitField's unnamed bits (#1487, #1495), aNumberField's bits outsidebit_length(#1488, #1497), and the bytes an untypedListFieldunpacked (#1489, #1494). An unnamedListField,PayloadFieldorSchemaFieldgets a default name (#1491, #1493). - each non-final
Infosubclass is finalised on its own state (#1490, #1496), and its constructor reads only the class's own annotations (#1692, #1702). - copying or pickling a stream proxy no longer recurses (#1524, #1544).
@preparereadslengthandpacketby keyword as well as by position, soSchema.unpack(data)andSchema.unpack(data, length=2)work where they raisedIndexError(#444, #450).ListFieldunpacks a schema item from the per-item copy its callback andlength_callbackconfigured, not from the unconfigured original (#433, #453).- an
*IntFieldgiven asigned=that contradicts its fixed sign, such asUInt8Field(signed=True), raisesFieldValueError; the argument used to be discarded silently (#545, #549). - negative lengths raise
ProtocolErrorrather than a bare stdlib error:FieldBase.lengthtells a malformed template from a negative resolved length (#825, #827), andNumberFieldchecks a negative resolvedlengthand a negativebit_length(#828, #829, #831, #834). ModuleDescriptor.klassraisesProtocolErrorfor a class name that resolves to nothing, or one never given, rather than a bareAttributeError; the registry modules share oneNULLsentinel compared by identity, where each had its own string (#832, #833, #835).PortEnumFieldgiven aprotowith no.name-- a string such as'tcp', or a composite such asTransportProtocol.tcp | TransportProtocol.udp, which is a bareint-- raisedAttributeErroron a port the registry does not hold. The miss takes its transport from the registry's__transport__, asAppType.get()does (#1774, #1781).EnumLookup.get()answers a value miss quietly, whetherdefaultanswers it or the miss is raised: a registry's_validate_valuerejection logs nothing and installs no exception hook there, whileEnumRegistry.register()still reports the same rejection loudly (#1845, #1873).- each sentinel is a singleton, as
Noneis: calling its type,copy,deepcopyandpickleat every protocol give back the one object, andNO_VALUEreprs as its own name rather than as an object address. Their new common base,Sentinel, is public; a sentinel's type cannot be subclassed, and neither it nor the sentinel takes an attribute write. A pickle written before this change at protocol 0 or 1 still loads, though not as the singleton (#1841, #1842, #1876). - defects off CPython on Linux. On GraalPy,
SeekableReader's reads aliased a buffer a later slide rewrote, so they copy out of it, and aMultiDictnested each value list, so it is built from adict(#1963, #1974); and a sentinel type refused the cache writes_py_abcmakes, soisinstanceagainst it raisedTypeError(#1960, #1966).Field.packhandsstructabytearrayasbytesfor thes,pandcformats, which PyPy refuses forsandpand GraalPy accepts forcwhere CPython does not, so it packs alike on every interpreter (#2004, #2007, #2015, #2018). On Windows,SeekableReader(buffer_save=True)with nobuffer_pathcould not reopen its temporary file; it creates one it can, and deletes it onclose()or collection (#2098, #2105). PortEnumFieldandSwitchFieldare exported frompcapkit.corekit,pcapkit.corekit.fieldsandpcapkit.all, andSchemaField,nested_packet_contextandBytesFieldfrom the modules defining them, soimport *binds them (#2174, #2191).- a
ListFieldcallback is handed the per-item copy it configures, not the shared field, and aSchemaFieldwith a callable length sizes its template from the resolved length rather than1024s. No parse in the library reached either (#2173, #2190). PortEnumFieldrange-checks the port it is handed, so a directly built TCP, UDP or SCTP schema raisesFieldValueErrorfor a port outside the field's width, wheresrcport=70000was masked to0x1170and-1to0xffff.NumberFieldkeeps masking, which is its documented contract; theNumberFieldandSchemadocstrings now say that direct construction range-checks nothing, and name where the checks live (#2314, #2335).
pcapkit.dumpkit
Fixed
- a flag value with no declared bits no longer dumps as
Type::None [0]. This changes user-visible output in every textual dump format.make_dumper'sobject_hookrenders each enumeration member asType::name [value]and interpolated the name unguarded, but aFlagvalue composed entirely of undeclared bits hasname is None, so the literalNonelanded in the name half:Flags(0)rendered asFlags::None [0]injson,tree,text,txt,plistandxml, out of bothExtractorandTraceFlow. A nameless member now renders with the value's decimal spelling, soFlags(0)givesFlags::0 [0]andFlags(8)givesFlags::8 [8]. That is the spelling the enumeration libraries use for an undeclared residue (Flags(2057).nameisACK|9), and it cannot be mistaken for a member name, since an identifier may not begin with a digit, whereasNONEis a real declared name elsewhere. The guard is onname is Noneand not on the value, because the defect was never about zero (Flags(1)andFlags(9)are equally nameless). Surfaced by #634, which made a flagless TCP segment reach this path (#648). tests/dumpkit/test_nameless_enum_rendering_unit.py(added by #670) carried probes assuming every flag-enum registry is 16 bits wide:test_scalar_return_renders_a_nameless_member_as_its_valueused one fixed tuple ending in65536, andtest_no_flag_registry_renders_the_literal_noneswept every registry but only againstregistry(0). Widths differ:ftp.command.CommandType3 bits,reg.apptype.TransportProtocol4 (computed at runtime), themhack/initiate flags 8, andmh.binding_update_flag.BindingUpdateFlagandtcp.flags.Flags16.tcp.flags.Flags(65536)correctly raises (0 <= value <= 0xFFFF), so the sweep was failing on correct behaviour. The probes now derive each registry's bound from its declared members, and a new test pins that one past the widest in-field value is refused. Test-only (#702).- the plist writer emitted mapping keys unescaped, so a key carrying
&,<or>produced a report that is not well-formed XML.dictdumper's_append_dictinterpolates the key into'<key>{item}</key>'and calls_encode_valueon the value only, so both branches ofpcapkit.dumpkit.commonthat build a mapping now escape their keys through oneescape_keyhelper -- aMultiDict, where #575 escaped an enum-derived key inline and nothing else, and a plain dict, where nothing was escaped. The json, tree and text writers are untouched. A non-strkey is escaped too: a PCAP-NG decryption secrets block keys TLS key log entries by a raw bytes client random whose repr can carry any of those characters. The upstream half isJarryShaw/DictDumper#125; this does not wait on it. The json writer's quoting defect stands, upstream asJarryShaw/DictDumper#121(#772). - the JSON writer emitted mapping keys unescaped, so the report of
test.pcapngdid not parse (Expecting ':' delimiter): its TLS key log is keyed by abytesclient random whose repr holds"and\.escape_keyinpcapkit.dumpkit.commonnow JSON-escapes keys for JSON output, as #772 made it XML-escape them for plist, and the JSON report of everyexamples/captures/file now parses. The upstream defect,JarryShaw/DictDumper#121, stands (#1152). The PCAP-NG end-to-end test of that report no longer skips; test-only (#1795, #1809). - a breaking change to the dump schema: the output writers dropped or corrupted what the packets carried. An
OrderedMultiDict, which every option list is, now dumps as a list of single-key entries in wire order (#1305). Inside an object the writers fall back on, a class, module or function is written as its qualified name rather than expanded, and an object met again on its own expansion path as<circular reference: T>(#1372, #1381). - tree output raised on a
complexvalue, and is now documented as lossy (#1266, #1366). - plist dates are written to whole seconds, so
plistlibreads them back (#1448, #1450). - a PCAP trace record scales its fraction to the trace's resolution (#1500, #1529), and writes the octets it has rather than a truncated record's claimed length (#1541, #1543).
- a subclass of a dumper
make_dumperbuilds recursed without end in itsobject_hookfallback, which calledsuper(type(self), self)(#2144, #2146).
pcapkit.foundation
Added
- the
engine='pypcap'andengine='pcap_ct'extraction engines, independent distributions of the samelibpcapinterface, andengine='pypcapfile'(#386, #405). They buy speed by doing less -- neitherpypcapnorpcap_ctdissects at all, so they offer no reassembly or flow tracing, andpypcapfilehas no IPv6 decoder. Install only one ofpypcapandpcap-ct: both own the top-levelpcapmodule, and with both presentpcap-ctwins the import and the other becomes unselectable. The missing interface constantsPyPCAP,PCAP_CTandPyPCAPFileare now exported alongsideDPKT,Scapy,PySharkandPCAPKit(#412). Python 3.11 is the last interpreter on which every built-in engine can run, and even therepypcapandpcap-ctcannot coexist. EngineBase.unsupported_reason(), a preflight every engine answers andExtractor.run()consults before anything is imported. An engine that cannot run in the current environment now gives one warning naming the real cause (a Python version, a missingtsharkorlibpcap, the wrongpcapdistribution) and a clean fall back topcapkit's own parser, rather than an error from inside the third-party package (#396, #405). The same holds for an engine whose package is not installed at all (#1045).conflicton the reassembly data models: absolute, inclusive ranges where two fragments claimed the same span with different bytes, previously lost silently on both the IP (#482) and TCP (#443, #478) paths.- the declarations that pair a
__proto__dispatch registry with the enumeration its codes come from, replacing the module-level tables that held the pairing apart from the__proto__they describe.__proto_key__, on each class owning such a registry (Link,Internet,Frame,PCAPNG,TCP,UDPandSCTP), names that enumeration and whether a bareintis also valid (#2392);__proto_dest__, on theLinkType,EtherType,TransTypeandPayloadProtocolIdentifierregistries, names the destination each enumeration keys, which is howcode=<member>infers one without an explicit destination. Both are rendered attributes, and the vendor crawlers generate the latter from a newPROTO_DEST(#2403).
Changed
layer=andprotocol=are honoured rather than inert. Both were read under the wrong names, so every value a caller passed was dropped into**kwargs; the CLI's-Lalso now validates its argument. The packet context reaches the schema layer for the first time, so a field the wire elides can be resolved from its enclosing packet (#404).follow_tcp_streamdispatches on the engine type, where both branches of the old test were dead and the native adapter ran against every engine's frames (#402).- reassembly and flow-tracing defaults moved (#435), visible to a caller.
Datagram.completedwidened fromboolto aCompletionenumeration (COMPLETE,PARTIAL,TIMEOUT); onlyCOMPLETEis truthy, soif datagram.completed:is unaffected butdatagram.completed == Trueno longer holds. TCP flow tracing is bidirectional by default, which merges each flow's two halves and closes one only once both have FINed -- 331 flows become 111 on the sample HTTP capture, the difference being single-frame stray tails; passtrace_bidirectional=Falsefor the old behaviour. IP reassembly also gained the 60-second timeout [RFC 1122, RFC 8200], clocked off the capture's own timestamps, tunable withreasm_timeout=; TCP reassembly gets no timeout by default.trace_analyse=is new, and reassembles each traced flow's application layer. - conflicting TCP overlaps resolve first-write-wins, per RFC 9293 section 3.10, where they had silently resolved last-write-wins (#443, #478). A deliberate, narrow behaviour break: a conforming retransmission carries identical bytes. IP fragment reassembly keeps last-write-wins, because RFC 791 specifies the opposite resolution, and records the disagreement instead (#482).
- renames with no compatibility alias left behind:
HoleDiscriptoris spelledHoleDescriptorand its package aliasTCP_HoleDiscriptorisTCP_HoleDescriptor(#350). - subclass registration is opt-in for
Engine,Reassembly,TraceFlowanddumpkit'sDumper(#514). Each registers if and only if its registry keyword is given --engine=forEngine,protocol=forReassemblyandTraceFlow,fmt=forDumper. Previously an absent keyword fell back to the class' own name, so every subclass of the public class was registered, and declining meant subclassing the parallel*Baseclass under an alias, which every built-in does (hence no public class had a subclass, while every built-in derives from its*Baseclass). This breaks out-of-tree code that subclasses one of these classes and relies on the derived key; pass the keyword, or call the matchingregister_*function. Nothing the library ships is affected, and the*Baseclasses remain importable. Silent failures are now loud: an unrecognised class keyword raisesUnsupportedCallinstead of being swallowed by**kwargs(passingname=to aReassemblysubclass silently ignored the key, sinceprotocol=is the real one), and so doesDumper'sext=withoutfmt=. A class attribute is not an opt-in:__engine_name__and__protocol_name__still set the name a class reports, registered or not. Each metaclass gained a class-levelregistryproperty mirroringEnumSchema.registry, and aDumpersubclass no longer touches the filesystem while itsclassstatement runs.Engine's keyword isengine=rather than thename=first shipped, becausemcls,name,basesandnamespacecollide withabc.ABCMeta.__new__'s parameters on Python 3.10 and raiseTypeErrorbefore the hook runs. There is noname=alias: a keyword that works on some interpreters and not others is the trap being removed. - a breaking change to
Extractor.register_engine(),register_reassemblyandregister_traceflow, and theregister_extractor_*wrappers over them: each now accepts only a subclass of the publicEngine,ReassemblyorTraceFlow(#1016). They had accepted a subclass of the matching*Baseclass since #513, which was a tolerance for pcapkit's own built-ins (every built-in derives from the base, none from the public class) rather than a contract:*Baseis documented as internal, and the public class is the one carrying the registration hook. Third-party code that subclassesEngineBase,ReassemblyBaseorTraceFlowBasedirectly and registers through these functions now getsRegistryErrornaming the public class; subclass the public class instead. The error messages are unchanged. The built-ins never pass through a registrar;Extractor._register_internal_engineand its sibling internal registrars check the base for internal registration, and are what the #513 regression test now exercises.register_protocolis untouched (#1020). - extraction is around 46% faster on a 1,117-frame HTTP capture, with byte-identical output (#420). A reassembled datagram's payload is analysed on first read rather than eagerly, cutting IP reassembly's own cost by 90.7% and TCP's by 23.7% (IP reassembly submits a datagram for every frame, fragmented or not) (#424). Flow tracing over the same capture went from 1416.6 ms to 744.0 ms, because the flow dumper handed each record to a
Frameconstructor that re-dissected the whole stack to return the bytes it had just been given; options are no longer parsed twice either (#427). - a breaking change to
Extractor.engineandExtractor.record_header(): both are now declared to returnEngineBase[_P]rather thanEngine, and the private_exengattribute follows. The built-in engines subclassEngineBasedirectly, so none was ever anEngine, which adds only the__init_subclass__registration hook. A type checker sees the change: code passing the result to anEngine-typed parameter must now acceptEngineBase; and because the old declarations were unparameterised,engine.read_frame()andrecord_header().read_frame()used to beAnyand are now the extractor's own frame type, so an assignment that relied onAnythere no longer type-checks (#1022). - the documentation of the engine base classes: the member stubs (
name,module,registry,extractor,unsupported_reason,run,read_frame,closeand the rest) moved fromEnginetoEngineBase, where they are defined, andEnginenow documents only the__init_subclass__registration hook it adds, so theEngineBase[_P]thatExtractor.enginereturns leads to an API. Cross-references toEngine.run,Engine.read_frame,Engine.closeandEngine.unsupported_reasonnow point atEngineBase, which stays out of__all__, as #1016 settled (#1024). - a breaking change to
register_mh_extension:schemais keyword-only, as in the other schema-taking registrars (#516, #526). - the
pypcapandpcap_ctengines probe thepcapbackend once per extraction rather than three times, and find its distribution without parsing each candidate'sMETADATA, whichimportlib.metadatadoes on Python 3.15 and which madepcap_ctextraction about 2.4x slower there (#2110, #2112, #2113, #2121).
Fixed
- TCP reassembly mixed absolute sequence numbers with buffer-relative slicing, so on any capture carrying a SYN with a realistic initial sequence number incomplete datagrams were dropped silently, and the
completed=Falsebranch of the public API was unreachable (#349, #376). format='text'raisedAttributeErrorbefore writing anything, naming adictdumper.Textthat has never existed. It now points atTree, as the'txt'alias beside it already did.Extractorhad the ownership of its input stream inverted: a handle it opened itself, fromfingiven as a path, was never closed, while a stream the caller supplied and still needed was._cleanupclosed undernot self._flag_s(the flag that gated theopen()), and theSeekableReaderwrapping a non-seekable input asked forstream_closing=not self._flag_sin the same wrong direction. The leak was one descriptor per extraction, which surfaced as aResourceWarningfailing the test suite (#577, #596, #600; #606 only made the test immune). Both conditions now follow ownership via a singleExtractor._owns_input, and a finaliser releases the handle of an extraction abandoned before end of file (auto=False, iterated part way and dropped); the files #610 names emit no such warning. Leaving awithblock follows the same rule, so a caller's stream stays open after it (#1999, #2000). The finaliser is aweakref.finalize, so it also runs where__del__never does, as on GraalPy (#1997, #1998).extract(..., no_eof=True)never returned. End of stream was detected andExtractionWarning: EOF reachedfired, but every loop that handles it (record_frames,__next__,__call__) readif self._flag_n: continue, so the flag suppressed the error that ended the loop without supplying another ending and an exhausted input was retried forever. The flag is not pointless --__main__setsno_eofforfin='-', and the end of what has arrived on a live capture is not the end of the capture -- so this is a termination condition rather than a deletion. The input's own position tells a paused capture from a finished one:prepareraises end of stream when zero bytes remain and restores the position first, so two consecutive ends of stream at the same position mean nothing arrived between them (Extractor._note_eof_progress). A pipe is unaffected, since a read there blocks while the writer is open but idle. This also fixespcapkit -, which hung on any finished stdin. One deliberate narrowing: a seekable input does not block, so a file still being appended to ends at the data present when the extraction reached it; a timed grace period would make the cut-off intermittent rather than absent, so following a growing file is left to a policy of its own. Pre-existing and untouched: an append landing mid-record raisesValueError: read length must be non-negative or -1. The end-to-end tests are bounded by a child process rather thantests._support.time_limit, because the in-process deadline proved intermittent on this loop, and an intermittent guard against a hang leaves a wedged suite rather than a red test. Guesses at the cause are ruled out: the parse path'sexcept Exceptionhandlers are never entered during the spin, and the memory is the retry loop'sEOF reachedrecords, which the runner retains (#620).register_protocoldisplaced one protocol class with another and said nothing. The registry behindpcapkit.protocols.__proto__is keyed oncls.__name__.upper(), and several dispatchable classes are namedHTTP(the basepcapkit.protocols.application.http.HTTPand thehttpv1andhttpv2implementations), so a bare assignment decided the winner:register_protocol(httpv2.HTTP)moved__proto__['HTTP']to thehttpv2class with no exception, log line orRegistryWarning, andProtocolBase.expand_comp('HTTP')then resolved to whichever registered last. The overwrite now raises aRegistryWarningnaming the displaced and the replacing class. The sharpest path is not a call to the registrar:Protocol.__init_subclass__()registers unconditionally, so merely defining a subclass of the publicProtocolunder a name a built-in holds displaced that built-in. A plainimport pcapkitemits zero new warnings: the built-inHTTPclasses derive fromProtocolBase, so__init_subclass__never fires for them, and import-time seeding keys off the distinct namesHTTP,HTTPv1andHTTPv2. The guard deliberately reads "key present and the incumbent is a different class", not mere presence (#726 later gave its sibling registrars the same identity guard): this function is the funnel every wrapper registrar ends in, so registering one class under two codes (register_tcpthenregister_udp, a documented path) reaches it twice with nothing displaced. The collision is reported, not resolved: re-keying is a registry-format change that none of the bare-name readers can absorb (each degrades silently to a string orRawon a miss), so it belongs to the registry redesign in #514. The issue called this the one registrar with no warning; in fact none warned by itself, the siblings only by delegating to a guarded classmethod (#675).- the
#:autodoc comments inpcapkit/foundation/traceflow/traceflow.pythat named a bareType, which Sphinx resolves against every class namedTypein the project rather thantyping.Type, and silently linked topcapkit.const.l2tp.type.Type, an L2TP field-type enum; once #684 renderedTraceFlow._foutio, the built docs pointed at the wrong class with no warning. These sites now spell it~typing.Type[Dumper], among the sites #709 lists. The other bareTypesites, in hand-written:type:fields underdocs/source/pcapkit/foundation/, are fixed separately by #714 (#709). register_protocol's overwrite warning could claim a protocol was replaced with itself. The guard was correct (incumbent is not protocol, an identity check from #681), but the message built both operands with a barerepr(). A factory defining a same-named closure-local class on every call (astests/protocols/test_construction_keyword_check_unit.py's_protocol_classdoes) produces two distinct classes sharing one__module__and__qualname__, so a real overwrite read "overwriting X with X." The message now compares the two reprs and, only when they coincide, appends each object'sid();__module__/__qualname__would not disambiguate, being what the repr already renders (#710).- more registrars warned on mere key presence rather than an actual overwrite, outside the wording of #718's identity guard (
incumbent is not None and incumbent is not new, landed by #726), whose issue named only code-keyed registrars:register_engine,register_reassemblyandregister_traceflowonExtractor, andregister_dumperon bothExtractorandTraceFlow. Each now compares the incumbent by identity before warning; bothregister_dumpersites compare only the stored dumper, so re-registering with just a new file extension stays silent too. Non-breaking: a correct caller sees strictly fewer warnings (#739). - the class registrars listed below now raise
RegistryErrorfor a non-class argument, where a bareTypeError("issubclass() arg 1 must be a class") used to escape -- from the guard itself for theregister_dumpermethods, and from insideabcfor the others. They areregister_dumper,register_engine,register_reassemblyandregister_traceflowonExtractor,TraceFlow.register_dumper()(reached throughregister_traceflow_dumper), andpcapkit.foundation.registry.protocols.register_protocol(). Each guard gains an explicitisinstance(x, type)test ahead of itsissubclass; in those that accept aModuleDescriptorit runs after the descriptor is unwrapped, so a descriptor naming a non-class attribute is rejected too, whileregister_protocoltakes no descriptor. No guard's target class changes, and a wrong class still raisesRegistryErroras before.RegistryErrorsubclassesTypeError, soexcept TypeErrorstill catches it; a caller matching the exact type, or the old message, does not. The same guard in the protocolregisterclassmethods is fixed separately by #1026 (#1021). - a missing extraction engine package now gives a single
EngineWarningrather than a duplicate (#1045, #1046).Extractor.import_test()warned that the package was absent, thenExtractor.run()warned again when it fell back to the default engine;runnow stays silent there, andimport_testsays whatrunused to:engine <name> (<module>) is not installed; using default engine instead, the module name in backticks. Only code that matchedimport_test's old wording, or that expects the duplicate warning, breaks. Extractordefects:Extractor.output'sUnsupportedCallnamedformatrather thanoutput;make_namelooked an unknown format up in the__output__defaultdictand so inserted it; and the Scapy engine never set_offmt, soExtractor.formatraisedAttributeErrorunderengine='scapy'(#1095).- the DPKT, Scapy and PyShark engines call
Extractor.record_header()at setup again, as they did until 2023, so the global-header step is shared with the default engine and sets_offmtfor each of them. Their dumped output changes: JSON, plist and tree reports gain the leadingGlobal Headerrecord (Section Header 1for PCAP-NG) the default engine already writes, identical to its, andfiles=Truewrites that record's file too; frame records are unchanged.Extractor.formatnow also answers on a capture with no frames.format='pcap'failing on every engine is a separate defect, #1127 (below) (#1104). format='pcap'failed on every engine with aTypeErrorfromPCAPIO.__init__(), missingprotocol:Extractor.__init__()created the writer before any global header was read, soPCAPIOhad no link type. The header step now creates it throughExtractor._open_output(), andrun()opens it for engines with no such step; a PCAP writer takes the frame itself and skips theGlobal Headerrecord, so the default engine writesin.pcapback byte for byte. The third-party engines hand the writer no frame octets, so they fall back to JSON with aFormatWarning, as their flow tracing already does, and a PCAP-NG input raisesFormatError(#1127). The written global header also dropped the input'sthiszone,sigfigsandsnaplen, so most sample PCAP files did not come back byte for byte; the PCAP engine now passes those fields through_open_outputtoPCAPIO, and every one does (#1165).- IP reassembly reports only the octets it holds: a truncated fragment, a strict run and a zero-length datagram no longer report octets from the declared Total Length (#1368). Only the final fragment's partial block counts, and a datagram too long for its length field never completes (#1582, #1585, #1598).
- TCP reassembly judges completion from each buffer's own gaps, compares sequence numbers modulo
2**32, and holds a FIN until reordered data arrives (#1375). A TSO fragment's length is its captured length, andRCVBTcovers the last block (#1564). - a second TCP flow with the same label truncated the first flow's file; a reused label now gets the next free suffix (#1360, #1371).
- the engine, reassembly, traceflow and dumper registrars accept a built-in back under the name it shipped with, restoring the shipped entry after an override; the engine, reassembly and traceflow registrars still refuse any other built-in (#1363, #1376).
- PCAP-NG blocks are numbered across the whole file, not per section (#1256, #1426).
format='pcap'is kept when an unavailable engine falls back to the default one (#1264, #1430).extract()accepts a file object with nonameasfin(#1506, #1509), and reads every frame from a pipe or other non-seekable stream (#1586, #1595).- the third-party engines read what the capture holds. The dpkt engine keeps a frame dpkt cannot parse rather than leaking
AttributeError(#1351, #1377), applies each PCAP-NG interface's link type and timestamp resolution rather than the first one's (#1379, #1396), and reads float timestamps, Simple Packet Blocks and raw-IP frames (#1550). The scapy engine resets its interfaces at each PCAP-NG section (#1522, #1546), takes a packet's link type and clock from its interface (#1563), and reads each record whole (#1717, #1733). The pyshark engine skips non-packet PCAP-NG blocks (#1515, #1531), and the pypcapfile engine reads the PCAP records pypcapfile drops (#1512, #1569). dpkt and scapy write real PCAP traces (#1507, #1511). - a callable passed as
verbose=is called on every engine. The scapy, dpkt, pyshark, pypcap, pcap-ct and pypcapfile engines replaced it with their own printer, which now runs only forverbose=True, its output unchanged (#1874, #1880). - an
Extractorwhose constructor raises closes what it owns, the input it opened and then the engine it chose, so a PyShark capture'stsharkdoes not outlive it; the original exception still propagates, and a caller's stream stays open (#2100, #2106, #2107, #2119). engine='pypcap'warns and falls back to the default engine when thepcapmodule is installed but cannot load, asengine='pcap_ct'already did, where it raised the loader'sOSError(#2132, #2133).- TCP reassembly bounded its holes with
sys.maxsize, which on GraalPy and 32-bit builds is2**31 - 1, inside TCP's sequence space, so a stream past it never completed. The bound isHOLE_OPEN_BOUND,2**63 - 1, the value 64-bit builds already used (#1959, #1965). - the method-name registrars, such as
register_ipv4_option, andregister_linktype,register_pcap,register_pcapng,register_ethertypeandregister_transtyperefuse acodethat is not an enum member withRegistryErrorbefore touching a registry, where they registered it and then raisedAttributeError, leaving the entry behind (#2186, #2193). pcapkit.foundation.registry.protocols.register_protocol_code(), and thecode=class keyword, check every destination's key before registering any of them. A key belonging to another enum, such as{Frame: TransType.ICMP}, is refused withRegistryErrorwhere it replaced the shippedLinkType.ETHERNETentry, and a bad target late in the list no longer leaves the earlier ones registered. A destination keeping a registry of its own, such asLoopbackandIPX, is not checked against another enum (#2268, #2343).- the number registrars
register_tcp,register_udp,register_sctpandregister_apptyperefuse aboolcode withRegistryError, whereregister_tcp(True, ...)registered port 1, sinceboolis anintsubclass (#2269, #2344). - IP reassembly reports a datagram that is not itself fragmented but whose BUFID is already buffered: it submits that buffer as incomplete and then reassembles the datagram in a buffer of its own, per RFC 791 Section 3.2, where the datagram was never reported at all. On IPv6 an atomic fragment is reassembled in a buffer of its own and submitted without flushing the datagram already being reassembled, so that one's final fragment still completes it, per RFC 6946 Section 4 (#2270, #2345).
- the PyPCAP, pcap-ct and pypcapfile engines warn with
AttributeWarning, naming the engine, that they ignore a parsing context supplied throughExtractor'scontext=, as the dpkt, scapy and pyshark engines already did, where they dropped it silently (#2313, #2332). Deferredcould not be pickled under protocol 0 or 1 -- a slotted class with no__getstate__raisesTypeErrorthere, independent of theanalyze/protoregression #2417 fixed. It now defines__getstate__and__setstate__, mirroring the(dict_or_None, slots)pair Python's own default slotted-class reduction already used for every released build, so aDeferredpickled by one of those still unpickles. The slot names come fromcopyreg._slotnamesrather than a hand-rolled walk oftype(self).__mro__, which would otherwise shadow a base class's slots under a subclass declaring its own, miscount a bare-string__slots__ = 'name', or read a private, name-mangled slot under its unmangled spelling (#2423, #2425, #2424).- that same
Deferredstill failed to pickle under protocol 0 or 1 on GraalPy 25.4, after #2423's fix:copyreg._reduce_ex, which handles protocol 0 and 1 before__getstate__is ever consulted, walkscls.__mro__for a base to compute a base-object state from, and on GraalPy that walk stops atGenericitself rather than atobject, raisingTypeError: typing.Generic() takes no arguments.Deferred.__reduce_ex__now hard-codes thebase=object, state=Noneoutcome the walk always reached on CPython for protocol 0 and 1, so the walk -- andGeneric-- is no longer part of the route pickling takes. The same override also checks for, and defers to, a subclass's own__reduce__before building that hard-coded tuple -- without that check aDeferredsubclass overriding__reduce__had it silently ignored at protocol 0 and 1 while protocol 2 and above (viaobject.__reduce_ex__unchanged) kept honouring it (#2433, #2435).
pcapkit.interface
Added
pcapkit.interface.registry, making public the registry-entry helperssame_entry,restore_shippedandrestore_shipped_dumper(#1519, #1568).
Fixed
- the messages the interface functions and the helpers behind them raise and warn read in the house
<prefix>: <lowercase description>: ... {value!r}form.reassembleandtracelowercase their unsupported-protocol error and quote the protocol;follow_tcp_streamwarnsusing default engine instead, the wordingExtractoralready used, and quotes an engine that has no reassembly adapter;TraceFlowBase's output-format warning is lowercased; andExtractor.make_namequotes the format it does not know, where an empty one ended the message at the colon with nothing after it (#2303, #2374).
pcapkit.protocols
Added
- ESP parsing and construction [RFC 4303], with optional payload decryption and ICV verification through
cryptography(pip install pypcapkit[crypto]) (#378). Keys reach the dissector through a new caller-state channel,pcapkit.corekit.context, surfaced as thecontext=keyword onextract()andExtractor, carrying anesp.SecurityAssociation. Nothing here raises: with no association the SPI and sequence number are still reported and the ciphertext is left opaque withstatus=NO_SA, and a failed decryption or ICV check is recorded on the parsed result the same way. Extended Sequence Numbers, TFC padding and anti-replay are not implemented, and an unsupported cipher or MAC is refused with a clear error rather than half-processed. - SCTP as a transport protocol [RFC 9260]: every chunk type, chunk parameter and error cause it defines, with the CRC32c both recorded and verifiable (it covers the SCTP packet alone, with no IP pseudo-header). Upper layers register on the DATA chunk's Payload Protocol Identifier through
register_sctp, not on a port (#379). - NGAP over SCTP (3GPP TS 38.413), decoding aligned PER through
pycrate(pip install pypcapkit[NGAP]) (#251, #417). Decoding is generic by ASN.1 shape rather than per-procedure, so every elementary procedure and protocol IE works and a new 3GPP release needs no code change. Registered as a default on PPID 60 and 66, but only 60 decodes: PPID 66 is an NGAP PDU inside a DTLS record, and there is no DTLS dissector.pycratewas at first left out of theallextra -- it is LGPL-2.1+ where this package is BSD-3-Clause, and lands some 238 MB to obtain one module -- until #910 counted it amongall's core addons, soallinstalls it (theallentry under Project infrastructure). - the Mobility Header registry, completed (#383, #437): the RFC 5568 fast-handover messages and options, then every registered message data type, every registered option (with nested sub-option registries for the flow identification, access network identifier, quality-of-service and LMA-controlled MAG parameter families) and every CGA extension. The CGA Parameters option, the only option that can carry a CGA extension on the wire, was the last to come off the generic handler, which is what let the CGA extensions round-trip end to end.
- dispatch entries for dissectors that existed but were reachable from no registry (#436): FTP-DATA on TCP 20, HTTP/1 on TCP 8080, HTTP on UDP 8080,
L2TPv2on UDP 1701 and OSPF atTransType89.VLANbecame an abstract base withC_Tag(802.1Q) andS_Tag(802.1ad) as concrete subclasses, so a Q-in-Q frame no longer collapses into one opaqueRaw;L2TPlikewise became a base, withL2TPv2carrying the RFC 2661 implementation. Protocol/ProtocolBasegain acode=class keyword for the next-layer dispatch registries (Link.__proto__,Internet.__proto__,TCP.__proto__,UDP.__proto__,SCTP.__proto__,Frame.__proto__,PCAPNG.__proto__), the opt-in treatment #514 gaveEngine,Reassembly,TraceFlowandDumperabove, extended to the one family that needed a registration key invented. Omittingcodeleaves a subclass unregistered, as before; the built-in tables are still filled by literal assignment in each layer module, so nothing the library ships moves.codeaccepts a bare enum member, whose type infers the destination (EtherTypemeansLink,TransTypemeansInternet,PayloadProtocolIdentifiermeansSCTP, andLinkTypemeans bothFrameandPCAPNG, mirroringregister_linktype), or a{destination: key}mapping, required for a rawintsuch as a port number, which cannot say which transport it belongs to. Either form may appear in an iterable, so one declaration can register a class into several registries (e.g. aL2TPsubclass reachable by IP protocol number and by a UDP port), and the mapping form is accepted even where inference would work. Inference refuses rather than guesses: an enum member whose type names no known destination raisesRegistryError, and an unrecognised class keyword raisesUnsupportedCall, matching the other registrar families. Backed bypcapkit.foundation.registry.protocols.register_protocol_code(), also callable directly for a class that declined at definition time. It was written up as the mechanism #548 needs; #548 turned out to need a missing class instead (115 is RFC 3931 L2TPv3 over IP, not RFC 2661; see the corresponding Fixed entry below), so its worked example namesL2TPv3(#570).tests/protocols/test_dispatch_reachability_unit.py, the coverage #548 asked for: everyProtocolBasedescendant whose__index__returns an enum member must be reachable under that code in the registry its enum type designates, read from the same__proto_dest__declaration backingcode=so the two cannot drift. It walks the classes rather than the registry entries, so it catches a class nothing registered, the shape in whichOSPFonce shipped (#548).tests/protocols/transport/test_tcp_mptcp_join_flag_ordering_unit.py, covering all three MP_JOIN layouts through the public constructor, the construct-pack-parse cycle for each, the stale-flags case that rules out a zero-valued default, the statement order, and controls for the parse path and the flag-independent options. The old cases bypassed the public path, so full coverage coexisted with a broken one. The staletcp-mptcp/MP_JOINentry is deleted fromEXPECTED_FAILURES(#587).tests/protocols/test_option_generator_tcp_base_unit.py, asserting that everyTCP_BASEkey is a parameterTCP.make()declares (derived frominspect.signature) and that the built segment carries the stated header in both data model and packed octets (#602).- ILNP nonce sizing coverage in
tests/protocols/internet/test_ipv6_extension_unit.py, a test per protocol, asserting the declared length, the exact packed octets and the construct-pack-parse cycle over a spread of nonces, most with a bit length that is not a multiple of eight, where floor division and the ceiling disagree (#601). SOLUTIONparameter width coverage intests/protocols/internet/test_hip_unit.py: a spread of widths, each asserting the declared length, the reader's acceptance and the construct-pack-parse cycle, plus a 57-bit case pinning the 20-octet length RFC 5201 Section 5.2.5 requires of HIPv1. Some widths are deliberately not multiples of eight, where the defectiveceil(bits / 4)differs from the correct width (#608).- the BSD loopback layer for
LINKTYPE_NULLandLINKTYPE_LOOP, and dissection ofLINKTYPE_RAWby the version nibble (#1594). - one home for each RFC- or format-fixed constant spelled out in more than one module -- the IPv4, IPv6, AH and TCP header lengths and offsets,
IP_MAX_LENGTH, and the PCAP and PCAP-NG framing constants -- in the module that owns the protocol (#1739, #1752).
Changed
- a breaking change to
OSPF,RARPandDRARP: they move frompcapkit.protocols.linktopcapkit.protocols.application, with no deprecated re-export at the old path (#719). Layer is decided by designed function, with the IETF as the single source of truth -- RFC 1812 Section 7 places routing protocols in the application layer and RFC 1122 Section 1.1.3 lists RARP there -- and the rule is recorded indocs/source/contributing/conventions/protocol-layer-placement.rst.
Import paths, before to after:
pcapkit.protocols.link.ospftopcapkit.protocols.application.ospfpcapkit.protocols.data.link.ospftopcapkit.protocols.data.application.ospfpcapkit.protocols.schema.link.ospftopcapkit.protocols.schema.application.ospfpcapkit.protocols.link.rarptopcapkit.protocols.application.rarp
from pcapkit import OSPF and from pcapkit.protocols import OSPF (likewise RARP and DRARP) are unaffected; from pcapkit.protocols.link import OSPF and the deep paths above break. RARP has no data or schema module of its own, as it reuses ARP's.
layer changes from 'Link' to 'Application' for OSPF, RARP and DRARP. OSPF now takes Application as its only base, and RARP becomes class RARP(Application, ARP) -- layer base first, because ARP's chain reaches Link, which owns __layer__. ARP and InARP stay 'Link'. Dispatch keys are unchanged: OSPF is still reached from Internet.__proto__ at TransType.OSPFIGP and RARP from Link.__proto__ by EtherType. Leaving Link repoints OSPF.__proto__ at ProtocolBase.__proto__, so OSPF no longer sees Link's EtherType entries -- which nothing used, since it is reached by protocol number, and a -1 lookup falls back to Raw in either registry without inserting. register and _read_protos still resolve, on ProtocolBase rather than Link. RARP keeps Link's through ARP.
Layer-limited extraction: layer='link' no longer sets the termination flag on OSPF and layer='application' now does -- the same inversion applies to RARP -- but the extracted protochain is unchanged at every layer= value, including 'internet' (IPv4:OSPFIGP before and after, as IPv4 ends the chain itself).
- renames with no compatibility alias left behind: PCAP-NG
Optionsubclasses spell the namespace class keywordns=instead ofnamespace=(#439). tests/protocols/transport/test_tcp_udp_unit.pynow reaches the MP_JOIN dispatchers throughTCP()itself, instead of assigning a Pythonsetto_flagson a bareTCP.__new__(TCP). Asetanswers the same membership tests, so every branch ran while the attribute had neither the type production assigns nor the ordering that governs when it exists -- how #587 stayed invisible. Reverting #587's hoist, or restoring thecast('Enum_Flags', 0)seed, now fails the file. Test-only (#603).- building a protocol through its constructor with a keyword that names nothing now raises
UnsupportedCallinstead of discarding it. This is a behaviour change to a public API: everymakein the tree ends its signature with**kwargsand reads nothing out of it, so a misspelled keyword was accepted, dropped, and the field it named kept its default -- wrong octets, nothing said, as in #602, #541 and #556. The check sits inProtocolBase.__init__()rather thanmake, because__post_init__passes one**kwargsto the construction and to the parse of what it has just constructed, so a keyword declared only byreadlegitimately travels throughmake(HIP.read()declaresextension). The accepted set is the union of every keyword-taking parameter ofmake,read,pack,unpack,__post_init__and__init__across the MRO. Parsing is deliberately untouched: a protocol cannot know which keywords its parent passed on. A new__keywords__opts a class out: a set, for a keyword read out of**kwargsby name (ESP.read()withpacket); andNone, for a dispatcher whose real signature belongs to a class chosen at call time (onlyHTTP.make). The message names the near neighbour (seqreports did you mean 'seq_no'?).from_datawarnsUnknownFieldWarningrather than raising for a key its own_make_datareturned, which the caller cannot fix; a keyword passed tofrom_dataitself raises, as for a direct construction (#1147). The leftover copies of #602's misspelled TCP base, inexamples/generators/dispatch.pyandtests/protocols/transport/, are fixed. Some_make_datakeys thatmakedoes not declare are reported, not fixed:Frame._make_data()returnsts_srcwheremakedeclarests_sec,L2TPv2._make_data()returnspriowhere it declarespriority, andHeader._make_data()returns amagic_numberthatHeader.make()does not take. A directSomeProtocol.make(...)call is not checked and still discards silently (#617). - a breaking change to
Application: it no longer forbids payload dispatch outright. It encodes "no further protocol layer above this one", which undissected trailer bytes are not, so_decode_next_layerand_import_next_layeraccept the-1sentinel -- the rest of the packet, resolving toRaw, or toNoPayloadwhen nothing remains -- and__post_init__keeps a payload thatreadset instead of overwriting it withNoPayload. A real protocol number (anyprotoother than-1) is still refused withUnsupportedCall. A subclass that calls either method with-1changes from raising to succeeding; no subclass in the package does (#719). - the
ProtocolandProtocolMetadocstrings.Protocolclaimed to be the base of every protocol family, though nothing in the package subclasses it, andProtocolMetaclaimed to generate attributes its empty body does not.Protocolis now documented as the base for third-party protocols, andProtocolBaseas the one the families derive from (#1101). - a breaking change to
OSPF_CrytographicAuthentication: the misspelling is corrected toOSPF_CryptographicAuthentication, andCrytographicAuthenticationinpcapkit.protocols.schema.application.ospfandpcapkit.protocols.data.application.ospftoCryptographicAuthentication, with no alias left for the old spelling. Both OSPF names also move to the application group ofschema.__all__anddata.__all__, after #719's move (#1090). - a breaking change to a declared length that runs past the data: it is refused with
ProtocolErrorrather than accepted and zero-filled on rebuild, so a malformed option looks different ininfo(#1325, #1398). - dissection is bounded: a frame at 64 layers (#1610, #1620) and an IPv6 extension-header chain at 32 headers (#1604, #1608). Both limits are
PCAPKIT_FRAME_LAYER_LIMITandPCAPKIT_EXTENSION_HEADER_LIMITinpcapkit.utilities.config(#1743). Rebuilding a frame cut at the chain bound warns once (#1619, #1630). - a breaking change to IPv6 extension headers: an unrecognised or unparsable one is parsed generically, per RFC 6564, rather than collapsing the packet to
Raw(#891, #904).IPv6_Extdoes it, and is also the shared base of every extension header (#917, #924). - a breaking change to
pack(): it no longer keeps the header it builds on the instance it is called on, which in 1.4.1 too leftschemadisagreeing withinfo:UDP(srcport=1234).pack()resetschema.srcportto 0 whileinfo.srcportkept 1234. Once an instance has been parsed or constructed,pack()builds on a shallow copy, soschema,infoand whatmake()sets, such as TCP'sconnection, stay as they were. TheFrame,PCAPNGandHTTPv2