11.2.2

Security

• Crafted email messages no longer take seconds to minutes and hundreds of megabytes to parse (GHSA-qcc2-4x3v-v4g7, affects 11.2.1 and earlier). parse_report_email() (mailbox, mbox and file input) and the failure-sample parser parsedmarc.utils.parse_email() gave the whole message to mail-parser and Python's email package with no limit on blank lines, header lines, header field size, MIME parts or nested multiparts. Parsing builds one Message per blank line after a message/delivery-status part (512 KiB of blank lines took 6.1 s and 283 MiB of extra memory on 11.2.1 under Python 3.12) and tests every line against every open boundary of nested multiparts (900 levels around 30,000 parts, 0.3 MB, took 10.9 s); a 4 MiB flood of header lines took 1.4 s, and a 1 MB To: address group of 250,000 members took 1.9 s, or 44 s through mailsuite's own parse_email(), which parsed the group whole. With n_procs = 1 the rest of the mailbox waited behind such a message, and a process with a memory cap was killed before the message was moved, so it was parsed again on the next run. Both parsers now call mailsuite.utils.check_message_limits(), new in mailsuite 2.3.5 (GHSA-748w-c9w9-79xp), before mail-parser sees the message. It parses the message with the email package's own feed parser under the compat32 policy, the parser mail-parser and email.message_from_string() use, and stops as soon as the message has more than 10,000 parts (the message itself, its MIME parts, the bodies of message/* parts and each block of a message/delivery-status body), nests them more than 100 levels deep, or makes the parser test a line against an open boundary more than 20,000,000 times. It also stops when the message's header blocks hold more than 100,000 lines in all (folded lines and lines the parser keeps only as defects included). It then rejects a parsed message with a header field longer than 65,536 characters (name, colon and raw value, folded lines included), or whose From, Sender, Reply-To, To, Cc, Bcc and Delivered-To headers hold more than 262,144 characters in all, counted in every part; that limit bounds mail-parser's own address parsing (it parses From, Reply-To, To, Cc, Bcc and Delivered-To), which does work for each malformed address item. When a part lacks its start boundary, mail-parser parses the top-level epilogue again, and that text is checked with the same limits. parse_report_email() also no longer renders every part it walks as text: it renders a part only where it reads it, because rendering each level of a deep nesting rendered everything inside it again (98 nested multiparts around a 20 MiB leaf took 26.6 s, a chain of 99 message/rfc822 parts around a 10 MiB leaf 11.3 s, and a chain of 99 message/feedback-report parts around a 10 MiB leaf 11.6 s, all within the limits; each now takes under a second). An over-limit email raises InvalidDMARCReport("Message rejected: Message exceeds <limit> (<value>)"), which the mailbox loop files under Invalid; an over-limit failure-report sample raises InvalidFailureReport("Sample message rejected: …"). Each crafted message from the report is now rejected in under half a second (the 4 MiB header flood, the slowest, in 0.2 s). mailsuite 2.3.5 or later is now required. The floor applies to the base dependency and to the msgraph, gmail and all extras. mailsuite 2.3.5 also removes RFC 5322 group syntax from address header values over 8 KiB before parsing them, caps the address header value its own parse_email() reads at 65,536 characters, and fixes a quadratic url_regex; all only affect callers of mailsuite's parse_email(), which parsedmarc does not use. A message that still takes too long to parse can now be set aside with the new parse_timeout option (see Added). Compatibility: the limits are fixed; no option changes them. A message with more than 10,000 parts or parts nested more than 100 levels deep is now rejected as invalid instead of being parsed or, when nested past Python's recursion limit, failing with a ParserError; in mbox input such a message (the nested message of GHSA-qxwv-2vwj-v828 is one) is now skipped with a warning instead of stopping the run. A message/rfc822 attached inside a failure report's sample no longer replaces the sample, since a feedback report has one sample (RFC 6591 §3.1), and only the first message/feedback-report part of a message is read, since a feedback report has one (RFC 6591 §3). Reported by helixnebulatech.
• Deeply nested or very wide aggregate report XML no longer exhausts memory (GHSA-q7gm-8cj9-94hr, affects 11.2.1 and earlier). parse_aggregate_report_xml() passed the decompressed report, up to the 100 MiB decompression cap, to xmltodict.parse() with no limit on nesting depth or element count. xmltodict keeps one stack entry per open element and expat reports missing end tags only at the end of the input, so a gzip attachment of about 30 KB holding 30 MiB of unclosed <a> tags needed more than 2 GB of memory (2 MiB of them cost 255 MiB on 11.2.1). The out-of-memory killer ended the process before the message was moved, so every restart parsed it again. A document of millions of sibling elements did the same at a smaller scale, and a document whose first parse failed was parsed again through lxml recovery and a second xmltodict pass. While expat reads the document, before xmltodict adds each element to its tree, parsedmarc now counts nesting depth and nodes (elements plus attributes); a report deeper than 100 levels (MAX_XML_DEPTH) or with more than 1,000,000 nodes (MAX_XML_NODES) is rejected as InvalidAggregateReport("Invalid XML: …") without trying lxml recovery. Before any parser sees the document, a tag longer than 65,536 characters (MAX_XML_TAG_LENGTH) is rejected too, because expat and libxml2 build an element's whole attribute list before its attributes can be counted (one element with 900,000 attributes, 9.3 MiB, within the node limit, took 1.1 s and +200 MiB; it is now rejected in under 0.1 s). Because a quoted attribute value may hold a raw >, which splits a tag for that scan, a document with more = characters than the node limit is rejected first, in one pass (2,000,000 x=">" attributes, 23.7 MiB, took 1.4 s and +310 MiB before the node limit rejected them; they are now rejected in under 0.1 s). The decompression cap for a report attachment is lowered from 100 MiB to 32 MiB (MAX_DECOMPRESSED_REPORT_SIZE), and an uncompressed report of more than 32 MiB of UTF-8 is rejected the same way, as is a report file of that size, which is no longer tried as a report email: at the node limit a legitimate report is at most about 28 MB, and with that cap a report's text is bounded along with its node count. When the first parse fails and lxml recovery runs, a parser target that builds no tree first counts the document's elements, attributes and namespace declarations against the same node limit (a 22 MiB wide document behind a malformed XML declaration took 5.0 s and 1,532 MiB of extra memory before, 0.57 s and 23 MiB now), and the recovered XML then goes through the same depth and node limits in the second xmltodict pass. A mailbox message that still kills the process before it is filed can now be set aside after two attempts with the new [mailbox] attempts_file option (see the last entry in this section). Compatibility: there is no new option. A real report is fewer than 10 levels deep and the largest sample (909 KB) has 32,019 nodes, so the node limit is a report of about 28 MB of XML; a larger report is now rejected. A report at the size cap needs about 400 MB of memory on the lxml recovery path and about 350 MB otherwise when its text is parsed directly, and about 500 MB and 450 MB through parse_report_file(), which also holds the file's bytes, where a report under the old 100 MiB decompression cap could need gigabytes. An aggregate or SMTP TLS report of more than 32 MiB of UTF-8, compressed or not, is now rejected, where 100 MiB was accepted (an uncompressed SMTP TLS report had no cap: 60 MiB of JSON cost +1,505 MiB). parse_report_file() no longer tries a file that begins with < as SMTP TLS JSON or a report email, nor one that begins with { as a report email; its error message is unchanged, and the exception's cause now matches the format the file begins with. It also keeps only each failed attempt's message while it tries the next format, so a file at the size cap on the lxml recovery path takes +500 MiB there, where it took +1,046 MiB. The XML parse step is about 20% slower, and a malformed document that needs lxml recovery is now parsed twice by lxml (once to count, once to build the tree). Reported by helixnebulatech.
• An aggregate report with a huge date range can no longer expand into millions of records or fill an Elasticsearch or OpenSearch cluster with indexes (GHSA-p85v-578x-3wcf, affects 11.2.1 and earlier). An aggregate report whose <date_range> was longer than normalize_timespan_threshold_hours was split into one record per calendar day, with no limit on the number of days. A report under 2 KB declaring a range of millions of days with a large <count> expanded one <record> into millions of records (200,000 days gave 200,000 records in 0.8 s; the reporter's 2.9 million days exhausted 2 GB of memory). With Elasticsearch or OpenSearch output each of those days created its own dated index of one shard until the cluster reached cluster.max_shards_per_node (1,000 by default), after which it refused every new index for every application using it. A report whose date range is negative or longer than 366 days (MAX_REPORT_SPAN_DAYS) is now rejected with InvalidAggregateReport. A report whose split would produce more than 250,000 records (records times days, MAX_NORMALIZED_RECORDS) is not split: its records are kept whole over the report's own date range, and one warning is logged (10,000 records over 366 days, each with a large <count>, produced 3,670,000 records in 13.5 s and 1.7 GB of memory before). In Elasticsearch and OpenSearch a single report creates at most 31 dated indexes (MAX_INDEXES_PER_REPORT): any further records are saved to the report's first index with their own dates, and one warning is logged for the report. Dated index names still come from each report's own dates, so reports with many different dates still create one index per date; data streams (use_data_streams) are affected by neither. Compatibility: a genuine report spanning more than 366 days is now rejected; a report too large to split has timespan_requires_normalization: false and records with normalized_timespan: false; there is no new option. Reported by helixnebulatech.
• One crafted report can no longer make an output fail and stop every run on the same mailbox (GHSA-4272-4r8g-6c5h, affects 11.2.1 and earlier). Some reports made a configured output raise an exception that process_reports() did not catch. It escaped the mailbox save callback, parsedmarc logged "Mailbox Error" and exited with status 1, and the batch stayed in the mailbox, so every one-shot run failed on the same message, outputs that do not deduplicate received the batch's legitimate reports again on each attempt, and watch mode finally moved the whole batch to Unsaved. The triggers were a failure report whose sample has an empty From:, To: or Reply-To: header (a legitimate To: undisclosed-recipients:; too) with Elasticsearch or OpenSearch output (IndexError); an SMTP TLS report with a field of the wrong type, such as "policy-string": [1] with file output (TypeError in the CSV writer) or "start-datetime": 5 with Elasticsearch, OpenSearch, Splunk HEC or PostgreSQL output; and an aggregate report whose policy override <reason> has no <type>, or a <type> with an attribute or given twice, with file output (KeyError, TypeError). Every output destination in process_reports() now records any exception as an output error, as the aggregate Elasticsearch and OpenSearch blocks already did (the failure and SMTP TLS Elasticsearch and OpenSearch blocks, every PostgreSQL and Splunk HEC block and the file output lacked that catch-all), so the batch is reported unsaved and fail_on_output_error still applies. SMTP TLS reports are now type-checked against RFC 8460 §4.4 and rejected as InvalidSMTPTLSReport otherwise: a string for organization-name, report-id, policy-domain, policy-type, result-type and the optional failure-detail strings, a string or a list of strings for contact-info, a list of strings for policy-string, an integer or null for each session count (true and false are rejected), and a list of objects for failure-details. start-datetime and end-datetime must convert with the same functions the outputs use and be at most 64 characters long, because dateutil took 12.4 s to fail on a date of 1,000,000 digits. A string holding a lone surrogate (JSON "\ud800"), which I-JSON (RFC 7493 §2.1) forbids, is rejected too: it passed every type check and made printing the results to standard output raise UnicodeEncodeError, which process_reports() now also records as an output error; a report that cannot be printed to the console (a non-UTF-8 console, a closed pipe) is now recorded as an output error instead of ending the run. A policy override reason's type and comment are normalised to a string or null (an element with an attribute to its text, a repeated element to null), so the CSV writer and the Elasticsearch and OpenSearch saves no longer fail on them. An address header with no address is left out of the Elasticsearch and OpenSearch failure document and of the duplicate query. The Splunk HEC SMTP TLS save raises SplunkError on a date it cannot convert, and the PostgreSQL date helper passes a non-string value through unchanged. Compatibility: an SMTP TLS report with a field of the wrong type that used to parse is now invalid and goes to Invalid; a policy-string that is not a list is now rejected (it was silently dropped), while a null one is still accepted as absent. A policy override reason without a type now has type: null and is written as none in CSV; one whose type has an attribute is stored as its text, and a repeated one as null. Reported by helixnebulatech.
• One crafted report can no longer change the field types of an Elasticsearch or OpenSearch index and get every later report rejected (GHSA-qpcg-29vr-jwm3, affects the Elasticsearch and OpenSearch outputs in 11.2.1 and earlier, data streams included). Indexes and the data-stream index template were created without field mappings, so the cluster typed each field from the first document that held it, with date detection on, and a dated index is named from the report's own begin date. One crafted report could make report_id a date field (<report_id>2026-11-09</report_id>), turn sample.headers.subject into an object (with a sample header named Subject.x), or use up index.mapping.total_fields.limit (1,000 fields) with invented sample header names. Every later legitimate report for that index, or for that data stream until its next rollover, was then rejected with a mapping error, retried and moved to Unsaved; this was confirmed against Elasticsearch 8.19.7 and OpenSearch 2.19.3. New indexes and data-stream templates are now created with date detection off and the date fields declared as dates (@timestamp, date_range, date_begin and date_end; for failure reports @timestamp, arrival_date and sample.date); when an aggregate report is parsed, every XML attribute is dropped and every field the schema types as a string is stored as a string or null, so a report cannot change a field's type (an element with an attribute or a child element, such as <p a="1">none</p> or <p><x>1</x></p>, became an object that mapped the field as one). In failure documents sample.headers is mapped with dynamic: false and indexes only 29 headers, those the shipped dashboards reference plus cc; every header stays in _source except a From, To or Reply-To header that holds no address (see GHSA-4272-4r8g-6c5h above), and a . in a header name is stored as _. create_indexes() and create_data_streams() take a new optional mappings argument. Compatibility: existing dated indexes are not changed; the protection applies to dated indexes created by 11.2.2 or later, and, because parsedmarc puts the data stream template again in every process, to an existing data stream's next backing index after a rollover. On those, sample headers outside the list are no longer searchable. The new "Field mappings" sections of the Elasticsearch and OpenSearch documentation show how to add one through the <stream>@custom component template (Elasticsearch data streams), an index template (dated indexes) or the put-mapping API (OpenSearch data streams, repeated after each rollover). For string fields, an aggregate report element with text and a stray child element now yields its text, where it yielded an empty string or an object before: samples/aggregate/invalid_xml.xml, whose <header_from>bad<xml.net</header_from> lxml recovery turns into mixed content, now gives the header-from domain bad. A <begin> or <end> with mixed content is still rejected. This narrows, but does not replace, the explicit mappings proposed in #782. Reported by helixnebulatech.
• A message that kills parsedmarc while it is being parsed is no longer parsed again on every run (the retry-loop part of GHSA-qcc2-4x3v-v4g7, GHSA-q7gm-8cj9-94hr and GHSA-4272-4r8g-6c5h, affects 11.2.1 and earlier). A mailbox message is moved out of the reports folder only after it has been parsed, so a message whose parse ended the process (the out-of-memory killer, a container memory limit) stayed in place, and every later one-shot run or service restart parsed it again and died before reaching the reports behind it. The fixes above reject the messages those advisories describe; the new [mailbox] attempts_file option is the backstop for any other. When it names a file, parsedmarc records each message in it before parsing it and removes the entry as soon as the parse returns a report or rejects the message, so an entry left behind marks a parse that never finished. A message found with two unfinished attempts (parsedmarc.attempts.MAX_PARSE_ATTEMPTS) is logged at error level ("Message UID … in … has 2 unfinished parse attempts; the limit is 2") and moved to Invalid, or deleted under delete_invalid, without being parsed or counted again; its entry is removed once it has left the folder, and a message left in place (a failed move, test mode) keeps the same count. With n_procs above 1 or parse_timeout set, a message that was being parsed when a run died is parsed alone on the next run, after the rest of its batch, so the messages parsed beside it are not blamed; it is parsed in the main process, or, with parse_timeout set (see Added), in a worker process of its own, so the timeout still applies to it. The entry is removed before the batch is saved, so a failed save still follows max_unsaved_retries and Unsaved and is never counted as a crash. A message is identified by the reports folder, its id and a hash of its first 64 KiB, because mailsuite's IMAP backend does not expose UIDVALIDITY, so a new message that reuses an old UID starts at zero. The file is JSON, written atomically and readable by its owner only, and entries older than 30 days are dropped; a file of another shape is replaced with a warning, and a file that cannot be written logs an error and turns the protection off for the rest of the batch without stopping the run. PARSEDMARC_MAILBOX_ATTEMPTS_FILE sets it from the environment (it names the file directly, like PARSEDMARC_GENERAL_LOG_FILE, so PARSEDMARC_MAILBOX_ATTEMPTS_FILE_FILE is its Docker-secret form). get_dmarc_reports_from_mailbox() and watch_inbox() take a new attempts_file argument. The systemd and Docker Compose examples in the documentation now keep the file in a state directory and a named volume. Compatibility: the option is unset by default, which keeps the previous behaviour; there is no option for the number of attempts.

Added

• [general] parse_timeout stops a report file or message that takes too long to parse (GHSA-qcc2-4x3v-v4g7). The parse limits above bound every input known to take long, but a message they do not cover could still hold up the rest of the run. With parse_timeout set to a number of seconds (300 is a reasonable value), a parse that runs longer is stopped and the input is treated as invalid: a report file is logged as failed and, with archive_directory, moved to Invalid; an mbox message is skipped with a warning; a mailbox message is moved to the Invalid archive folder, or deleted with delete_invalid. A parse can only be stopped by ending the process that runs it, so with a timeout set parsing always runs in worker processes, even with n_procs = 1; when one times out, every worker process is stopped and restarted, and the other inputs that were still being parsed start again. parsedmarc checks the time only while it waits for a parse result, so a parse is stopped within one second of parsedmarc next waiting for a result after its time is up, and can run on past the timeout while parsedmarc is busy between results (moving messages, for example); its clock starts only when a worker begins it, not while it waits for one. With a timeout set, workers hand each result back through a file in a private temporary directory (under TMPDIR), so that stopping a worker while it returns a large result cannot leave the process pool, or the interpreter at exit, waiting forever; that directory must be writable. get_dmarc_reports_from_mbox(), get_dmarc_reports_from_mailbox() and watch_inbox() take a matching parse_timeout keyword argument, and parsedmarc.parallel.parallel_map() takes parse_timeout and timeout_result; a timed-out message is a ParseTimeoutError, a subclass of InvalidDMARCReport. The setting is reloaded on SIGHUP. Compatibility: the default, 0, means no limit, so nothing changes unless it is set.

Bug fixes

• A failure report's RFC 6591 feedback report and sample can no longer be replaced by legacy report text. parse_report_email() also reads the legacy text format, a text/plain part beginning "A message claiming to be from you has failed" that carries the source IP and the sample, and a later such part replaced the message/feedback-report part and the sample already read. A text/plain part inside the sample, which is the spoofed message's own text, could therefore set the report's Source-IP and sample. Once a message/feedback-report part is read, legacy text parts are ignored, and the RFC 6591 part and its sample win over a legacy part that came before it; a legacy part followed by a message/feedback-report part with no sample after it (RFC 6591 §3.1 puts the sample after the report part) is now rejected as InvalidDMARCReport, where a mix of the two was returned. Parts inside the sample, which are the reported message's own attachments, are no longer read as reports either: an SVG, zipped HTML or application/json attachment there made the whole email invalid, and a gzipped aggregate report or SMTP TLS JSON there was returned in place of the failure report. This holds for a second sample part too, which is not read as the sample; a report attached beside the sample, outside it, is still read.
• SMTP TLS reports keep their MX host patterns (#942). RFC 8460 §4.4 keys a policy's MX host patterns mx-host; mx-host-pattern is the name of the value's type ("a JSON array of strings"), and the example in Appendix B gives a plain string. parsedmarc read only mx-host-pattern, so every RFC-conformant report lost the field. mx-host is now read first, as a list of strings or a plain string (which becomes a one-element list); mx-host-pattern is kept as a fallback, and a plain string there is now kept too. The output key mx_host_patterns and its PostgreSQL TEXT[] column are unchanged, and samples/smtp_tls/rfc8460.json now yields ["*.mail.company-y.example"]. Reported by kowax.
• A failure report's duplicate check no longer lets a search error escape the Elasticsearch and OpenSearch outputs. save_failure_report_to_elasticsearch() and save_failure_report_to_opensearch() called the duplicate-check search without the handler the aggregate and SMTP TLS saves have, so a connection or query failure raised the search library's own exception. It is now an ElasticsearchError or OpenSearchError, which process_reports() records as an output error.
• A date string of hundreds of thousands of digits no longer takes seconds to fail. human_timestamp_to_datetime() had no length limit, so a failure report's Arrival-Date, a message's or failure sample's Date header, or any other date it converted went to dateutil whole (a digit string of 262,144 characters took 0.87 s to fail and one of 1,000,000 characters 12.4 s). Only SMTP TLS dates had a limit (64 characters). The function now raises ValueError for a string longer than 128 characters once comments are removed (parsedmarc.utils.MAX_TIMESTAMP_LENGTH, above any RFC 5322 date; a comment has no length limit and is not counted), and each caller turns that, and any other date that does not convert, into the matching error: InvalidFailureReport("Invalid Arrival-Date: ...") for a failure report, InvalidDMARCReport("Invalid Date header: ...") in parse_report_email(), and InvalidFailureReport from the Elasticsearch and OpenSearch failure report saves for the sample's Date. An unparseable Arrival-Date was reported as "Unexpected error" and an unparseable Date header as a plain ParserError.
• A failure sample header with an empty name is left out of the Elasticsearch and OpenSearch failure document. The save lower-cases each sample header name and replaces . with _; a header whose name was empty after that became a field with an empty name, which the cluster rejects. The header is now skipped and the sample's other headers are saved.
• A parsing worker process that dies, or a Ctrl-C, no longer leaves the run hanging or ends it in a traceback. With n_procs above 1 (or parse_timeout set) on a Python whose default start method is fork (Linux before Python 3.14), each worker inherited the SIGTERM and SIGINT handlers the CLI installs, which only note the signal. When a worker died, the process pool stopped the others with SIGTERM and waited for them, and they ignored it, so the run hung instead of failing; under the other start methods the pool's BrokenProcessPool ended the run in a traceback. Workers now take the default action on SIGTERM and ignore SIGINT, which the parent process handles: a Ctrl-C lets the batch in progress finish, as it does with one process. A worker that dies (the out-of-memory killer, a crash) now ends the batch with a logged error: parallel_map() raises ParserError("A parsing worker process died ...") after stopping the other workers. A one-shot run then exits with status 1, from report files, mbox files and mailboxes alike, after saving the reports parsed from report files and mbox files before the error; for a mailbox, earlier batches are saved and the batch in progress stays in the mailbox; in watch mode the IMAP and Maildir watchers log it as a warning and check the mailbox again, and the Microsoft Graph and Gmail watchers exit with status 1. A report file or mbox error (a worker that died, a missing mbox) no longer discards the reports already parsed from earlier files, which with archive_directory set had already been moved out of the input. Without parse_timeout, a library caller that stops reading parallel_map() early, or is interrupted, no longer waits for the jobs in progress: the workers are stopped. The systemd example in the documentation now sets KillMode=mixed, so stopping the service signals only the main process and lets the batch finish within TimeoutStopSec.