You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Sibling selectors retain document-level history across multiple HTML roots. Content after </html>
can now match html + html and html ~ html, including preceding :has()/:contains() predicates.
The minimized malformed-HTML regression is covered by the rule audit and both language suites.
Python and Rust Item.get_all() now consistently respect the field declaration: a single-valued field returns zero or one raw match, while field_all and field_join return every raw match.
Previously, adding an unrelated field could expose additional matches by disabling early exit.
Declare field_all when every match is needed; single-valued schemas retain their early-exit behavior.
The corpus gate now uses the main differential's verdict without discarding empty or whitespace-only
values. Regressions that drop an empty attribute or change value count can no longer pass as
whitespace-only differences. Command-level regression tests cover the verdict, report and exit status.
CSS class and ID selectors now decode literal and hexadecimal escapes, including utility classes
such as .sm\:text-lg. Escape terminators stay part of the identifier instead of becoming descendant
combinators. Non-executable character escapes and ambiguous decoded-backslash escapes are refused throughout
the compiler, including quoted attribute values.
Grouped subfields support immediate-child XPath (./h2/text()), own attributes (@id), own text
(text()), subtree text/attributes (.//text(), .//@id) and context HTML (.). Child paths are
anchored to the actual container; nested descendants cannot satisfy the first child step.
Grouped extraction and schema audits share input normalization: malformed subfield pairs are
rejected instead of unpacking a string or mapping key into unrelated selectors. Extraction also
accepts subfield mappings directly. Page.many/one rejects malformed cardinality tuples and
non-string separators at declaration time instead of ignoring extra arguments.
Page always applies a supplied callable transform, even when that callable has a false boolean
value. First, all and joined fields use the same rule.
Saved-response verification uses independent item shaping, so a cardinality bug cannot change both
the engine result and its reference. It preserves escaped CSS whitespace, handles padded XPath
consistently and checks types inside tuple-valued transforms.
Rule-mutation checks propagate failures while restoring both normal builds, including after failed
setup. Release wheels now exercise the expanded public API suite and run a core smoke check before
optional dependencies are installed.
Developer experience
Documentation starts with reusable Page schemas for ordinary Scrapy callbacks and an API chooser.
URL and grouped-item recipes are self-contained and tested with missing values; migration examples
are executed from the docs. The compatibility guide distinguishes selector support from value parity.
Release checks render Markdown and check repository links against heading IDs; Twine skips that step.
Source audits use the grouped compiler for literal Page.many/one and web-poet Many/One fields,
with context-specific diagnostics. Dynamic-only scans report unknown coverage; --require-complete
can fail CI on unresolved sites or empty scans. Full schema audits still check shared budgets.
Group detection follows field modifier chains; callbacks and unknown factories cannot make an
unresolved group look complete.
Page.extract_response() uses response bytes and encoding without a Scrapy dependency. Page.frost_schema() exposes the same named selector format as the web-poet integration.
Item.validate() checks required processed values, retained match counts and required group
subfields. Reports distinguish absent matches, empty matches, empty processed values and transform
failures, and can record Scrapy stats. FieldProcessingError adds field/selector context while
preserving the original exception. An empty row still counts as a matched group container.
The migration workflow now includes complete-item verification on hashed saved responses, an
inspectable fixture manifest, parity-gated timing reports and a manual Linux benchmark workflow.
Historical benchmark ratios are no longer described as a guaranteed performance floor.
Default pytest discovery is scoped to tests/, so Cargo package snapshots under target/ cannot
be collected as duplicate test modules.
Performance
Immediate first-value text and attribute fields stop retaining later matches even when an all-value
field or group requires a complete scan. Deferred predicates, normalization and raw-HTML captures keep
their existing ordering and retention rules. The A/B harness now includes a mixed-cardinality workload;
measurement details and limitations are in docs/BENCHMARKS.md.
Page and Item share stable field definitions instead of copying parallel metadata lists on each
response. Named item access uses direct lookups, and required-field validation uses set membership.
Extending a page preserves earlier items' schemas. Page.one() shapes only the first matched row;
the engine still scans and collects grouped matches as before.