Repository navigation
Frostwork 0.1.5
Python API and tooling
-
Selectors can declare their syntax instead of being routed by prefix: pass
syntax="css"or
syntax="xpath"toextract,extract_grouped,checkand everyPagefield,manyandone
(where it covers the container and its sub-fields); in Rust, pass aQuerywherever a selector
string is accepted. Without a declaration the prefix rule is unchanged —/,./and
normalize-space(are XPath, everything else is CSS.Relative XPath is what needs it. Declared as XPath,
h1ortd/text()is refused as unsupported,
like every relative path; read as CSS,h1matches every<h1>instead, which is a wrong value
rather than a refusal.frostwork-auditnow audits each call site in the language of the call it
came from, so a.css()site is explained as CSS and an.xpath()site as XPath. -
Charset labels accept the Python codec names of WHATWG encodings —
cp932,euc_jp,big5hkscs,
cp949,utf-16-leand the rest — which is whatw3lib.encoding.resolve_encoding, and therefore
Scrapy'sresponse.encoding, returns from aContent-Typeheader. Such a label was previously
ignored and the document sniffed instead, so a Shift_JIS page passed ascp932could decode wrongly.
Every Python entry point resolves labels the same way now,Planand the web-poet integration
included. A label naming a real encoding WHATWG has no equivalent for (utf-7,cp437) is still
ignored, the way a browser ignores it, and sniffing continues.