What about this?

We implement uri(), which returns the first URI found in a selector or selector list, as a string, resolved to absolute URIs when possible.
We implement uris(), which iterates through all the URIs found in a selector or selector list, as strings, resolved to absolute URIs when possible.
When a selector matches an attribute or element text, it is assumed to be an URI, and is resolved to an absolute URI accordingly when possible. This allows using uri() and uris() to extract resolved URIs from arbitrary element strings and attributes, including custom data-* attributes.
When a selector matches an element, we look for all known HTML URI attributes in the matched element and all child elements.
- If users only want URIs from attributes from the specified element, and not its children, they must perform further filtering with xpath or css beforehand, pointing to the desired attributes.
- If users only want specific URI attributes, they must perform further filtering with xpath or css beforehand.

As for a version that returns Selector instances, what’s the use case? Maybe providing some predefined, importable XPath expressions (e.g. from parsel.xpath import URI_XPATH) would be a better approach.

[From Scrapy] Selector.extract_links and links #26

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions