Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add support for :has(<relative operator>) #115

Merged
merged 18 commits into from
Aug 18, 2021

Conversation

annbgn
Copy link
Contributor

@annbgn annbgn commented Jun 21, 2021

todo: uncomment tests and watch them fail, then fix

tests/test_cssselect.py Outdated Show resolved Hide resolved
cssselect/parser.py Outdated Show resolved Hide resolved
@elacuesta elacuesta marked this pull request as ready for review June 22, 2021 20:28
@elacuesta elacuesta marked this pull request as draft June 22, 2021 20:28
cssselect/xpath.py Outdated Show resolved Hide resolved
Co-authored-by: Eugenio Lacuesta <1731933+elacuesta@users.noreply.github.com>
@codecov
Copy link

codecov bot commented Jun 30, 2021

Codecov Report

Merging #115 (b4cbd4e) into master (9edc6c3) will increase coverage by 0.22%.
The diff coverage is 97.10%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #115      +/-   ##
==========================================
+ Coverage   95.39%   95.61%   +0.22%     
==========================================
  Files           3        3              
  Lines         803      889      +86     
  Branches      139      152      +13     
==========================================
+ Hits          766      850      +84     
  Misses         20       20              
- Partials       17       19       +2     
Impacted Files Coverage Δ
cssselect/xpath.py 94.65% <96.29%> (-0.12%) ⬇️
cssselect/parser.py 96.16% <97.61%> (+0.42%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9edc6c3...b4cbd4e. Read the comment docs.

@annbgn annbgn marked this pull request as ready for review July 6, 2021 22:17
tests/test_cssselect.py Outdated Show resolved Hide resolved
tests/test_cssselect.py Outdated Show resolved Hide resolved
Copy link
Member

@wRAR wRAR left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great!

@elacuesta
Copy link
Member

elacuesta commented Jul 21, 2021

Looks good! Do you think it would be possible to add some of the tests from #96?

Edit: In particular, I'm interested in this test, which doesn't work under the current version (b64eacf):

Python 3.8.2 (default, Apr 18 2020, 17:39:30) 
[GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from cssselect import parse
>>> parse("div:has(div.foo)")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/eugenio/zyte/cssselect/cssselect/parser.py", line 475, in parse
    return list(parse_selector_group(stream))
  File "/home/eugenio/zyte/cssselect/cssselect/parser.py", line 488, in parse_selector_group
    yield Selector(*parse_selector(stream))
  File "/home/eugenio/zyte/cssselect/cssselect/parser.py", line 497, in parse_selector
    result, pseudo_element = parse_simple_selector(stream)
  File "/home/eugenio/zyte/cssselect/cssselect/parser.py", line 603, in parse_simple_selector
    arguments = parse_relative_selector(stream)
  File "/home/eugenio/zyte/cssselect/cssselect/parser.py", line 649, in parse_relative_selector
    raise SelectorSyntaxError(
cssselect.parser.SelectorSyntaxError: Expected an argument, got <DELIM '.' at 11>
>>>

tests/test_cssselect.py Outdated Show resolved Hide resolved
…_translation_change

Revert xpath translation change to make it consistent
@elacuesta elacuesta merged commit f564dfd into scrapy:master Aug 18, 2021
@errorcode7
Copy link

cssselect 1.1.0 parsel 1.6.0 Scrapy 2.6.1

`In [58]: response.css("a:has(strong)")

TypeError Traceback (most recent call last)
in
----> 1 response.css("a:has(strong)")

/usr/local/lib/python3.7/dist-packages/scrapy/http/response/text.py in css(self, query)
129
130 def css(self, query):
--> 131 return self.selector.css(query)
132
133 def follow(self, url, callback=None, method='GET', headers=None, body=None,

/usr/local/lib/python3.7/dist-packages/parsel/selector.py in css(self, query)
280 .. _cssselect: https://pypi.python.org/pypi/cssselect/
281 """
--> 282 return self.xpath(self._css2xpath(query))
283
284 def _css2xpath(self, query):

/usr/local/lib/python3.7/dist-packages/parsel/selector.py in _css2xpath(self, query)
283
284 def _css2xpath(self, query):
--> 285 return self._csstranslator.css_to_xpath(query)
286
287 def re(self, regex, replace_entities=True):

/usr/local/lib/python3.7/dist-packages/parsel/csstranslator.py in css_to_xpath(self, css, prefix)
105 @lru_cache(maxsize=256)
106 def css_to_xpath(self, css, prefix='descendant-or-self::'):
--> 107 return super(HTMLTranslator, self).css_to_xpath(css, prefix)
108
109

/usr/local/lib/python3.7/dist-packages/cssselect/xpath.py in css_to_xpath(self, css, prefix)
198 return " | ".join(
199 self.selector_to_xpath(selector, prefix, translate_pseudo_elements=True)
--> 200 for selector in parse(css)
201 )
202

/usr/local/lib/python3.7/dist-packages/cssselect/xpath.py in (.0)
198 return " | ".join(
199 self.selector_to_xpath(selector, prefix, translate_pseudo_elements=True)
--> 200 for selector in parse(css)
201 )
202

/usr/local/lib/python3.7/dist-packages/cssselect/xpath.py in selector_to_xpath(self, selector, prefix, translate_pseudo_elements)
227 if not tree:
228 raise TypeError("Expected a parsed selector, got %r" % (selector,))
--> 229 xpath = self.xpath(tree)
230 assert isinstance(xpath, self.xpathexpr_cls) # help debug a missing 'return'
231 if translate_pseudo_elements and selector.pseudo_element:

/usr/local/lib/python3.7/dist-packages/cssselect/xpath.py in xpath(self, parsed_selector)
265 if method is None:
266 raise ExpressionError("%s is not supported." % type_name)
--> 267 return method(parsed_selector)
268
269 # Dispatched by parsed object type

/usr/local/lib/python3.7/dist-packages/cssselect/xpath.py in xpath_relation(self, relation)
293 "xpath_relation_%s_combinator" % self.combinator_mapping[combinator.value],
294 )
--> 295 return method(xpath, right)
296
297 def xpath_matching(self, matching):

/usr/local/lib/python3.7/dist-packages/cssselect/xpath.py in xpath_relation_descendant_combinator(self, left, right)
407 def xpath_relation_descendant_combinator(self, left, right):
408 """right is a child, grand-child or further descendant of left; select left"""
--> 409 return left.join("[descendant::", right, closing_combiner="]", has_inner_condition=True)
410
411 def xpath_relation_child_combinator(self, left, right):

TypeError: join() got an unexpected keyword argument 'closing_combiner'
hello ,cant use:hasin scrapy? how can i use pseudo-classhas` in scrapy css()?

@errorcode7
Copy link

i have installed cssselect with pip install cssselect-master.zip ,download from github.

@Gallaecio
Copy link
Member

Please, open a separate issue with a minimal reproducible example instead of commenting on an already closed pull request. You can link this pull request from your new issue for reference.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants