Idea: warn users when trying to use TextResponse functionality with plain Response

[eliasdorneles]

Currently, if we try to use TextResponse functionality like response.text or css()/xpath() methods with a plain Response (e.g. in case of binary content), we get an AttributeError:

>>> response.css
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-1-7d6e256164d4> in <module>()
----> 1 response.css

AttributeError: 'Response' object has no attribute 'css'
>>> response.xpath
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-2-4f61f6e9fc6e> in <module>()
----> 1 response.xpath

AttributeError: 'Response' object has no attribute 'xpath'
>>> response.text
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-3-be6a4a00df5e> in <module>()
----> 1 response.text

AttributeError: 'Response' object has no attribute 'text'

Would it make sense to add a few methods/properties to explain what's going on for new users?

I was thinking instead of AttributeError, a better behavior could be a ValueError with a message giving a bit more context.

So, in plain Response, we could have:

def css(self, *args, **kw):
    raise ValueError('Response content is not text')

def xpath(self, *args, **kw):
    raise ValueError('Response content is not text')

@property
def text(self, *args, **kw):
    raise ValueError('Response content is not text')

This would be nice, because we'd had to explain fewer things when teaching people about responses and also about using .css and .xpath methods.

What do you think?

[kmike]

This change could be backwards incompatible; code may rely on presence of these attributes. I'm using if hasattr(response, 'text') sometimes to check for response type without having to import scrapy.http.TextResponse.

[eliasdorneles]

Hmm, gotcha, you're relying on the AttributeError.
What if we kept the AttributeError exception for the text property, then?

[kmike]

Sounds good, that'd be just providing more explanatory error messages for AttributeErrors.