Review unicode references

[Gallaecio]

The documentation still has some references to Python 2’s unicode. We need to review them, and update them as needed (usually changing them to str or “string”).

[divyabhushan]

Hi @Gallaecio, Can I work on this issue, please?

[Gallaecio]

All yours!

[divyabhushan]

@Gallaecio
So to get started, I could find one python code file under docs - scrapy/docs/utils/linkfix.py with python 2's unicode style.
So I would use the function to convert it to a string type which I have already tested and verified in my environment.
Is this the expected solution? Kindly verify.
Thanks.

[elacuesta]

@divyabhushan I believe what @Gallaecio means is refactoring the docs themselves. For instance, take a look at the description for Request.body:

body (str or unicode) – the request body. If a unicode is passed, then it’s encoded to str using the encoding passed (which defaults to "utf-8"). If body is not given, an empty string is stored. Regardless of the type of this argument, the final value stored will be a str (never unicode or None).

which should be something like the following, now that we only support Python 3:

body (bytes or str) – the request body. If a str is passed, then it's encoded to bytes using the passed encoding (which defaults to "utf-8"). If body is not given, an empty bytes object is stored. Regardless of the type of this argument, the final stored value will be an instance of bytes (never str or None).

[Gallaecio]

Request.body is always bytes, but yes, that’s what I meant.

The changes needed on the request-response.rst file can be copied from #4090 (in case this pull request is merged first).

[elacuesta]

Request.body is always bytes, but yes, that’s what I meant.

For the record, you can in fact instantiate a Request with a str body, which gets encoded:

In [1]: from scrapy import Request

In [2]: r = Request("http://example.org", body="hello!")

In [3]: type(r.body)
Out[3]: bytes

In [4]: r.body
Out[4]: b'hello!'

[divyabhushan]

Thanks, @Gallaecio @elacuesta I make the necessary changes.