Migrate to the next version of Python requests when released

[potiuk]

Hello Maintainers, 

I am a PMC member of Apache Airflow, and I wanted to give you a bit of heads-up with rather important migration to the upcoming version of requests library in your Python release. 

Since you are using requests library in your project, you are affected.

As discussed at length in https://issues.apache.org/jira/browse/LEGAL-572  we found out that the chardet library used by requests library was a mandatory dependency to requests and since it has LGPL licence, we should not release any Apache Software with it. 

Since then (and since in Airflow we rely on requests heavily) we have been working with the requests maintainers and "charset-normalizer" maintainer to make it possible to replace chardet with MIT-licensed charset-normalizer instead so that requests library can be used in Python releases by Apache projects.

This was a bumpy road but finally the PR by @ashb has been merged: https://github.com/psf/requests/pull/5797 and we hope soon a new version of requests library will be released. 

This is just a heads-up. I will let you know when it is released, but I have a kind requests as well - I might ask the maintainers to release a release candidate of requests and maybe you could help to test it before it is released, that would be some re-assurance for the maintainers of requests who are very concerned about stability of their releases.

Let me know if you need any more information and whether you would like to help in testing the candidate when it is out.

[eolivelli]

Thank you @potiuk

@nicoloboschi @zymap would you be interested in following up on this task?

[zymap]

Hi @eolivelli, I checkout the python code. Looks like we only have a python client for stream storage, and I don't find any file is import the requests, maybe we can remove it from the setup.py?

[potiuk]

Requests 2.26.0 released without the mandatory chardet dependency:

https://pypi.org/project/requests/2.26.0/

Instead of chardet, use the MIT-licensed charset_normalizer for Python3 to remove license ambiguity for projects bundling requests. If chardet is already installed on your machine it will be used instead of charset_normalizer to keep backwards compatibility. (#5797)

You can also install chardet while installing requests by specifying [use_chardet_on_py3] extra as follows:

shell pip install "requests[use_chardet_on_py3]" 

Python2 still depends upon the chardet module.

[eolivelli]

@zymap you are right.
Can you try your proposal? CI will validate it.

IIRC we are also using it on the bk-merge script as well, but in that case the user has to manually install it.

[zymap]

Resolved by #2754, close this issue