Potential dependency conflicts between pywb and chardet

[NeolithEra]

Hi, as shown in the following full dependency graph of pywb, pywb requires chardet (the latest version), while the installed version of requests(2.22.0) requires chardet>=3.0.2,<3.1.0.

According to Pip's “first found wins” installation strategy, chardet 3.0.4 is the actually installed version.

Although the first found package version chardet 3.0.4 just satisfies the later dependency constraint （chardet>=3.0.2,<3.1.0), it will lead to a build failure once developers release a newer version of chardet in the near future, which is greater than 3.1.0.

Dependency tree--------

pywb(version range:)
| +-six(version range:)
| +-chardet(version range:)
| +-requests(version range:)
| | +-chardet(version range:>=3.0.2,<3.1.0)
| | +-idna(version range:>=2.5,<2.9)
| | +-urllib3(version range:>=1.21.1,<1.26)
| | +-certifi(version range:>=2017.4.17)
| +-redis(version range:)
| +-jinja2(version range:)
| +-surt(version range:>=0.3.0)
| | +-six(version range:)
| | +-tldextract(version range:>=2.0)
| | | +-setuptools(version range:)
| | | +-idna(version range:)
| | | +-requests(version range:>=2.1.0)
| | | | +-chardet(version range:>=3.0.2,<3.1.0)
| | | | +-idna(version range:>=2.5,<2.9)
| | | | +-urllib3(version range:>=1.21.1,<1.26)
| | | | +-certifi(version range:>=2017.4.17)
| | | +-requests-file(version range:>=1.4)
| +-brotlipy(version range:)
| +-pyyaml(version range:)
| +-watchdog(version range:)
| | +-PyYAML(version range:>=3.10)
| | +-argh(version range:>=0.24.1)
| | +-pathtools(version range:>=0.1.1)
| | +-pyobjc-framework-Cocoa(version range:>=4.2.2sys)
| | +-pyobjc-framework-FSEvents(version range:>=4.2.2sys)
| +-webencodings(version range:)

Thanks for your attention.
Best,
Neolith

[NeolithEra]

Solution

1. Fix your direct dependencies to be chardet>=3.0.2,<3.1.0 and requests==2.22.0, to remove this conflict.
   I have checked this revision will not affect your downstream projects now.

2. Remove your direct dependency chardet, and use the library chardet(>=3.0.2,<3.1.0) transitively introduced by requests.

Personally, I prefer solution 2.
What's your opinion?

@ikreymer May I pull a request to solve this issue?

[ikreymer]

Hey, sure, it looks like we're not using chardet directly at this point, so it can be removed. Thanks.

[ikreymer]

Fixed in 2.3.2