Version 0.9.5 - improvements Google News and honor robots.txt

• lang: Add ISO 639-3 language code support for Kurdish (ckb, kmr)
• tests: add robots tests
• feat: added robots.txt check with hook in do_request
• feat: add hooks to get_html
• parse: prioritize datePublished over dateCreated in JSON-LD extraction
• docs: Readme improvements
• feat: add nltk as an optional dependency for leaner deployments
• docs: added additional documentation for GoogleNews and Cloudscraper integration
• rework: type annotations removed deprecated types (python 3.10+)

What's Changed

• lang: Add ISO 639-3 language code support for Kurdish (ckb, kmr) by Muzaffer Cikay in #691
• tests: add robots tests by Andrei in 030e50d
• feat: added robots.txt check with hook in do_request by Andrei in 62dece9
• feat: add hooks to get_html by Andrei in 708cc10
• parse: prioritize datePublished over dateCreated in JSON-LD extraction by Pontus Svensson in cdadb9e
• docs: Readme improvements by Andrei in 18ca21c
• feat: add nltk as an optional dependency for leaner deployments by Andrei in e073459
• docs: added additional documentation for GoogleNews and Cloudscraper integration by Andrei in aceb853
• rework: type annotations removed deprecated types (python 3.10+) by Andrei in bd82a41

Bugs Fixed

• skip null entries in JSON-LD arrays during extraction (fix #692) by ghxm in 77d6ccc
• ArticleException f-string not interpolating status_code (fix #684) by Andrei in 7caa2a5
• accept relative paths for categories (PR #667) by BRNMan in #667
• use w3lib to detect webpage encoding by Andrei in 3bd4f00

New Contributors

• Muzaffer Cikay made their first contribution in #691

Full Changelog: 0.9.4...0.9.5