[BUG] On macOS additional dependencies are required after installing bs4

[101Dude]

Describe the bug

Not a bug perse, and maybe extra notes could be added under bs4 for macOS users?

On macOS I had to install two dependencies for bs4 to work properly:

pip install html5lib
pip install lxml

ERROR: 33_NASA - SC Records (https://www.teamunify.com/team/isnasa/page/team-info/short-course-yards#2)

bs4.FeatureNotFound: Couldn't find a tree builder with the features you requested: html5lib. Do you need to install a parser library?

To Reproduce

Expected behavior

Screen scrape/screenshots

Version info

2023-09-02 02:22:19,394 cli[140704747579136] INFO: webchanges: 3.14 Copyright 2020- Mike Borsetti
2023-09-02 02:22:19,394 cli[140704747579136] INFO: CPython: 3.11.0 ('main', 'Sep  2 2023 02:06:10') Clang 14.0.3 (clang-1403.0.22.14.1)
2023-09-02 02:22:19,405 cli[140704747579136] INFO: System: macOS-13.5.1-x86_64-i386-64bit

Additional context
I think we discussed this a few years back, you worked on it and I failed to respond.... I apologize if that's the case.

After installing those additional dependencies my rules ran without issue.

2023-09-02 02:22:30,062 storage[140704747579136] INFO: Closed main sqlite3 database file /Users/john/Library/Caches/webchanges/cache.db
2023-09-02 02:22:30,062 command[140704747579136] INFO: Exiting with exit code 0

I followed advice from this page

https://stackoverflow.com/questions/24398302/bs4-featurenotfound-couldnt-find-a-tree-builder-with-the-features-you-requeste

[mborsetti]

Hi @101Dude,

Thank you very much for taking the time to report this.

I tried to track the issue down, and confirmed that lxml should be installed with webchanges since it's listed as a dependency (try pip show webchanges). However, as per html5lib, that's required only if you explicitly select it as a parser in your job files, e.g.

filters
  - html2text:
      method: bs4
      parser: html5lib

and while the documentation states here that

html5librequires having the html5lib Python package already installed,

the way it's currently documented is very poor (the comment is very easy to miss) and html5lib does not appear anywhere in the dependencies help pages.

This will be fixed thanks to you.

Appreciated!

[101Dude]

@mborsetti
my pleasure... I appreciate your response, and the manner in which you responded.

This is a very well maintained repository and I find webchanges extremely useful.