Novel Crawler

Tools for crawling the novels, currently support the following websites.

• UU看書
• 鉛筆小說
• 小說狂人

Usage

Setup Virtual Environment

python -m venv .venv
source .venv/bin/activate

Install dependencies

pip install -r requirements.txt

Install current project as well

pip install -e .

Modify the config.json, change the save directory

{
  "store_dir" : "C:\\path\\you\\want\\to\\save\\novels",
  "file_format" : "txt"
}

Execute the crawler

python src/novel_crawler/main.py [novel_url]