README.md

Web Pages Loader

Simple tool which downloads web pages (and its resources) for local using.

Table Of Content

• Web Pages Loader
  • Table Of Content
  • Learn More
  • Installation Guide
    • Package:
    • Library:
  • Usage
  • Troubleshooting

Learn More

The tool downloads web page and saves it to directory (can be configured - see Usage). New directory is created for page resources with name "<page-name>_files".

Moreover, it will show the progress of downloading resources.

Installation Guide

Package:

$ python3 -m pip install --user hexlet-code
$ page-loader <url>

Library:

from page_loader import download

downloaded_page_path = download(url, directory)
print(downloaded_page_path)

Usage

$ page-loader https://example.com
Downloading progress: ◉◉◉◉◉◉◉◉◉◉◉◉◉◉◉◉◉◉◉◉◉◉◉◉◉◉◉◉◉◉◉◉ 100%
<script_path>/example-com.html

$ ls -h
example-com.html	example-com_files	page-loader

Troubleshooting

It is easy to use the verbose argument for troubleshooting ("-v" - WARNING, "-vv" - INFO "-vvv" - DEBUG):

./page-loader https://example.com -vvv
2021-11-22 19:38:38,340 page_loader.content_helper DEBUG Processing the page tag: "link" with resource: "/bitrix/js/main/core/css/core.min.css?16256473552854"
2021-11-22 19:38:38,340 page_loader.content_helper DEBUG The resource is suitable: "link" with resource: "/bitrix/js/main/core/css/core.min.css?16256473552854"

Powered by