diff --git a/README.md b/README.md index b651604..7fb6bca 100644 --- a/README.md +++ b/README.md @@ -8,19 +8,136 @@ Library for scraping free proxies lists ### Setup -Proxy-scrapper library is built on top of [HTTPlug](http://httplug.io/) and requires a compatible HTTP client. Available clients are listed on Packagist: https://packagist.org/providers/php-http/client-implementation. To use the library you have to install any of them, e.g.: +Proxy-scraper library is built on top of [HTTPlug](http://httplug.io/) and requires a compatible HTTP client. Available clients are listed on Packagist: https://packagist.org/providers/php-http/client-implementation. To use the library you have to install any of them, e.g.: ```bash composer require php-http/guzzle6-adapter ``` -Then install proxy-scrapper library itself: +Then install proxy-scraper library itself: ```bash composer require vantoozz/proxy-scraper ``` +### Usage -### Tests +##### Single scraper +```php +get() as $proxy) { + echo (string)$proxy . "\n"; +} +``` + +##### Composite scraper +You can easily get data from many scrapers at once +```php +addScraper(new Scrapers\FreeProxyListScraper($httpClient)); +$compositeScraper->addScraper(new Scrapers\MultiproxyScraper($httpClient)); +$compositeScraper->addScraper(new Scrapers\ProxyDbScraper($httpClient)); +$compositeScraper->addScraper(new Scrapers\SocksProxyScraper($httpClient)); +$compositeScraper->addScraper(new Scrapers\SpysMeScraper($httpClient)); +$compositeScraper->addScraper(new Scrapers\UsProxyScraper($httpClient)); + +foreach ($compositeScraper->get() as $proxy) { + echo (string)$proxy . "\n"; +} +``` + +##### Error handling +Sometimes things goes wrong. This example shows how to handle errors while getting data from many scrapers: +```php +addScraper(new class implements Scrapers\ScraperInterface +{ + public function get(): \Generator + { + throw new ScraperException('some error'); + } +}); + +// No exceptions +$compositeScraper->addScraper(new class implements Scrapers\ScraperInterface +{ + public function get(): \Generator + { + yield new Proxy(new Ipv4('192.168.0.1'), new Port(8888)); + } +}); + +// Set exception handler +$compositeScraper->handleScraperExceptionWith(function (ScraperException $e) { + echo 'An error occurs: ' . $e->getMessage() . "\n"; +}); + +foreach ($compositeScraper->get() as $proxy) { + echo (string)$proxy . "\n"; +} +``` +Will output +``` +An error occurs: some error +192.168.0.1:8888 +``` + +_Note. Examples uses Guzzle as HTTP client._ + + +### Testing ##### Unit tests ```bash