A simple Crawler for getting Google results.
This component can be used to retrieve the 100 first results for a search term.
Since google detects a crawler and blocks the IP when several requests are made, this component is prepared to use some online proxy services, such as hide.me.
Install the latest version with
$ composer require cviniciussdias/google-crawler
CViniciusSDias\GoogleCrawler\Crawler::__construct(
SearchTermInterface $searchTerm, GoogleProxyInterface $proxy = null,
string $googleDomain = 'google.com', string $countryCode = ''
)
- $searchTerm Term that will be searched on Google
- $proxy Online proxy service that will be used to access Google [optional]
- $googleDomain Your country specific google domain, like google.de, google.com.br, etc. [optional]
- $countryCode Country code that will be added to
gl
parameter on Google's url, indicating the location of the search. E.g. 'BR', 'US', 'DE' [optional]
<?php
use CViniciusSDias\GoogleCrawler\{
Crawler, SearchTerm
};
$searchTerm = new SearchTerm('Test');
$crawler = new Crawler($searchTerm); // or new Crawler($searchTerm, new NoProxy());
$results = $crawler->getResults();
<?php
use CViniciusSDias\GoogleCrawler\{
Crawler, SearchTerm, Proxy\CommonProxy
};
$searchTerm = new SearchTerm('Test');
$commonProxy = new CommonProxy('https://us.hideproxy.me/includes/process.php?action=update');
$crawler = new Crawler($searchTerm, $commonProxy);
$results = $crawler->getResults();
To know more details about which proxies are currently
supported, see the tests/Functional/CrawlerTest.php
file.
There you'll see all the available proxies.
- This component works with PHP 7.1 or above
- This component requires the extension php-ds to be installed
Vinicius Dias (ZCE) - carlosv775@gmail.com - https://github.com/CViniciusSDias/ - http://www.zend.com/en/yellow-pages/ZEND030134
This component is licensed under the GPL v3.0 License - see the LICENSE
file for details