Skip to content
GP4C - Genetic Programming for Crawling
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Type Name Latest commit message Commit time
Failed to load latest commit information.

GP4C: Genetic Programming for Crawling

The quality of a Web search engine is influenced by several factors, including coverage and the freshness of the content gathered by the web crawler. Focusing particularly on freshness, one key challenge is to estimate the likelihood of a previously crawled webpage being modified. Such estimates are used to define the order in which those pages should be visited, and thus, can be exploited to reduce the cost of monitoring crawled webpages for keeping updated versions.

GP4C is genetic programming based framework to generate score functions that produce accurate rankings of pages regarding their probabilities of having been modified.

This code was used for the experiments in the following publications:

You can’t perform that action at this time.