Building an inverted index using Python
Building an inverted index (or "indexing") is the second step in building a search engine (the first step being crawling). This code builds an inverted index (with varbyte compression), the corresponding lexicon and page table. It uses data (in the form of WET files) from CommonCrawl.
Refer to readme.pdf for more details.