Skip to content

Commit

Permalink
Instructions from starting from scratch
Browse files Browse the repository at this point in the history
  • Loading branch information
Kagee committed Jun 7, 2018
1 parent da06279 commit 112dc73
Showing 1 changed file with 8 additions and 0 deletions.
8 changes: 8 additions & 0 deletions README.md
Expand Up @@ -3,3 +3,11 @@
Part of my [MCN](https://github.com/search?q=user%3AKagee+mcn+in%3Aname&type=Repositories) (make clean no)-project.

Scripts for downloading and extracting .no domains from the data of the commoncrawl.org project.

Howto:
* git submodule init
* git submodule update
* sudo apt install python-bs4 parallel
* ./get-indexes.sh
* ./verify-indexes.sh
* ./list_domains.sh

0 comments on commit 112dc73

Please sign in to comment.