README.md

Crawly

Overview

Crawly is an application framework for crawling web sites and extracting structured data which can be used for a wide range of useful applications, like data mining, information processing or historical archival.

Requirements

1. Elixir "~> 1.7"
2. Works on Linux, Windows, OS X and BSD

Install

1. Generate an new Elixir project: mix new <project_name> --sup
2. Add Crawly to you mix.exs file

   def deps do
       [{:crawly, "~> 0.1"}]
   end

3. Fetch crawly: mix deps.get

Documentation