Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
A web automation library for Haskell
branch: master

Fetching latest commit…

Cannot retrieve the latest commit at this time

Failed to load latest commit information.
Network
.gitignore
LICENSE
LICENSE.CURL
README
Setup.hs
config.log
config.status
configure
configure.ac
curl.buildinfo
curl.buildinfo.in
curlc.c
shpider.cabal

README

Shpider.  Copyright 2009-2010 John Morrice.

Shpider is a web automation library for Haskell.   It allows you to quickly write crawlers, and for simple cases ( like following links ) even without reading the page source.
    
It has useful features such as turning relative links from a page into absolute links, options to authorize transactions only on a given domain, and the option to only download html documents.

It also provides a nice syntax for filling out forms.

An example:

runShpider $ do
   download "http://apage.com/"
   theForm : _ <- getFormsByAction "http://anotherpage.com/"
   sendForm $ fillOutForm theForm $ pairs $ do
   "occupation" =: "unemployed Haskell programmer"
   "location" =: "mother's house"

Something went wrong with that request. Please try again.