Scraping Framework in C#
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
Samples/WolframSample
Scrappy
ScrappyTest
WebProxy
.editorconfig
.gitignore
LICENSE
README.md
Scrappy - small.png
Scrappy.png
Scrappy.sln
Scrappy.svg

README.md

Scrappy Logo Scrappy

If you want to automate tasks in the web, this is your go to library. Navigate through websites with ease, fill out forms comfortably from your couch, let Scrappy do all the dirty work and get you what you want.

Features:

  • Form-Handling
  • Link-Handling
  • Header-Mimicking
  • HTML-Parsing and CSS-Selectors using CsQuery

####Usage####

How to fetch your IP from whatsmyip.org:

var browser = new Browser();

var page = await browser.Open("http://www.whatsmyip.org/");

var myIp = page.Select("#ip").Text();

####Samples####

Samples are probably outdated as the web is quite volatile!

  • WolframCalculator: Commandline tool which sends an expression to wolframalpha.com and displays the plain-text result to the user.
  • Tests: Check out the tests for Scrappy to see what can be done with it.

####Planned Features####

  • Resource downloader and management
  • Reactive extensions
  • Javascript interpreter

####Continous Integration####

scrappy MyGet Build Status

Get the latest build from this NuGet feed: https://www.myget.org/F/scrappy/api/v2/