Skip to content
.NET based webcrawler
C# HTML PowerShell Batchfile
Branch: master
Clone or download
Esben Carlsen
Esben Carlsen Renamed Source -> src
Latest commit ee3e2af Dec 12, 2016
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
src Renamed Source -> src Dec 12, 2016
.gitattributes Moved from Codeplex to github Feb 7, 2016
.gitignore Added TextDocumentProcessorPipelineStep Mar 20, 2016
LICENSE Initial commit Feb 7, 2016
README.md Updated readme Mar 20, 2016

README.md

NCrawler

.NET based webcrawler

Simple and very efficient multithreaded web crawler with pipeline based processing written in C#. Contains HTML, Text, PDF, and IFilter document processors and language detection(Google). Easy to add pipeline steps to extract, use and alter information.

Total rewrite of NCrawler from 2010 using more modern programming. Now on v4

You can’t perform that action at this time.