WebCrawler

This project is currently in early development, and likely not of any use to anyone yet!

Plans are to use boost::asio for the http(s) client, Gumbo for HTML parsing, sqlite for data storage, and Lua for exstensions.

This project includes code from Tomaka17's luawrapper Copyright (c) 2013, Pierre Krieger All rights reserved. https://github.com/Tomaka17/luawrapper

TODO:

Development
Code cleanup
Possibly support other databases

Dependencies:

You can probably install all of these through your package manager:
Boost (http://boost.org)
SQLite3 (http://sqlite.org)
Lua 5.2 (http://lua.org)
OpenSSL (http://www.openssl.org)
Gumbo (https://github.com/google/gumbo-parser)