Webmagician is a configurable web crawler platform。Its philosophy is: Less Coding, More Power.
It will support you to write scripts for achieving the most flexibility and scalability.
At present, my focus is on building a front-end user interface and I'll improve this project in the future. Stay tuned.
UI for this: https://github.com/Jkanon/webmagician-ui