dyer is designed for reliable, flexible and fast web-crawling, providing some high-level, comprehensive features without compromising speed.

dyer provides some high-level features:

• asynchronous, concurrent streaming and I/O, make the best of thread pool, network, and system resource.
• Event-driven, once you set the initials and recursive generator, dyer will handle the rest of it.
• User-friendly and flexible, dyer offers high-level, flexible, easy to use wrappers and APIs what does a lot for you.

Get started by installing dyer-cli and looking over the examples.