Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

New example: simple crawler based on libxml2 #2706

Closed
wants to merge 3 commits into from

Conversation

@jeroen
Copy link
Contributor

commented Jul 3, 2018

A fun example that can push libcurl to the limit 馃槑

@jeroen jeroen force-pushed the jeroen:crawler branch 2 times, most recently from 1691b29 to c35b28c Jul 3, 2018

@jeroen jeroen force-pushed the jeroen:crawler branch from c35b28c to e457458 Jul 3, 2018

@bagder

This comment has been minimized.

Copy link
Member

commented Jul 3, 2018

Cool, thanks! The code style is however not quite following how the other examples look, and I suspect make checksrc will tell you so...

@bagder bagder added the documentation label Jul 3, 2018

@jeroen

This comment has been minimized.

Copy link
Contributor Author

commented Jul 3, 2018

Ah okay I wasn't sure where to find the style guide. Let me fix that.

@jeroen

This comment has been minimized.

Copy link
Contributor Author

commented Jul 3, 2018

OK fixed everything to pass make checksrc now :)

@jeroen jeroen force-pushed the jeroen:crawler branch from 228a617 to f3bc02d Jul 3, 2018

Default to only follow absolute links.
This results in hitting much more different servers

@jeroen jeroen force-pushed the jeroen:crawler branch from af2d0ec to 039a867 Jul 3, 2018

@bagder

This comment has been minimized.

Copy link
Member

commented Jul 5, 2018

Thanks!

@bagder bagder closed this in 74e0bde Jul 5, 2018

@lock lock bot locked as resolved and limited conversation to collaborators Oct 3, 2018

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
2 participants
You can鈥檛 perform that action at this time.