You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
How to set colly.MaxDepth(1) for non allowed domains and infinite depth for AllowedDomains ?
My aim is to get example.com pages and if this site contains a link to example2.com I want to get it but no more.
The text was updated successfully, but these errors were encountered:
For example, if https://example.com contains <a href="https://domain1.com/page.html"> and <a href="https://domain2.com/page.html"> , I need to visit those domains pages, but I don't want to visit any other pages of domain1 neither domain2.
As a hint, MaxDepth and AllowedDomains won't help you, as they're not flexible enough. I think you'll have to introduce your own logic in OnHTML handler, and call Visit conditionally.
How to set
colly.MaxDepth(1)
for non allowed domains and infinite depth forAllowedDomains
?My aim is to get example.com pages and if this site contains a link to example2.com I want to get it but no more.
The text was updated successfully, but these errors were encountered: