fix: Fix link filtering in enqueueLinks in AdaptivePlaywrightCrawler #3021
Conversation
janbuchar
commented
Jun 18, 2025
janbuchar
commented
- closes Enqueue strategy check after redirects is not working with adaptive crawler #2525
janbuchar
added
the
t-tooling
github-actions
bot
added
the
tested
There was a problem hiding this comment.
the changes seem sound to me, thank you 👍
I'm starting to get lost in the AdaptiveCrawler design (partially since it's a subclass of PlaywrightCrawler and the HttpCrawler features are glued on top of it). Not sure what the way out is, refactoring this would be tough for sure.
| @@ -590,15 +627,14 @@ export class AdaptivePlaywrightCrawler extends PlaywrightCrawler { | |||
|
|
|||
| return $; | |||
| }, | |||
| async enqueueLinks( | |||
| enqueueLinks: async ( | |||
There was a problem hiding this comment.
nit: this function to arrow function because of the this binding is sneaky and got me confused for a bit (but I don't really see any better way).
fwiw, the new enqueueLinks impl doesn't interact with this at all (could be static / completely separate).
The Python counterpart is a bit DRY-er, I hope we can make this implementation more like it in v4. |
