Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.Sign up
Make it possible to debounce requests #305
I just read Ben Alman on jQuery's Throttle and Debounce and wondered: can DebounceRequest be added to Scrapy?
The use case: recently I've been scraping a website where I wanted to gather Facebook Likes per URL. Now that comes cheap using Facebook API, particularly the Facebook Query Language. The problem is that FB will eventually stop answering my request, on a undocumented rate limit. But what if I could define a way for those requests to be joined and called only after a certain while, asking for all the parameters in a user-defined way? I'd expect the callback to be called just once, too (or maybe several times but with the full response).
Let's say my
I think this should already be possible in a middleware.