Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Event listeners get registered multiple times #68

Closed
dodomedia opened this issue Oct 4, 2022 · 1 comment
Closed

Event listeners get registered multiple times #68

dodomedia opened this issue Oct 4, 2022 · 1 comment

Comments

@dodomedia
Copy link

It seems the event system is not able to handle the execution of multiple spiders.
I have written a console command which triggers multiple runs of various spiders.

For example, there are spiders A and B which are called by Roach::startSpider in the same command.

They get executed and the requests are performed once which is correct, but it seems the event listeners are registered for each spider, so spider B gets logged twice when the LoggerExtension is active:

local.INFO: Run starting  
local.INFO: Dispatching request {"uri":"xxxxxxxxxxxxxxxxxx1"} 
local.INFO: Run finished  
local.INFO: Run starting  
local.INFO: Run starting  
local.INFO: Dispatching request {"uri":"xxxxxxxxxxxxxxxxxx2"} 
local.INFO: Dispatching request {"uri":"xxxxxxxxxxxxxxxxxx2"} 
local.INFO: Run finished  
local.INFO: Run finished  

If there are three spiders running in one script call, event listeners are getting registered thrice and the output is as follows:

local.INFO: Run starting  
local.INFO: Dispatching request {"uri":"xxxxxxxxxxxxxxxxxx1"} 
local.INFO: Run finished  
local.INFO: Run starting  
local.INFO: Run starting  
local.INFO: Dispatching request {"uri":"xxxxxxxxxxxxxxxxxx2"} 
local.INFO: Dispatching request {"uri":"xxxxxxxxxxxxxxxxxx2"} 
local.INFO: Run finished  
local.INFO: Run finished  
local.INFO: Run starting  
local.INFO: Run starting  
local.INFO: Run starting  
local.INFO: Dispatching request {"uri":"xxxxxxxxxxxxxxxxxx3"} 
local.INFO: Dispatching request {"uri":"xxxxxxxxxxxxxxxxxx3"} 
local.INFO: Dispatching request {"uri":"xxxxxxxxxxxxxxxxxx3"} 
local.INFO: Run finished  
local.INFO: Run finished 
local.INFO: Run finished 

For 5 running spiders I had the problem, that each event was listened 5 times (and so on)...
The problem also occurs when the same spider is called multiple times in the same script.

@ksassnowski
Copy link
Contributor

Closing this in favor of #36

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants