You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
It seems the event system is not able to handle the execution of multiple spiders.
I have written a console command which triggers multiple runs of various spiders.
For example, there are spiders A and B which are called by Roach::startSpider in the same command.
They get executed and the requests are performed once which is correct, but it seems the event listeners are registered for each spider, so spider B gets logged twice when the LoggerExtension is active:
local.INFO: Run starting
local.INFO: Dispatching request {"uri":"xxxxxxxxxxxxxxxxxx1"}
local.INFO: Run finished
local.INFO: Run starting
local.INFO: Run starting
local.INFO: Dispatching request {"uri":"xxxxxxxxxxxxxxxxxx2"}
local.INFO: Dispatching request {"uri":"xxxxxxxxxxxxxxxxxx2"}
local.INFO: Run finished
local.INFO: Run finished
If there are three spiders running in one script call, event listeners are getting registered thrice and the output is as follows:
local.INFO: Run starting
local.INFO: Dispatching request {"uri":"xxxxxxxxxxxxxxxxxx1"}
local.INFO: Run finished
local.INFO: Run starting
local.INFO: Run starting
local.INFO: Dispatching request {"uri":"xxxxxxxxxxxxxxxxxx2"}
local.INFO: Dispatching request {"uri":"xxxxxxxxxxxxxxxxxx2"}
local.INFO: Run finished
local.INFO: Run finished
local.INFO: Run starting
local.INFO: Run starting
local.INFO: Run starting
local.INFO: Dispatching request {"uri":"xxxxxxxxxxxxxxxxxx3"}
local.INFO: Dispatching request {"uri":"xxxxxxxxxxxxxxxxxx3"}
local.INFO: Dispatching request {"uri":"xxxxxxxxxxxxxxxxxx3"}
local.INFO: Run finished
local.INFO: Run finished
local.INFO: Run finished
For 5 running spiders I had the problem, that each event was listened 5 times (and so on)...
The problem also occurs when the same spider is called multiple times in the same script.
The text was updated successfully, but these errors were encountered:
It seems the event system is not able to handle the execution of multiple spiders.
I have written a console command which triggers multiple runs of various spiders.
For example, there are spiders A and B which are called by
Roach::startSpider
in the same command.They get executed and the requests are performed once which is correct, but it seems the event listeners are registered for each spider, so spider B gets logged twice when the LoggerExtension is active:
If there are three spiders running in one script call, event listeners are getting registered thrice and the output is as follows:
For 5 running spiders I had the problem, that each event was listened 5 times (and so on)...
The problem also occurs when the same spider is called multiple times in the same script.
The text was updated successfully, but these errors were encountered: