step_walker execution slows bot

[hew-xavier]

Expected Behavior

Bot should walk at speed similar to value as defined within configuration.
Due to bot design (single event loop) it is accepted that actual speed maybe slow due to other blocking tasks.

Actual Behavior

Walker (step_walker.py) blocks for extended period of time (9 to 14 second) for each 'step()'.
Profiler highlights this issue

ncalls tottime percall cumtime percall filename:lineno(function)
3 0.001 0.000 43.265 14.422 step_walker.py:40(step)

Potential issue could be the heartbeat

3 0.000 0.000 40.163 13.388 init.py:938(heartbeat)

Steps to Reproduce

python -m cProfile pokecli.py -cf

Reproduced by three different installations (3 differnt people)

Other Information

OS: raspbian, osx
Git Commit: fc4e802
Python Version: (run 'python -V' and paste it here)

[AcorpBG]

#2811 Seems this is the problem that is causing the issue I linked. Thanks for finding that. I belive there is already discussion and changes planned to the current loop system see this #2889

[hew-xavier]

This is separate issue; profile highlights that writing out resources as a blocking call halts the bot.

pokemongo_bot/__init__.py -> def heartbeat(self): ... self.update_web_location() ...

Above is a blocking call to file system on bot event path. Kills performance

[douglascamata]

This is going away with the incoming web ui with websocket support. When it is finished and working I'll remove all json files being written by the bot.

[jcoinas]

@hew-xavier : while using raspbian did u had any particular configuration/modification that is not the wiki? I reported 2 issues, could u see if had the same while installing on raspbian?

[hew-xavier]

@jcoinas

I manually installed protobuf 'apt-get' (had it for other reasons)
I use 'virtualenv' no issues with dependencies (although numpy had to build locally - no distro)

[hew-xavier]

@douglascamata

I understand the issue is going away; however considering the performance impact 14 seconds to 0.5 seconds I think putting in configuration option would be suitable in short term. Especially in the context of other PR for similar bot issues.

[reconka]

I have a same issue

[ajurna]

fix for this issue is here
#3100

[davidson-git]

Not sure 3100 is a fix for this issue. Response in that thread seems to indicate differently.

[douglascamata]

#3100 doesn't fix it. Handling a Queue that is being spammed with Threads in Python is not cheap at all and introduces a lot of overhead.

[k4n30]

@hew-xavier is this still an issue