Remove hacks and refactor Runtime and RuntimeService/Synchronizer

[awintel]

The Runtime and RuntimeService do currently contain a number of hacks that need to be cleaned up:

• The Runtime has a 'current_ts' attribute. But the Runtime must not be aware of algorithmic time steps. That's up to the Synchronizer to and ProcessModels to implement.
• The Runtime currently misuses the runtime_to_service_cmd channel to send num_steps.
• The RuntimeService should be split into a RuntimeService and a pure Synchronizer. A universal RuntimeService should service the commands received by the Runtime and then delegate these commands to its Synchronizer which can be different for each RuntimeService.
• The RunContinuous RunCondition is not supported.
• The Asynch Synchronizer is currently not thought through. No clear contract between Process and Synchronizer is defined, hence there are numerous ways this can deadlock.

Side question: Should we use asyncio in PyProcModels and PyRuntimeService instead of busy waiting on channels? Will this be more efficient?

[awintel]

I started addressing some of these issues.

It should be possible to create a completely generic PyRuntimeService. The responsibility of this PyRuntimeService is merely to communicate RUN, PAUSE, STOP commands between Runtime, and PyProcModels as well as any SET/GET requests. These are generic communication capabilities that are independent of the specific SyncProtocol.

The current LoihiPyRuntimeService conceptually covers these capabilities and can be refactored to become this generic PyRuntimeService except for its run() method.

Only the part of LoihiPyRuntimeService dealing with the different SPK, MGMT, ... phases are specific to a particular SyncProtocol (i.e. the LoihiSyncProtocol) and should therefore be factored out and provided as an object to a refactored PyRuntimeService.

While beginning to refactor LoihiPyRuntimeService, I began to set up a cleaner state machine structure:

• The PyRuntimeService should have a dictionary mapping commands to function objects, aka 'command handlers'.
• Each command handler (i.e. what to do when RUN, PAUSE, ... is received) is a separate method of the PyRuntimeService.
• The core _run(..) method merely iterates over all channels, waits for a command and when it receives one, get and executes the appropriate command handler from the dictionary.
• The basic PyRuntimeService only has RUN, PAUSE, STOP, GET, SET in its command handler dictionary.
• Similarly the basic AbstractPyProcessModel would implement a similar state machine with command handlers for commands received from the PyRuntimeService. In addition, any subclass of AbstractPyProcessModel like PyLoihiProcessModel could just add more command->handler mappings to the command handler dictionary of the PyRuntimeService for e.g. handling SPK, PRE_MGMT, etc.

[awintel]

This is by far not done, just checked my current experimental state into my private fork to show where I'm going:

https://github.com/awintel/lava/blob/main/src/lava/magma/runtime/runtime.py
https://github.com/awintel/lava/blob/main/src/lava/magma/runtime/runtime_service.py
https://github.com/awintel/lava/blob/main/src/lava/magma/core/model/py/model.py

This has at least enabled me to effectively enable PAUSE, STOP, SET, GET for the AsyncProtocol for a minimalistic use case in this new tutorial:
https://github.com/awintel/lava/blob/main/src/lava/tutorials/end_to_end/tutorial00_tour_through_lava.ipynb

This does currently conflict 99.99% with Harry's changes for performance improvements since we asynchronously started working on the same sections of code in parallel.