Alternative to Injector/Extractor Processes

[gkarray]

Issue Number:

Objective of pull request: Give an alternative (better) implementation of Injector and Extractor main functionality without wrapping it into a Lava Process.

Pull request checklist

Your PR fulfills the following requirements:

• Issue created that explains the change and why it's needed
• Tests are part of the PR (for bug fixes / features)
• Docs reviewed and added / updated if needed (for bug fixes / features)
• PR conforms to Coding Conventions
• PR applys BSD 3-clause or LGPL2.1+ Licenses to all code files
• Lint (flakeheaven lint src/lava tests/) and (bandit -r src/lava/.) pass locally
• Build tests (pytest) passes locally

Pull request type

Please check your PR type:

• Bugfix
• Feature
• Code style update (formatting, renaming)
• Refactoring (no functional changes, no api changes)
• Build related changes
• Documentation changes
• Other (please describe): Alternative implementation of a feature

What is the current behavior?

• Injector and Extractor Processes are used to communicate data in and out of Lava.

What is the new behavior?

• Communication between Lava and the outside would be done differently. The API would look like this:

relay = Relay(shape=(1,)) # Process with an InPort (in_1) and an OutPort (out_1)
...
relay.run(condition=RunStep(num_steps=1, blocking=False), ...)
...
send_data = np.array([5])
relay.in_1.external_pipe_csv_send_port.send(send_data)
...
recv_data = relay.out_1.external_pipe_csv_recv_port.recv()

• Advantages of this alternative:
  • Uses the same MultiProcessing instance as the Runtime (addressing pending problem highlighted in IO bridge Processes #686).
  • Communication to/from the target Lava ports is direct, while with the Injector/Extractor, it had to go through an intermediary channel.

Does this introduce a breaking change?

• Yes
• No

• If we delete the Injector and Extractor Processes in this PR, any code using them has to be adapted to use the alternative instead.

[bamsumit]

This looks nice guys. Does it mean that we will get rid of injector and extractor processes as it is now? I would not delete them right now and maybe provide deprecation warning that it will be removed in lava vX.Y.Z. This will allow all the subsequent apps to make the transition.

[PhilippPlank]

This looks nice guys. Does it mean that we will get rid of injector and extractor processes as it is now? I would not delete them right now and maybe provide deprecation warning that it will be removed in lava vX.Y.Z. This will allow all the subsequent apps to make the transition.

We have rewritten the Injector/Extractor classes, so they are fully backwards compatible with the new change. They do offer some functionality which might be handy, e.g., what happens to the data when the buffer is full (drop, stall, etc.). So one could continue using this processes in the future, but it is not needed anymore. I am open to deprecate them in the future, but can also see value in keeping them.

[bamsumit]

This looks nice guys. Does it mean that we will get rid of injector and extractor processes as it is now? I would not delete them right now and maybe provide deprecation warning that it will be removed in lava vX.Y.Z. This will allow all the subsequent apps to make the transition.

We have rewritten the Injector/Extractor classes, so they are fully backwards compatible with the new change. They do offer some functionality which might be handy, e.g., what happens to the data when the buffer is full (drop, stall, etc.). So one could continue using this processes in the future, but it is not needed anymore. I am open to deprecate them in the future, but can also see value in keeping them.

Great. In that case we do not need to deprecate injector and extractor.

[weidel-p]

I have nothing to add to this discussion. The code looks fine to me, great work!