"Merge" buffer implementation and seriously consider introducing it into the generic SensorAgent's interface

[BjoernLudwigPTB]

Max has got some implementation ready using a Python's deque und we got some in the examples. We should merge these into one residing in the master.

[BjoernLudwigPTB]

I did some research on efficient implementations and the most straight-forward approach seems to me using Python's built-in collections.deque as @mgrub already proposed and implemented. Although, as I had in the back of my mind, a deque is not the right datatype in general, because it allows to append and to pop elements on both sides, which we actually do not want, as we discussed today, it is in this case most efficiently implemented to do these operations in almost O(1) as they state themselves, whatever this means in detail.

@mgrub well done, to come up with the proposal!

One last remark is on this before we discuss on Friday how tot tackle this issue is taken from the linked page as well:

the name is pronounced “deck” and is short for “double-ended queue"

[BjoernLudwigPTB]

... although on second glace we might be better off for the agents with a queue.Queue which includes locking mechanisms in case several data sources are delivering measurement values at the (almost) same time (to the same buffering/queueing SensorAgent). If we can exclude this possibility in all use cases, we might stick to collections.deque but aiming for higher samping rates, we should consider this and choose carefully.

[BjoernLudwigPTB]

As discussed on Wednesday and on Friday this ideally includes a test implementation with a two node setup, one sender, one receiver, to visualize when frequencies get higher than processing capabilities. That means the receiver has to do some easy processing task as well. The idea is to monitor, when a buffer overflow is probable.

[bangxiangyong]

A suggested processing task is to compute the mean which acts as a filter

[BjoernLudwigPTB]

For the implementation of the interface, especially against the background of the high-speed requirement, we should be aware that we do not necessarily have to process at 10 kHz, but could also use an aliasing agent and low-pass filter to work at lower speeds.

[BjoernLudwigPTB]

In the meantime @mgrub provided a Buffer class in PyDynamic.

[bangxiangyong]

There are two implementations here : deque and dict to be compared -
https://github.com/bangxiangyong/agentMET4FOF/blob/develop/agentMET4FOF/develop/buffer_deque.py

[bangxiangyong]

We can abstract the buffer implementation further -
e.g
self.update_data_memory({'from':self.name,'data':sine_data})
into :
self.update_data_memory(agent_name,sine_data)

and a check for buffer filled instead of :
if len(self.memory[self.name][next(iter(self.memory[self.name]))]) >= self.buffer_size:

we can have -
self.buffer_filled()
preferably, a better way would be using decorator-

@property
def buffer_filled(self):
     ...

[bangxiangyong]

Updated the implementation for self.update_data_memory and self.buffer_filled here:

185927d

[bangxiangyong]

Need to resolve these two tasks-

1. Remove the buffer_deque.py from the develop folder
2. Provide a parameter switch for the underlying mechanism : e.g "deque" , "dict"

[BjoernLudwigPTB]

This is already resolved and thus can be closed. We introduced a neat buffer class AgentBuffer and will continue using and extending it if needed.