Return raw array data for further processing with numpy

[johannesloibl]

I'm transferring several float arrays with millions of samples each second by using read_by_name with PLCTYPE_ARR_REAL(1000000).
After profiling i noticed the function is spending most of its time in converting the ctypes array to a Python list:
if type(data_type).__name__ == "PyCArrayType": return [i for i in data]
Adding a "raw" argument, that returns the ctypes array directly, and reading it into a numpy array via np.frombuffer afterwards/manually reduces the time spent in this function by ~80% and increases the overall throughput a lot.

[johannesloibl]

Update: I compared the normal implementation with the one i quickly patched to return a normal ctype array. The speedup is up to factor 20.

import numpy as np
from pyads import Connection, constants as pyadsconstants
from mctrl_tb.pkg.pyads import Connection as Connection_patched

if __name__ == '__main__':
    import timeit
    c = Connection("172.22.24.3.1.1", 852)
    c.open()
    c_patched = Connection_patched("172.22.24.3.1.1", 852)
    c_patched.open()

    buffer = pyadsconstants.PLCTYPE_ARR_REAL(1000000)

    ##### Quick'n'Dirty Patch - pyads_ex.py, line 539 #####
    # 
    # if type(data_type).__name__ == "PyCArrayType":
    # if getattr(data_type, "__raw__", False):  
    #     return data
    # else:
    #     return [i for i in data]
    #
    
    buffer.__raw__ = True

    print(timeit.timeit("""data = c.read_by_name("MAIN.fbDataBuffer_array1.aTransferBuffer", buffer)""",
                        globals=globals(),
                        number=20) / 20)

    print(timeit.timeit("""data = np.frombuffer(c_patched.read_by_name("MAIN.fbDataBuffer_array1.aTransferBuffer", buffer), dtype=np.float32)""",
                        globals=globals(),
                        number=20) / 20)

First is 0.166s per call, second is 0.008s per call.

[stlehmann]

@pyhannes That is definetely a massive amount of time it takes to convert to a Python list. Thanks for bringing that up. I think my preferable way to go here is to add a parameter raw to the read/write functions to prevent type conversion if wanted.

[stlehmann]

I added a parameter return_ctypes to read functions to allow omit the conversion to Python data types.