A fast and light Python implementation of the Streaming Protocol for the Exchanged of Astronomical Data (SPEAD)
License
ska-sa/PySPEAD
master
Name already in use
Code
-
Clone
Use Git or checkout with SVN using the web URL.
Work fast with our official CLI. Learn more about the CLI.
- Open with GitHub Desktop
- Download ZIP
Sign In Required
Please sign in to use Codespaces.
Launching GitHub Desktop
If nothing happens, download GitHub Desktop and try again.
Launching GitHub Desktop
If nothing happens, download GitHub Desktop and try again.
Launching Xcode
If nothing happens, download Xcode and try again.
Launching Visual Studio Code
Your codespace will open once ready.
There was a problem preparing your codespace, please try again.
Latest commit
Git stats
Files
PySPEAD This is a fast and light Python implementation of the Streaming Protocol for the Exchange of Astronomical Data New in 0.5.1: - Sanitised variable length string handling so as not to return array containing single string New in 0.5: - Added experimental string handling using mkfmt(('s',8)) ToDo: - Unpacking is now pretty quick, but for arrays, could we get one more speed-up by unpacking data directly into a numpy array, rather than to a tuple? - Right now, packets are pulled straight from the socket and put into a packet ringbuffer. The packet ringbuffer then feeds a callback that consumes the packets and creates a heap. While heap creation from packets is performed in C, a Python callback is currently used to hand the packets to SpeadHeap. (a) We could make heap assembly happen automatically, so that the Python callback is fed heaps directly. (b) While implementing this, we should also incorportate packet sorting by heap_cnt so that packets can arrive out of order, but heaps are assembled in order to the extent allowed by the available buffer. - Confusingly, there are currently two kinds of "heap" in spead.py: one is a SpeadHeap that is generated by a receiver, and one is a dictionary that is used by a transmitter. We might move the heap => packets pipeline into C as well, and make all heaps of the SpeadHeap type. - Packet sorting is done as each packet is inserted, but it could be done faster by buffering them and then quicksorting - _spead.pack() is currently the limiting factor on transmit. Can it be sped up? - tests/test_spead.py uses lots of explicit constants. We should migrate to using the named constants provided in the module. Known Issues: - Occasionally MacOSX hangs forever in unittest test_spead.py TestTransportUDPrx.test_get_packets_term. This seems to be b/c no data is being received. Hanging seems to be more frequent when buffer_socket.cpp has the DEBUG flag set. Linux doesn't seem to have the same problem. Cause unknown.
About
A fast and light Python implementation of the Streaming Protocol for the Exchanged of Astronomical Data (SPEAD)