Home
If you found this sort of project useful, come and checkout the Pothos project, where we continued to develop features such as message passing, python blocks, network distributed topologies, excellent APIs, minimal boilerplate and more...
Attention: This is the old GrExtras project page. The supporting branch for this page is called grextras_v3.6 Due to compatibility concerns, this branch will not be compatible with the GNU Radio 3.7 API.
The GrExtras project provides a set of extensions and features to the GNU Radio Project. GrExtras can be compiled against and used with any recent gnuradio installation. See the README for build and installation instructions, or dive right in with the Blocks-Coding-Guide.
Table of Contents
The typical GNU Radio coding model is
- write signal processing in C++
- connect the flow graph in python
This is smart. Python is not fast, but efficient to code. C++ is faster, but more effort is required. But why not code the signal processing blocks themselves in python?
- Quickly develop and debug your algorithm, before implementing it in C++.
- If the processing can be handled in python, why not leave it in python?
- Do work with external libraries with python bindings, like Numpy.
So, writing signal processing in python is a time saver at the very least, and doing so isn't necessarily going to be a performance issue.
Using this feature, users can get access to all available API functionality in a typical C++ block. Users can:
- Write a signal processing work() function that will be called by the GNU Radio scheduler
- Access to standard block methods like history, item counts, produce, consume...
- Access to stream tags - ie sample decoration and metadata
- Access to message passing (see below for description)
- See the Blocks-Coding-Guide to learn how to write blocks in python.
- See the following example python blocks.
Whats a PMT? PMT is a serialization library in GNU Radio, and a PMT is a serialization object used for stream tags and message passing. See pmt.h for more details.
The goal behind the PMT extensions feature was to make memory allocation more efficient when using PMTs for message passing. Just like the GNU Radio scheduler preallocates buffers to reuse at runtime, so should users have the ability to reuse PMTs to avoid allocation in the fast path.
The blob PMT is for passing chunks of memory, basically a pointer and a length in bytes. A blob can be used as a datagram (think UDP) to pass a bounded message from one block to another. Many of the message passing blocks in this project exclusively use blobs to form a message-passing domain between blocks (more on this below).
The PMT blob extensions turn blob from an immutable data type into a mutable type. Basically, a user can create an uninitialized blob, resize it, and get a writable pointer to fill the blob with data. This way, a user can allocate a pool of blobs a init-time, and simply reuse them at runtime, but more on that in the PMT manager section.
Also, python wrappers are provided for the blob type so users can get access to the blob's memory in python, as a numpy data type, and read from and write to the memory.
The PMT manager is actually not a PMT, but a regular C++ object. But what is special about the manager is that it allows the user to create a pool of reusable PMTs, and to track downstream consumption of PMT objects in the manager.
Normally, a PMT is created and passed to a downstream consumer. When all downstream consumers delete their references to the PMT, the object is deconstructed and freed back into nothingness. With the PMT manager, rather than being deconstructed, the PMT will return to the manager, where the user can reuse the deleted PMT without re-allocation.
Also, the PMT manager creates the concept of backpressure for stream tags and messages. Because the user can block on the manager, waiting for PMTs to be returned, we can effectively block the upstream from processing while waiting on the downstream to use and free our PMT resource.
The PMT library obviously has its uses, but it can be cumbersome to make and later extract arbitrary netstings of PMT objects. Fortunately, when coding in Python, this is no longer necessary with the pmt.from_python and pmt.to_python function calls.
Arbitrary nesting of native python types and PMT types can be effortlessly converted between python native types and PMT objects. Almost all native python types are supported, the list of supported types includes:
- None
- bool
- string
- int, float, complex
- dict, list, tuple
- numpy arrays
Example using pmt.to_python:
from gruel import pmt a_pmt_obj = pmt.PMT_T a_python_obj = pmt.to_python(a_pmt_obj)
Example using pmt.from_python:
from gruel import pmt a_python_obj = True a_pmt_obj = pmt.from_python(a_python_obj)
- See pmt_blob.h header for API info.
- See pmt_mgr.h header for API info.
- See stream_to_blob.cc for both extensions in-action.
The message passing feature allows blocks to pass arbitrary messages to downstream consumers. Messages can be anything: data packets which allows the user to implement a packet layer, or control packets so the user can implement a control plane. The sky is the limit.
The ability to consume messages and produce messages is implemented as new API calls in the user's custom block. The source and destination of messages, much like streaming data, are determined by the top-level connections of the user's flow graph.
The messages are actually just gr_tag_t, the same type used for stream tags; and the message passing is implemented on top of the existing stream tag interface. Therefore, the message passing implementation is merely abstracts away the streaming part of stream tags to give the user nothing but messages.
The advantage to this implementation: to do message passing, no scheduler or gnuradio companion modifications are necessary to use message passing. Simply connect message passing ports like normal streaming ports and the GrExtras implementation will do the rest.
The downside to this implementation: GNU Radio streams do not implement a many-to-one model. If you need multiple message sources to one message sink, simply use the msg_many_to_one/m21 block that comes with GrExtras.
Using messages, the packet boundary is preserved between the TCP packet and the packet de/framer:
- See block.h for API details.
- See the message passing section in the Blocks-Coding-Guide.
- See python examples
- See c++ examples
To support these various features, GrExtras has its own block interface. This interface supports message passing and is the basis for the python blocks implementation as well. GNU Radio offers gr_block, gr_sync_bloc, gr_decim_block, and gr_interp_block, each having a different API for different producer/consumer modes. However, this interface can provide one API for all stream production/consumption models of operation.
The GrExtras block interface has only one constructor and one type of work function to overload. All of the other familiar class methods are present. The user simply sets the auto consume mode and relative rate, and whoolah, the same work function API is applicable to all producer/consumer modes.
So, if you are not message passing, or writing blocks in python, you can still have fun coding to this new API. See block.h for more details.
Here at the GrExtras project, we like to find new and efficient ways to accomplish mundane tasks. Consider the example of the adder block in GNU Radio; for each data type we would like to support, a new block must be created, where most of the implementation is simply copy-pasta (gr_add_ff, gr_add_ii....).
Now, one my save some effort by generating each copy with a simple sed script. Or, we can make use of the native C++ features, namely, C++ templates. By using templates, we can write one interface, one templated implementation, and instantiate it for each data type we desire to support.
Lets see some examples:
Not only was that simple and fun, but its also easy to integrate an alternative work function; for example, calling into a volk routine:
Here is a brief summary of the blocks GrExtras has to offer:
Message passing blocks
- Blob to stream - message/blob domain to stream domain
- Stream to blob - stream domain to message/blob domain
- Blob to socket - message/blob domain to udp/tcp socket
- Socket to blob - udp/tcp socket to message/blob domain
- Blob to filedes - message/blob domain to file descriptor
- Filedes to blob - file descriptor to message/blob domain
- Tun/Tap - a tuntap interface with blob IO
- Packet framer - blob input, framed bytes output
- Packet deframer - bytes input, deframed blob output
- UHD async msg source - reimplementation with message passing
- PMT RPC block - make remote calls on an object with message passing
- Many to One - combine multiple message sources into one port
Math operator blocks
- add, subtract, multiply, divide
- add const, multiply const
- vector forms of all blocks
- volk optimizations w/ float32
- fixed point arithmetic is supported
Performance blocks
- Signal source - look-up table optimized signal source
- Noise source - look-up table optimized noise source
Misc blocks
- Stream selector - runtime configurable stream muxing
- No need to start and stop the flow graph!
- Delay - block with runtime configurable delay of output