Getting Started

Welcome to the RaftLib wiki!

There's a nice intro into the history, and goals of RaftLib here: raftlib page. That's not what this page is for.....what you can find here are the instructions and background you need to actually use RaftLib. If after following this example you'd like more, there are more in /examples, /testsuite, /benchmarks.

Lets start with a simple example, not quite a hello world since well..that doesn't make too much sense in a multi-threaded streaming environment so we're going to look at a sum application which as the name suggests takes a potentially infinite stream of numbers, adds each pair and writes the output to an output stream.

#include <cassert>
#include <iostream>
#include <cstdint>
#include <cstdlib>
#include <raft>
#include <raftio>
#include <raftrandom>
#include <raftmath>
#include <raftutility>

template< typename T  > class Sum : public raft::kernel
{
public:
   Sum() : raft::kernel()
   {
      /** declare ports **/
      input.template addPort< T >( "input_a", "input_b" );
      output.template addPort< T  >( "sum" );
   }
   
   virtual raft::kstatus run()
   {
      T a,b;
      input[ "input_a" ].pop( a );
      input[ "input_b" ].pop( b );
      /** smart obj allocate directly on output port **/
      auto out( output[ "sum" ].template allocate_s< T >() );
      /** like an iterator, dereference the out to write to it **/
      (*out) = a + b;
      /** out will automatically release to the next kernel on scope exit **/
      return( raft::proceed );
   }

};


int
main()
{
   const auto count( 1000 );
   /** define type for streams **/
   using type_t = std::int64_t;
   /** 
    * random_variate is a threaded wrapper around the C++ random number gen **/
   using gen = random_variate< std::default_random_engine,
                               std::uniform_int_distribution,
                               type_t >;
   /** this will be our worker 'sum' kernel **/
   using sum    = Sum< type_t >;
   /** make shorter name for print kernel **/
   using p_out  = raft::print< type_t, '\n' >;

   gen a( count ), b( count );
   sum s;
   print p;
   raft::map m;
   /** link the only output port of a to the "input_a" port of s **/
   m += a >> s[ "input_a" ];
   /** link the only output port of b to the "input_b" port of s **/
   m += b >> s[ "input_b" ];
   /** take the only output port of s and link it to the only input port of p **/
   m += s >> p;
   /** 
    * NOTE: this will be going away soon, 
    * to be called on scope exit, an explicit
    * barrier call will enable integration with
    * sequential code.
    */
   m.exe();
   return( EXIT_SUCCESS );
}

Ok, lets take it line by line starting from main. Which is declared below

int
main()
{

Moving on we get to our declarations and instantiations, where first we'll define a variable which determines how many numbers to sum.

 const auto count( 1000 );

Next we'll define some types.

   /** define type for streams **/
   using type_t = std::int64_t;
   /** define random number generator types **/
   /** 
    * random_variate is a threaded wrapper around the C++ random number 
    */
   using gen = random_variate< std::default_random_engine,
                               std::uniform_int_distribution,
                               type_t >;
   /** this will be our worker 'sum' kernel **/
   using sum    = Sum< type_t >;
   /** make shorter name for print kernel **/
   using p_out  = raft::print< type_t, '\n' >;

So the above code defines the types (c++11 style), which shorthand the namespaces and types for the code below.

RaftLib defines streams as links within a raft::map object. An advanced user can define multiple maps (more on this later). After the user defines the map, it must be executed (with the exe function, future versions will execute on scope exit..but the exe will always be usable for immediate execution and blocking till completion).

The code below shows a link statement to the m map object.

m += a >> s[ "input_a" ];

There are several versions of the link function, but the basic syntax is this

map_object += source_kernel >> destination_kernel;

if each source only has a single output and the destination only has a single input. Otherwise, the programmer can specify port names while linking as in

map_object += source_kernel[ output port name ] >> destination_kernel [input port name ];

If either the source or destination only have a single port to connect to, then the name can be omitted for brevity. As a convenience the return object of the map add assign operation contains references to the first and last kernels linked, which can be accessed like

auto ret( map_obj += source_kernel >> destination_kernel[ "y0" ] );
//get source
ret.getSrc()
//get destination
ret.getDst()
//usage
map_obj += another_source >> ret.getDst()[ "y1" ];

There are many more ways to link kernels that are described within the linking kernels section. There are a few rules for connecting ports, the types must be convertible, and the all ports must be connected before the graph is executed (both are verified by the runtime during the execution of the exe function).

Once a map is fully linked (run-time exceptions will be thrown for unconnected edges and type-mismatches), the map is executed. Currently there are quite a few schedulers and thread handlers for research purposes, however the default is to launch a thread per user-space kernel and execute. The run-time will dynamically handle the rest, optimizing the performance of the application as it executes. The next few sections will cover extending the base kernel object, accessing data from within a kernel, and the modalities to link kernels.