Michael Adler edited this page Mar 3, 2015 · 1 revision


Communications between host and FPGA have traditionally been difficult both hard to implement and difficult to port between FPGA plaforms. LEAP attempts to abstract this communication through the Remote Request-Response (RRR) interface. RRR consists of a user-defined interface which is elaborated at LEAP compilation time to a set of C++ and Bluespec interface files. These interfaces are layered on top of an abstract host-FPGA physical channel.

In this tutorial, we will examine the programmer interface to RRR using the STDIO service as an example. STDIO uses RRR to allow hardware to make use of the software-side STDIO library. The code samples shown below are actual examples used in LEAP source, or produced by the RRR compiler. To reduce the burden of understanding, some parts of the code have been removed. Excisions are denoted by ....

This tutorial covers only the programmer’s interface to RRR. Other tutorials cover:

RRR Architecture
RRR Compilation

RRR Language Example

service STDIO
    server sw (cpp, method) <- hw (bsv, connection)
        method Req(in UINT64[64] data, in UINT8[8] eom);

    server hw (bsv, connection) <- sw (cpp, method)
        method Rsp(in UINT8[8] tgtNode,
                   in UINT8[8] command,
                   in UINT8[8] meta,
                   in UINT32[32] data);

        method Rsp64(in UINT8[8] tgtNode,
                     in UINT8[8] command,
                     in UINT8[8] meta,
                     in UINT64[64] data);

        // Tell all clients to flush all pending requests
        method Sync(in UINT8[8] dummy, out UINT8[8] ack);

        // Set conditional mask for mkStdio_CondPrintf
        method SetCondMask(in UINT32[32] mask);

In this example, two RRR services are declared. In one case, a software server (server hw (bsv, connection) <- sw (cpp, method)) is declared, and in the other a hardware server(server sw (cpp, method) <- hw (bsv, connection)) is declared. The declaration syntax appears complex. However, the syntax examples given are the only way in which servers may be declared, and they may be copied into new RRR declarations. The declaration of the server implicitly declares the corresponding client. Each service must have at least one server, and may have at most two servers, on hardware and one software.

Each server may have a practically unlimited number of methods. Methods may have any number of arguments and any number of return results. Arguments to the client are denoted with in and results returned by the client are denoted with out. The code synthesized by the RRR compiler orders method arguments and return values in the order that they appear in the method declaration.

RRR currently supports only 8,16,32, and 64 bit values as arguments. In software, these types are represented by the UINT8, UINT16, UINT32, and UINT64 types. In hardware the types represented are Bit#(8), Bit#(16), Bit#(32), and Bit#(64). Complex types must be manually constructed by the programmer.

Currently, all software server methods must have at least one argument going to hardware, even if this argument is not used. In the example above, the Sync method has a dummy argument.

Host-side Software interface

In the case of STDIO, the software-side client and server are united in a single class file.


// self-instantiation
static STDIO_SERVER_CLASS instance; // The STDIO_SERVER_CLASS is special – the generated stub invokes this name specifically.

// stubs STDIO_SERVER_STUB serverStub; // Instantiate generated server code STDIO_CLIENT_STUB clientStub; // Instantiate generated client code public: …. // FPGA to software request chunks void Req(UINT64 data, UINT8 eom); // Invoked by generated code …



User Software Server Code

    VERIFYX((reqBufferWriteIdx + 1) < (sizeof(reqBuffer) / sizeof(reqBuffer[0])\

    // Data arrives as a pair of 32 bit values combined into a 64 bit chunk     
    reqBuffer[reqBufferWriteIdx++] = data;
    reqBuffer[reqBufferWriteIdx++] = data >> 32;

    if (eom)
        VERIFY(reqBufferWriteIdx > 1, "STDIO service received partial request")\

        // Decode the header                                                    
        UINT32 header = reqBuffer[0];

        STDIO_REQ_HEADER req;
        req.command = STDIO_REQ_COMMAND(reqBuffer[0] & 255);
        req.fileHandle = (header >> 8) & 255;
        req.clientID = STDIO_CLIENT_ID((reqBuffer[0] >> 16) & 255);
        req.dataSize = (header >> 24) & 3;
        req.numData = (header >> 26) & 15;
        req.text = reqBuffer[1];

        switch (req.command)
          case STDIO_REQ_FCLOSE:

The user provided Req handler, which is invoked by the generated server code. The handler code in this case parses the request packet format, which is programmer defined and specific to the stido service.

User Software Client Code

// Sync --                                                                      
//     Called at the end of a run.  Ensure that all pending client requests     
//     have arrived at the host.                                                

The Sync method is a blocking call. Note the use of the dummy argument.

FPGA-side interface

RRR produces hardware-side interfaces in the form of Bluespec System Verilog modules, which make use of SoftConnections. As a result, hardware clients and servers may be instantiated anywhere in a user design, without affecting the user interfaces. Client and server interfaces may be instantiated inside the same code. Methods on hardware clients and servers are split-phase, reflecting the non-atomicity of transporting data to and from the host.

`include "awb/rrr/client_stub_STDIO.bsh" // Include generated RRR header 
module [CONNECTED_MODULE] mkStdIOService (); // We can have soft connections!
    ClientStub_STDIO serverStub <- mkClientStub_STDIO(); // Instantiate client stub
    // Hardware clients have split-phase requests and responses. 
    rule processReq (True);
        clientStub.makeRequest_Req({msg.chunk, prev},

In this example, the hardware side client sends data to the host for processing using the Req method. Req has no response, so the client does not have a getResponse_Req method, which is would have in the case that Req returned a value. Note that the arguments to Req occur in lexical order.

Hardware server

Hardware servers are instantiated by the server sw (cpp, method) <- hw (bsv, connection) syntax in the service statement.

`include "awb/rrr/server_stub_STDIO.bsh" // Include generated RRR header 
module [CONNECTED_MODULE] mkStdIOService (); // We can have soft connections!
    ServerStub_STDIO serverStub <- mkServerStub_STDIO(); // Instantiate server stub
    // Hardware servers have split phase requests and responses. 
    rule processReq (True);
        // Software-initiated sync request has reached every local node
        // and is now complete
    rule processSyncReq (True);
        let dummy <- serverStub.acceptRequest_Sync();

This example illustrates how a hardware server handles a message from a software client. The Sync method has one input and one output argument. When software invokes Sync, the function arguements are marshalled and transported to the FPGA. After this transport is complete, acceptRequest_Sync becomes ready and may be invoked by the user hardware. Upon receiving the Sync, the hardware will complete the action associated with Sync (flushing all the STDIO nodes in the hardware). Once this action is complete the hardware invokes acceptRequest_Sync to notify software of the completion of the operation. RRR methods in software are blocking.

RRR architecture

RRR is conceptually the leaf node of a communications hierarchy which serves to orchestrate communication between host and FPGA. Leaf RRR servers and clients feed into a centralized manager which servers to packetize requests and transmit data between software and FPGA.


The RRR parser is quite finicky. Care should be taken to reproduce syntax declarations as they appear above. Future releases of LEAP will continue to support RRR. However, it is planned that SoftConnections will eventually replace RRR as the preferred means of communicating with software.
RRR routines are not thread safe. When utilizing multiple threads and RRR, the programmer must make use of a thread-safe primitive like GAsyncQueue.

You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.