Using vrpn_Connection

Russell Taylor edited this page Apr 2, 2015 · 1 revision

The heart of the VRPN library is the vrpn_Connection class. This class dispatches messages locally and across network connections, delivering the messages using callback handlers. Application code does not normally deal directly with the vrpn_Connection class, but rather access it through one or more device classes. Server code will usually create a "listening" instance of this class, and then call its mainloop() function each time through the loop, but will not otherwise deal with the class. It is the device-specific class which deals most intimately with the vrpn_Connection class. Thus, those writing device classes need to thoroughly understand how this class works.

A device-specific class that is derived from one of the existing classes should not have to deal with the vrpn_Connection class to a large extent, since the base class will have handled describing message types and itself as a sender. If the specific class can send messages that the parent class cannot send, then it will need to describe those messages to the connection. If it can receive messages, it will need to register callback handlers for these messages.

All messages in VRPN have a time, a sender and a type associated with them. The time is defined by the sender, and usually corresponds to the time at which the action generating the message occurred (local clock time). For example, the vrpn_Button classes report the time that a button was pressed or released. The sender of a message is a bit of a misnomer; think of this field as being a channel descriptor that describes either the sender or the receiver of a message. For example, this might be "Button0". The type of the message might be "Button Toggle," which tells that a single button was pressed or released. All senders and types are specified using a string name, but actually accessed by a token. Before they can be used, the names must be registered with the connection to get a token.

Initially, there are only two types of vrpn_Connections: one that listens on a socket and one that makes a remote connection to the listening kind. In the future, there are likely to be different classes for shared-memory communication, and for connections that actually deal with only one process. Hopefully, this will have minimal impact on the interface to the class.

Establishing a connection

For the case of a network VRPN connection (currently the only case), there are two actions that need to occur for the connection to be initiated: creating a listening connection and creating a remote connection. Once these are both started, the connection is initialized and remains until one or the other sides exit, thus dropping the connection.

Creating a listening connection

First, a listening endpoint for the connection needs to be created (usually by a server) using one of the overloads of the following function:

vrpn_Connection * connection = vrpn_create_server_connection();

This call creates a connection that will listen for requests on the UDP port that is specified as an optional parameter, or will use the default port if none is specified. This connection will check for incoming packets on this port whenever its mainloop() member function is called.

The current implementation of the listening connection will allow multiple simultaneous remote connections to one server, with all messages from the server duplicated to each endpoint. Messages coming in from one endpoint are not re-sent to the other endpoints.

Creating a remote connection

The application does not normally directly ask for a connection, but rather creates a device that handles getting its own connection. These remote devices get connections using the function (defined in vrpn_Connection.h):

vrpn_Connection * vrpn_get_connection_by_name (const char * cname, const char * local_logfile_name = NULL, long local_log_mode = vrpn_LOG_NONE, const char * remote_logfile_name = NULL, long remote_log_mode = vrpn_LOG_NONE, double dFreq = 4.0, int cOffsetWindow = 2);

This function is passed the complete name of the device that is to be opened (for example, Tracker0@ioglab.cs.unc.edu). The function will strip off the device name and @ character and then open the port. If the same program opens more than one device with the same connection, a pointer to the existing connection is returned. This is useful when the application opens more than one device connected to the same server.

vrpn_get_connection_by_name() works by passing as the second part of the device name (after the @) as a parameter to the constructor for the connection.

vrpn_get_connection_by_name() takes many optional parameters. The first four control logging; the last two control synchronization. A local logfile can have an arbitrary pathname and can log incoming messages, outgoing messages, or both. The same properties can be specified for a remote logfile. The difference between local and remote logging is especially significant when some messages are being sent unreliably, since what is sent by one end of the vrpn_Connection may not match what is received at the other.

Connection initialization

The connection negotiation is initiated by the remote connection, which opens a TCP socket for listening and then sends a UDP datagram to the well-known port on the listening connection that tells it the machine name and port number of the TCP socket. If there is no connection attempt within a small timeout period, another request is made. If several requests fail, the server is assumed to be down or already connected and the connection fails. This mechanism allows both fast connection and fast detection of failure to connect. Once the listening connection receives a UDP request for a connection, it establishes a TCP link to the remote connection.

Now that there exists a TCP link between the connections, each sends a "hello" message to the other, which tells what version of the VRPN protocol each is running. If the versions do not match, the connection fails and the TCP link is dropped. From this point, the behavior of the connections is symmetric; as if they were peers, until the link is shut down.

If the versions match, each side of the connection opens a new UDP port and sends a description of the port to the other across the TCP channel. Each then establishes a bound UDP port that connects to the other's (the original, well-known UDP port is not used for this). This allows a path for unreliable (and presumably lower-latency) traffic to flow between the connections. In the future, other protocols may be supported, with their descriptions sent over the TCP channel.

At this point, the connections describe to each other their local sender and message type information, as further described under registering a sender and registering a type.

Local devices can register a handler for the messages named by vrpn_got_connection or vrpn_got_first_connection. The vrpn_Connection will send a message whenever remote devices connect so that the local devices can react appropriately.

Connection shutdown

A vrpn_Connection detects that its peer has closed the connection when it receives an exception on one of its reads, writes, or selects. A select is performed on the TCP connection each time the mainloop() member function is called, so the determination will be fairly rapid unless the drop is caused by a network timeout. Once the connection has been dropped, a listening connection will begin accepting new requests on its well-known UDP socket while a remote connection simply stops sending messages.

Local devices can register a handler for the messages named by vrpn_dropped_connection or vrpn_dropped_last_connection. The vrpn_Connection will send a message whenever the remote device disconnects so that the local devices can react appropriately.

Registering a sender

The sender field in a message specifies the name of the device that is sending or that is to receive the message. This can be thought of as a channel descriptor. Senders are defined to a vrpn_Connection by a string name; the connection reports a token (a long integer) that is to be used to refer to that sender when sending and receiving messages. Examples of senders might be, "Tracker0@ioglab.cs.unc.edu" and "Button0@ioglab.cs.unc.edu". A device can get a token for a given sender name by calling:

long vrpn_Connection::register_sender(char *name);

For example, a program could open a listening connection and then describe a new sender called "Test0@ioglab.cs.unc.edu" by doing the following:

vrpn_Connection *connection = new vrpn_Connection();
long my_id = connection->register_sender("Test0@ioglab.cs.unc.edu");

The identifier my_id could then be used as the sender field to send a message or to register a message handler.

Each vrpn_Connection object maintains a list of senders that have been registered with it. When a link is established between two connections, each describes its mapping from strings to tokens to the other. Each vrpn_Connection maintains a table that it uses to translate the sender field coming across the link into its local value before delivering the message. This allows each connection object to allocate its own list of sender-to-token mappings independently.

This string-to-token mapping is done for two efficiency reasons. Firstly, it allows the connection to avoid doing string-matching to determine what to do with a given message. Second, it reduces the bandwidth sent over the link between connections (for names longer than 4 characters). Strings are used to give the device classes and user code flexibility in defining senders.

Registering a type

The type field in a message specifies the kind of message being sent. Like the sender field, this can be thought of as a channel descriptor. Types are defined to a vrpn_Connection by a string name; the connection reports a token (a long integer) that is to be used to refer to that type when sending and receiving messages. Examples of types are, "Button Toggle" and "Tracker Pos/Quat". A device can get a token for a given type by calling:

long vrpn_Connection::register_message_type(char *name);

For example, a program could open a listening connection and then describe a new type called "Test_type" by doing the following:

vrpn_Connection *connection = new vrpn_Connection();
long my_type = connection->register_message_type("Test_type");

The identifier my_type could then be used as the type field to send a message or to register a message handler.

Each vrpn_Connection object maintains a list of types that have been registered with it. When a link is established between two connections, each describes its mapping from strings to tokens to the other. Each vrpn_Connection maintains a table that it uses to translate the type field coming across the link into its local value before delivering the message. This allows each connection object to allocate its own list of type-to-token mappings independently.

This string-to-token mapping is done for two efficiency reasons. Firstly, it allows the connection to avoid doing string-matching to determine what to do with a given message. Second, it reduces the bandwidth sent over the link between connections (for names longer than 4 characters). Strings are used to give the device classes and user code flexibility in defining type names.

Sending a message

Once the sender and type of a message have been described, it is possible to begin sending messages over a connection. Messages are given to a connection using the pack_message() method:

virtual int pack_message(int len, struct timeval time,
                         long type, long sender, char *buffer,
                         unsigned long class_of_service);

The len parameter specified the length of the message, in bytes. In order to maintain alignment, the connection will internally pad this length to a multiple of 8 characters. The padding will be removed when the message is delivered. The connection will guarantee that messages begin on an 8-byte boundary when delivered.

The time parameter is the time at which the message was generated, according to the local clock on the machine generating the message. This need not be the time that the message was queued; for example, some tracker drivers put the time that the first character was received from a serial port for a tracker record.

The type and sender parameters are the tokens received when they were registered.

The buffer parameter points to the data that is to be sent, in architecture-independent byte order. This allows messages to be sent between hosts that differ in byte order. Use the vrpn_buffer() and vrpn_unbuffer() functions defined in vrpn_Shared.h to make sure that the byte ordering matches the network standard. In fact the buffer is delivered byte-for-byte to the receiver at the other end of a connection link, so any format is legal so long as the sender and receiver agree on what it will be. For example, a device might be implemented that used receiver-makes-it-right byte ordering by checking the order of a token long at the beginning of the message and swapping all values only if needed. All VRPN system devices use common-format, agreeing with the htonl() standard.

The class_of_service parameter describes the type of delivery required for this message. These classes are listed at the top of the vrpn_Connection.h file. This parameter is a bitwise OR of the flags. In the current implementation, all flags except vrpn_CONNECTION_RELIABLE are ignored; if this flag is set, the TCP channel is used for delivery; otherwise, UDP is used.

Important notes

For network connections, messages are not sent until the mainloop() method is called. There are separate buffers for UDP and TCP delivery; both are sent when mainloop() is called. In a possible future implementation of shared-memory or single-thread implementation, the messages may be delivered as soon as they are packed, so code using this should allow for delivery (and callback invocation) any time from when the message is queued until the next time mainloop() is called.

If a callback has been specified for the given sender and type on the connection object to which a message is packed, that message will be delivered locally as well as being sent across the connection.

For listening connections that do not currently have a link to a remote connection, messages that are packed will not be delivered, but will be silently discarded when mainloop() is called. Messages will still be sent to any local callback that has been specified.

Example

The following code will open a connection, describe a sender and type, and send a message reliably on the connection. This is a contrived example because there will almost certainly be nobody listening when the message is sent:

vrpn_Connection *connection = new vrpn_Connection();
long my_type = connection->register_message_type("Test_type");
long my_id = connection->register_sender("Test0@ioglab.cs.unc.edu");

struct timeval now;
gettimeofday(&now,NULL);

connection->pack_message(sizeof("Hi!"), now, my_type, my_id,
                         "Hi!", vrpn_CONNECTION_RELIABLE);
connection->mainloop();

Registering a callback handler

A vrpn_Connection delivers messages using callback routines. Devices should usually register interest in messages whose sender matches theirs for each type of message they generate. The call to register a callback function to a connection using its register_handler() method::

virtual int register_handler(long type, vrpn_MESSAGEHANDLER handler,
                             void *userdata, long sender = vrpn_ANY_SENDER);

The type and sender parameters are the tokens received when they were registered. These are used to filter messages so the handler is called only for messages of the appropriate type from the specified sender. The default sender matches any sender, so will deliver all messages of that type.

The userdata parameter is passed to the handler procedure; if non-NULL, it should point to a structure containing information that the callback handler will use to process the message (more on this below).

The handler parameter is the name of the function that is to be called when a message of the specified type is received. It must be of the type vrpn_MESSAGEHANDLER:

typedef int (*vrpn_MESSAGEHANDLER)(void *userdata, vrpn_HANDLERPARAM p);

This is a function that takes two parameters. The first is the userdata parameter that will be set to the value that was specified when the callback handler was installed. Normally, this will be typecast into a pointer to either a class or a structure that contains information that the handler needs to know in order to process the messages. For example, it may be a pointer to a transformation matrix that is to be filled in from a tracker report.

The second parameter passed to the callback handler is a structure that contains information about the message and the message itself:

typedef struct {
    long        type;
    long        sender;
    struct timeval  msg_time;
    int     payload_len;
    const char  *buffer;
} vrpn_HANDLERPARAM;

The type and sender parameters are those specified by the routine that called pack_message(), translated into their local values by the local connection object. These parameters can often be ignored by a handler, as it will only be called for the type and sender that it was specified to receive. If one handler was installed for multiple message types or senders, then these fields will be helpful.

The msg_time parameter is the one specified by the routine that called pack_message(). This is the time according to the sender's clock. A future implementation may synchronize clocks between the two ends of a connection, in which case this may in the future be translated into the clock on the local machine.

The buffer points to the actual message, which is of payload_len length. This is the message exactly as sent to pack_message(). This should be an architecture-independent representation, in case the byte order or other data representations (structure packing, floating-point format, alignment constraints) differ between the sender and receiver, which may be on different architectures. VRPN guarantees that the message will start on a 4-byte boundary. The sender and receiver of a message need to agree on the message format.

The handler routine should return 0 on success and -1 on failure. Failure means failure to parse the message, which indicates a faulty connection, so the connection will be shut down.

Example

The following code will open a connection, describe a sender and type, and register a callback handler. This example has not been compiled and tested, but hopefully there are no bugs. This example will read the messages generated by the example of sending a message and print them. This example is for a server-based system. A client would use vrpn_get_connection_by_name() rather than 'new vrpn_Connection()', as described under getting a connection.

int my_handler(void *userdata, vrpn_HANDLERPARAM p) {
    char    *prompt = (char*)userdata;  // Type-cast to the right type

    printf("%s:%s\n", prompt, p.;buffer);   // Assume it's a string message
    return 0;
}

main() {
    vrpn_Connection *connection = new vrpn_Connection();
    long my_type = connection->register_message_type("Test_type");
    long my_id = connection->register_sender("Test0");

    connection->register_handler(my_type, my_handler,
                 (void*)"Test message", my_id);

    while (1) connection->mainloop();
}

A complete vrpn_Connection example

This example creates a VRPN server that emits test messages to anyone that connects to it. It also registers a callback handler to handle any test messages that are sent. This callback is called both for the locally-sent messages and for any that are sent by a remote connection linked to its local one. The messages are sent once per second.

#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/time.h>
#include <vrpn_Connection.h>

// This is the callback handler that will print the string from any
// incoming message (and also the locally-packed ones).
int my_handler(void *userdata, vrpn_HANDLERPARAM p) {
    char    *prompt = (char*)userdata;  // Type-cast to the right type

    printf("%s:%s\n", prompt, p.buffer);    // Assume it's a string message
    return 0;
}

main() {
    // Open a connection and register my sender name and the type of
    // message that I care about.
    vrpn_Connection *connection = new vrpn_Connection();
    long my_type = connection->register_message_type("Test_type");
    long my_id = connection->register_sender("Test0");

    // Register the callback handler
    connection->register_handler(my_type, my_handler,
                     (void*)"Test message", my_id);

    // Once a second, pack a message and then call mainloop() to cause it
    // to be sent and also to call the callback for any messages.
    while (1) {
        struct timeval now;
        gettimeofday(&now,NULL);
        connection->pack_message(sizeof("Hi!"), now, my_type, my_id,
             "Hi!", vrpn_CONNECTION_RELIABLE);
        connection->mainloop();
        sleep(1);
    }
}

Note that a remote version of the above example would look exactly the same, except that it would pass the name of the server to the vrpn_Connection constructor.

Also note that devices form an abstraction layer above the vrpn_Connection class, hiding the details of packing and unpacking the messages from application and server code. It is possible for user-level code to access a connection directly, as described above; VRPN could be used as a message-passing system for application objects.

Forwarding

The vrpn_ConnectionForwarder and vrpn_StreamForwarder utility classes have been written to automate forwarding messages from one connection over another connection. For instance, in the distributed nanoManipulator, messages from a scanning probe microscope arrive at the main controller process over a vrpn_Connection. From there we also want to send them to a graphics process on another host. We open a second vrpn_Connection to the graphics host, create a vrpn_StreamForwarder between the two connections, and instruct it to forward messages of the types the graphics process needs to receive. After that, all we have to do is make sure both vrpn_Connection's mainloop()s get called regularly and the message streams will flow as necessary.

We can also create remote-controlled forwarders without having to code them explicitly. A VRPN server which has a vrpn_Forwarder_Server running can be contacted by a vrpn_Forwarder_Controller and instructed to open new server ports and forward arbitrary message types over them. In the above example, instead of forwarding via the main controller process the controller could instruct the microscope to open a connection on well known port and forward the important messages there, and the graphics host could connect directly to the microscope, reducing network latency by one hop.