Home
\cite{netbeams2009} describes NetBEAMS as a single-hop sensor network with a single data sink. Each of the nodes of the network is composed not only by the original YSI Sonde device used by SF-BEAMS, but also with two additional pieces of hardwares. This documentation describes the composition of these three components as the NetBEAMS Gateway node, described as follows:
- Gumstix console-vx \cite{gumstix it's a COTS ARM-based hardware that provides a computer-on-module environment for the development of small embedded systems. This is the main environment of data extraction and remote data processing;
- YSI 6600EDS V2 \cite{YSI-Sonde the original YSI sensor device responsible for observing the environment. It is connected to the Gumstix embedded system, where the DSP is executed;
- Huawei E220 USB Modem: a GSM modem used to transfer the collected data from the Gumstix to the RTC laboratory.
||
The Data Sensor Platform (DSP) is based on a micro-kernel architecture developed on top of OSGi \cite{osgi}, a Java modular framework \cite{netbeams2009}. The platform represents the foundation of the execution of NetBEAMS, since it is executed from the Gumstix \cite{gumstix}. In this way, the Gumstix uses a cross-compiled Gentoo Linux operating system, loaded with JamVM \cite{jamvm}, an alternative Java Virtual Machine (JVM) designed for embedded systems. In order to transport data from sensor hosts to the network sink, it uses the HTTP Protocol via the celular network using the GSM modem. Figure above shows the architecture of the DSP gateway node designed by the NetBEAMS research group.
The NetBEAMS Gateway Node is an embedded system that contains a set of layer found in any computer. However, each of the resources described from the system were customized for the NetBEAMS system. The main components of such system can be summarized as follows:
- Operating System it uses a cross-compiled Gentoo Linux \cite{gentoo}, which supports a wide variety of development tools, including the Java Virtual Machine (JVM), the underlying platform system;
- Java Virtual Machine it uses the JamVM, a ~200 KB version of the Sun Microsystems Java Virtual Machine, version 2.0;
- OSGi Framework it uses the Knopflerfish OSGi \cite{knopflerfish} implementation of the OSGi 4.1 specification. More details in the following sections;
- DSP Platform and other Bundles The plug-and-play DSP Components are based on the OSGi bundles capabilities, which can reuse services registered in the OSGi framework.
||
As depicted in the Figure above, the DSP Sonde Data Producer component opens a serial connection with the device through the Java Native Interface (JNI) driver and produces the data flow 1. The data stream format at this stage is as described on Table \ref{tab:ysi-data-stream}. Later, by parsing and transforming the data to DSP internal representation, it sends the data to the DSP Platform as depicted on flow 2. Finally, the DSP Platform decides to send the data to the DSP Wire Transport Client, as shown on flow 3, by following rules defined by the system administrator. At this point, the data is ready for transmission.
The architecture of the DSP, aligned with the OSGi capabilities, reuses the Service-Oriented Architecture that can easily enable and disable plug-and-play components. By following a publish-subscriber design-pattern \cite{gof}, the service producer registers into the service broker while the consumer looks for a given registered service in the Service Broker. For this reason, this approach allows decoupled interoperability among modules as it is shown in the Figure below:
||
The DSP infrastructure is based on a loosely-coupled architecture hosted in a given host computer with an OSGi container, where the entire system is composed by a set of plug-n-play OSGi modules. Each DSP component, as an instance of an OSGi Module, can perform specialized tasks according to its specification. In general, each DSP component can be categorized as follows:
- Data Producer (DP), whose main function is to produce data for the system;
- Data Consumer (DC), whose main function is to consume the received data.
||
The Data Producers and Data Consumers exchange data through the DSP Message Broker using a message unit called DSP Message. As it is depicted in Figure \ref{fig:DSP-Producers-Consumers-Components-Interactions}, a Data Producer creates a set of one or more DSP Messages wrapped up in a DSP MessagesContainer in order to be sent to the DSP Message Broker. When the broker receives the set of DSP Messages, it requests a list of Data Consumers to each of the DSP Messages based on rules managed by the DSP Matcher. Whenever the broker acquires the list of Data Consumers, it delivers a copy of the DSP Message to each of the Data Consumers in the receiving list. Upon the DSP Message arrival, each Data Consumer can extract the actual data from the DSPMessage. Finally, although the components can be distinguished between Data Producers and Consumers, any of them are entitled to send and receive administrative DSP Messages, which is covered in the following sections.
When the DSP Platform is installed into the OSGi framework, it follows the OSGi specifications without any changes to remain it in the installed state. In this way, it is important to note that the DSP Platform and its plug-and-play components are directly mapped with the OSGi Framework as follows:
- OSGi Bundle: DSP Component Artifact in Jar format
- SondeDSPProducer.jar is the component developed for the YSI sonde
- OSGi Activator: DSP Component Activator
- Class specified in the MANIFEST.MF artifact (next sections)
- OSGi Service Reference: DSP Component Class
- Class specified by the component author.
The DSP Platform, along with the DSP Framework, are the only DSP Components needed to be installed into the OSGi container. For this reason,the DSP Platform is responsible to install and activate the additional DSP components that makes part of the platform during the start up process, which leverages its bootstrap process. Nevertheless, in order to automate the process of automatic install and removal of the plug-and-play components, the DSP Platform provides a deployment configuration file that describes which DSP components must be installed with a given priority number. The name of this configuration artifact is called config.xml (see listing \ref{file:dsp-config.xml}), and is depicted by the XML Schema \cite{xml-schema} in the following Figure:
||
In order to show an example, consider the DSP Platform on the embedded system Gumstix in Figure \ref{fig:sf-netbeams-node}. It is started by providing a deployment configuration artifact config.xml included in the appendix. It lists the components that the DSP Platform needs to install and load for its operation. As it is shown, the DSPWireTransportClient and the DSPSondeProducer are the identification of the components, followed by the name of the physical file and an assigned loading priority. While the DSPWireTransportClient is responsible for the serialization and deserialization of the DSP Messages exchanged by the DSPSondeProducer and any remote DSP component. In this case, the DSPSondeProducer is the DSP component responsible to collect measurement data from the YSI Sonde and temporarily store them before being transmitted.
Therefore, in order to orchestrate the DSP Components together, the DSP Platform component is responsible for installing and uninstalling during its state "Starting" and "Stopping", respectively, as it is shown in Figure below.
||
When the system administrator starts the DSP Platform, it performs the regular OSGi procedures to start the bundle. Up until that state, it will get the list of DSP Components to be installed as described in the config.xml artifact, trying to install each of them in an optimistic way. After that, it will continue on the state "Active" until the system administrator decides to stop it.
When the DSP Platform is on the Active state, it is ready to manage the DSP Messages exchanges as described through figure \ref{fig:DSP-Producers-Consumers-Components-Interactions}. The following sections describe the participating classes in starting the DSP Platform and its execution.
In general, the DSP Platform is an extension of the OSGi Platform, as it can be seen in the UML Class Diagram \cite{uml} in Figure \ref{fig:DSP-Platform-Class-Diagram-Simple}. The class Activator contains a unique instance of the Platform class, which contains instances of the classes ComponentManager, Matcher, the MessageBroker and DSPContextImpl. The DSPBundleController is the main instance that manages the plug-and-play DSP components, and therefore, is tightly-coupled with the OSGi Framework classes. Additionally, this class is part of the publish-subscriber design-pattern, as it it implements the interfaces Bundle Listener and Service Listener, whose responsibility is to track listens to the changes of the state of ServicesReferences on the DSP Platform.
||
As shown in the previous section, a DSP Message is the main unit of communication among DSP Components. In general, a DSP Message can be seen as an abstraction of an envelop containing the sections: a header and a body. The former is used for identification and routing purposes, while the latter is used to carry the payload of the message, that is, the collected data from a sensor. The following Figure depicts the abstraction of a DSP Message and its main components.
||
- Header is composed by 2 blocks that identify the data producer and consumer of the message, as well as the time of creation, UUID, etc. It is an important piece of information for the data delivery and routing purposes;
- Body contains the payload of the message. This section carries the collected data from a sensor.
In general, the DSP Platform offers a variety of types of DSP Messages for different purposes. For example, any measurement data must be wrapped up in a Measurement Message, while a Query Message is used to exchange messages among the components for the purpose of management. In this way, the main DSP Messages can be summarized as follows:
- Measurement Message is used to transport any sensor collected data;
- Query Message is used to query a DSP component about its configuration properties;
- Update Message used to update a DSP component's configuration properties;
- Acknowledgement Message is used for the transport communication protocol, by acknowledging the reception of a DSP Message. More details in the Remote Data delivery section.
These DSP Classes and other can be seen in Figure the UML Class Diagram \cite{uml} of Figure \ref{fig:DSP-Messages-Classes}. Each DSP Message contains an instance of a Header and Body. Whenever a DSP component is ready to transmit messages, it wraps up the set of DSP Messages into the instance of the Messages Container, which contains information about the collection of messages being transmitted with its own identification. In this fashion, the Messages Container is the main communication unit between 2 different DSP Components.
||
Another use of the DSP Messages is during the DSP Platform activation process, where bootstrap messages are used to configure each DSP component. In order to send configuration parameters, the DSP Platform uses Update Messages using a unique type of Message Content called DSP Properties, which is described in the next section.
As mentioned in the previsou section, a DSP Message can carry any data on its body, also called the payload. In such a way, any data representation must just extend from the Message Content abstract class as it is shown \ref{fig:Sonde-MessageContainer-Class-Diagram}, as defines the data representation of the YSI Sonde data.
||
The properties measured from the YSI Sonde can be captured by the class SondeDataType as a direct render of the measurements data depicted in Table \ref{tab:ysi-data-stream}. In addition to the regular data types, the class also provides additional properties such as the attribute "samplingTimeStamp" that carries the time when the data was collected from the YSI Sonde. However, in order to capture a set of measurements at once, the SondeDataContainer class used to carry a set of SondeDataType classes, as it is composed by one or more instances of SondeDataType. In order to illustrate other examples, consider the UML Class Diagram \cite{uml} of the Figure below, as a payload of observations made from a mouse over a screen, which captures the name of the event, the button name and the x/y coordinates.
||
As described in the previous section, the DSP Platform uses DSP Update Messages to configure internal components' initial configuration values. In order to do so, the DSP Platform uses an instance of the DSP Properties, which is composed of a list of initial DSP Property and their relating Values instance depicted on the UML class diagram \cite{uml} of the next Figure.
- Property Instance = initial-rate
- Value Instance = 40
||
All in all, the DSP Platform requires the knowledge of the aforementioned DSP classes in order to initialize. The UML Sequence Diagram \cite{uml} in Figure \ref{fig:From-OSGi-DSP-Platform-Start-to-All-Components-Sequence-Diagram} depicts the exact moments when the additional the plug-and-play DSP components are installed and bootstrapped by the DSP Platform, as the UML State Diagram \cite{uml} in Figure \ref{fig:DSPPlatform-Install-Usage-State-Diagram} described.
- Start DSP Activator Gate after installing the Activator indicated by the DSP Bundle's MANIFEST.MF artifact, the OSGi Framework makes a call to the method Activator.start();
- Bootstrap DSP Component Gate after the component has been initialized (by the call the method init() from the class ComponentManager), the DSP Component is ready to be bootstrapped, and the optional bootstrap message is delivered.
||
In general, when a DSP Component finishes preparing the DSP Messages Container, it contacts an entity responsible for sending and proxying message to other components. The DSP Broker acts like a mailman, receiving messages and forwarding copies of it to other entities in the system, abstraction in which is shown in Figure \ref{fig:DSP-Message-Broker-Abstraction}. In this way, the DSP Broker receives a DSP Message and makes necessary deliveries to DSP components that it has knowledge of.
||||
||_A DSP Message Broker Abstraction||
The use of messaging exchange paradigm is used in NetBEAMS because it decouples the system by using messages take into account their structure to filter the messages. In this way, the DSP Broker uses a filter component called DSP Matcher, which provides a list of DSP Components that needs to receive a copy of the DSP Message. It turns out that the DSP Matcher bases its decisions on structured rules provided by the system administrator for an specific DSP Platform instance. The UML Class diagram for the DSP Broker, Matcher and particiaping classes is depicted in Figure \ref{fig:DSPBroker-Matcher-Class-Diagram}, and describes how the DSP Broker implementation is tightly coupled with the DSP Matcher, as well as to the lists of DSP Components sorted by name and type.
||
In general, the DSP Matcher is composed by a list of MatchRule, which contains a reference to a MatchCriteria and a MatchTarget. The former identifies the rule for matching a given DSP Message based on the producer and consumer's message type and location, while the latter describes the DSP Component that will receive the DSP Message, and if its delivery is done by means of a gateway component. In this way, the DSP Matcher can be seen as a function, as depicted in Table \ref{tab:matching-rule}, that takes a DSP Message as an input and returns a list of DSP consumers, and the matching configuration artifact provided by the administrator. See listing \ref{file:dsp-matcher-config.xml}) for details.
Data Consumers ( DSP Message )} := Verify the DSP Message's Header +
Verify the matcher rules, which contains the list of consumers for the given message
One example of such matching rule used by the DSP installed at the Gumstix can be seen in listing \ref{file:dsp-matcher-config-gumstix.xml}. the Gumstix system to configure the data delivery of the messages produced by the DSP Component DSP Sonde Producer. It describes a list of maching rules that reseambles a filtering system. This configuration contains 2 match rules: one related to internal DSP Configuration and another for a remote destination. The match criteria describes what the DSP Message Header properties must be, while the match target is the destination of the message. One important definition is the gateway Component Type, which defines the name of a remote DSP Component that is capable of receiving a remote DSP Message.
Upon receiving all the matching rules and analyzing the DSP Message against the rules, the DSP Broker selects a set of the running DSP components to receive a copy of the DSP Message object in two different ways:
- In-memory local message delivery if the receiving DSP Component is located in the current local host, a deep copy of the instance of the DSP message is delivered;
- IP Addresses are correctly resolved by using the Ethernet card in the device: localhost, 127.0.0.1, or the same IP Address for Producer and Receiver resolves into a local device. An example for such type of delivery can be seen in the match ruleID all messages sent to dsp manager of listing \ref{file:dsp-matcher-config-gumstix.xml};
- Serialized remote message delivery if the receiving DSP Component is located in a foreign/remote host, the message is serialized in a format defined by the transport protocol chosen the DSP Component responsible for the transport. The following section describes the existing DSP Data Transport component;
- IP Address from the Producer and Consumer are different, and are not resolved to be in the same host. An instance of such match rule can be seen in the match rule ID send-message-to-remote url dsp of the code snippet \ref{file:dsp-matcher-config-gumstix.xml}.
In order to send messages through the wire, the DSP Platform uses two symmetric components that serialize and deserialize DSP Messages to XML and back to Java POJO, respectively, and vice-versa. Named DSP Wire Transport Client and DSP Wire Transport Server \cite{netbeams2009}, these components use the HTTP \cite{http} protocol to transport the serialized version of the DSP Messages. Figure \ref{fig:DSP-to-DSP-Remote-Communication} depicts the scenario where a DSP Sensor Node transmits a DSP Message through an HTTP POST request.
||
- DSP Wire Transport Client responsible for making HTTP POST Requests to the service provided by the DSP Wire Transport Server component. The body of the HTTP POST request contains a serialized version of an instance of a DSP MessagesContainer in XML format;
- DSP Wire Transport Server exposes a Web Server listening to port 8080 that receives the HTTP POST request from the Client. Upon receiving the request, the server deserializes the DSP MessagesContainer back to the in-memory POJO and proceeds with the delivery to the DSP Broker. At this situation, the component packages acknowledgement messages and any queued message to the client and piggybacks \cite{xml-piggybacking} it to the HTTP response body.
As described by \cite{netbeams2009}, NetBEAMS is a single-hop sensor network whose sensors transmits data from the sensors to the server. Upon receiving the data stream, the DSP Wire Transport Server component desirializes the DSP MessagesContainer and its enclosing DSP Messages back to a Java class instance and delivers each of the DSP Messages to the DSP Broker, which delivers a copy of the message to each DSP Component listed by the maching rule.
Whenever a DSP bundle producer is ready to transmit messages, they are added into a DSP Messages Container (see UML Class Diagram in Figure \ref{fig:DSP-Messages-Classes}), which is serialized in XML to be transmitted. Listing \ref{file:dsp-message-serialized-ysi} shows an instance of a DSP MessagesContainer in serialized in XML, ready for transmission. In fact, the listing is an example of the transmitted data collected from a sensor located at the 192.168.0.103 to be transmitted to the host 192.168.0.106, and the process of transmission in the server-side can be followed in Figure \ref{fig:From-DSPSondeComponent-to-DSPWireTransportClient-Sequence}.
||
The Sonde DSP Component is responsible for starting a worker thread called SondeProducer, which is responsible for constantly read from the RS-232 serial port by using an instance of the SerialReader class. This process depicts the production of the data flow "flow 1" of figure \ref{fig:netbeams-software-stack}, which is a collection of SondeDataType for each of the measurement data read on a SondeDataContainer. As depicted in figure \ref{fig:From-DSPSondeComponent-to-DSPWireTransportClient-Sequence}, it first read the serial connection using the class SerialReader, which parses the data stream. After parsing the data, the reader creates an instance of the class SondeDataType, the POJO responsible for carrying out the measurement data. As described earlier, the data is added into the SondeDataContainer to dispatched to the DSP Broker, whose responsibility is to get the list of DSP Components that is setup to receive a copy of the DSP message (see previous code snippet). As for the setup done for the Gumstix, the measurement message is supposed to be sent to a remote destination of host 192.168.0.7. At this point, the DSP Broker makes a decision to send the DSP Message to the default gateway component DSPWireTransportClient.
After the message arrives in the DSPWireTransportClient, the message is maintained in a temporary repository of outbound DSP Messages called MessagesQueues after the message is hashed by the destination IP address and the DSP Client sender using an instance of QueueMessageData. As shown in Figure \ref{fig:From-DSPWireTransport-Client-to-DSPWireTransportServer-Sequence}, the DSP TransportSender is a worker thread that is started by the DSP Wire Transport Client and is constantly transmits the data in the outbound queues after a configurable rate. It makes a transport of the message and receives the response, updates the state of the messages to transmitted, to finally send the response messages to the DSP Message Broker.
||
While the DSPWireTransportClient makes an HTTP Request with the DSP Message, the DSPWireTransportServer's servlet receives the request. It first deserializes the MessagesContainer and delivers each of its messages to the DSP Broker. At this point, the server verifies if there are messages in the outbound queue that are addressed for the host that requested the messages delivery. By using a technique called piggybacking, the server adds to the HTTP Response a serialized version of the MessagesContainer with any DSP Messages. In addition to any DSP Message, the server prepares and sends acknowledgement messages back to the requesting host to the host.
||
The gate "From DSP WireTransport Server to Broker Gate" defines the actual delivery of the DSP Message to the DSP Broker. As this section is a tutorial about the DSP Platform, the remaining steps will be explained on Section 3, while showing the design and implementation of the component DSP Data Persistence.
Once the DSP Platform automatically installs all the components described by the configuration descriptor, the container verifies the dependencies and starts each of the DSP components of Figure \ref{fig:knopflerfish-execution}, the OSGi Framework container Knopflerfish \cite{knopflerfish}.
||||
||_The execution of the Knopflerfish Container||