Skip to content

Practical Tips for Using the MAL Java API

Jump to bottom
CesarCoelho edited this page Aug 20, 2015 · 2 revisions

Introduction to the concepts

Basics

Who should read this?

The intended audience of this tutorial is any software developer wishing to use the CCSDS Message Abstraction Layer (MAL) Java API in order to implement a new MO service or a new transport binding (and/or data encoding). It also might be helpful to integrators whishing to integrate available Java components into a complete MO stack. It is not intended for people whising to define new MO services or technology bindings or developers not using the MAL Java API. It is assumed you are familiar with the CCSDS Mission Operations (MO) framework in general and the MO reference model and the MO MAL in particular. It is the purpose of this document to help you get familiar with the MAL Java API.

The second part of this document tries to get you started with the reference test bed distributed by the CCSDS so that you can easily verify your implementation is compliant to the standard.

What is the MAL Java API?

The MAL Java API is a concrete technology binding of the abstract MAL. It provides an interface to the top, i.e. for consuming and providing MO services, and an interface to the bottom, i.e. for mapping the MAL messages to a concrete transport protocol. If any MAL implementation or transport binding implementation adheres to this standard, they can be plugged into the MO stack without major effort.

This tutorial gives you a hands-on approach for implementing a new transport binding and for implementing new MO services (the latter is planned for a future revision of the tutorial).

Where do I get the MAL Java API?

The MAL Java API is standardised in CCSDS 523.1-M-1. Based on this standard you may implement your own MAL Java API, but it is more likely you would like to use an existing implementation. CCSDS provides a reference implementation developed by ESA. Additionally, several transport bindings are available as well as a code generator to transform an abstract service specification in XML to stubs for different technologies (e.g. Java, Word documents, …).

The ESA contributions can be obtained from GitHub. In particular this tutorial assumes you have cloned the following repositories:

  1. CCSDS_MO_POM
  2. CCSDS_MO_XML
  3. CCSDS_MO_StubGenerator
  4. CCSDS_MO_APIS
  5. CCSDS_MO_MAL_IMPL
  6. CCSDS_MO_TRANS
  7. CCSDS_MO_APPS

These repositories contain a full MO stack to get you started (service definitions in XML, stub code auto generator, MAL API and reference implementation, different transport bindings and a test application). Please note that the test bed is contained in the CCSDS_MO_TESTBEDS repository, which helps you with testing and validating your implementations but is not part of an MO stack. Please refer to The MAL test bed section in this document to learn more about the test bed.

Install the reference implementation

In order to start implementing your own service or transport binding you first need to compile the cloned repositories and make them available to you and your team. This process heavily relies on Maven and it is assumed you are familiar with its concepts. Please execute

mvn install

in repositories 1 to 5 of the above list in exactly this order. If you wish to make use of one of the example transport bindings execute

mvn -P ESA install

in CCSDS_MO_TRANS (repository #6). You may specify a different profile in the POM in order to use a different transport binding. If you want to make the libraries available to your team using a respoitory manager replace install with deploy in the commands above and do not forget to change the POMs according to your infrastructure. If you want to attach sources and documentation the Maven goals source:jar and javadoc:jar are your friends.

Run the demo application

If you want to check that all artifacts installed successfully you can run the demo application which is included in CCSDS_MO_APPS (repository #7). The demo application consists of a data provider and several data consumers connecting to the provider. Compile it with mvn install and then execute

runProvider.bat

in the CCSDS_MAL_DEMO directory to start the provider (assuming you are on a Windows machine; running on different systems should be straight-forward with the batch file as starting point). You can start one or more consumers by executing

runConsumer1.bat

where you replace 1 with 2 or 3 to start a second and a third consumer. For each service entity a window pops up. Connect the consumers to the provider by clicking Consumer > Reconnect in the consumer window. Start delivering telemetry by clicking Publisher > Start TM in the provider window, optionally try different settings. You should see the consumers displaying the data.

TODO: Making the connection between MAL and its realization in the Java API

The concepts set forth in the MAL are realized in the MAL Java API. Unfortunately, the connection between the two is not always obvious. This section aims to explain how some of the MAL concepts are realized in the Java API.

  • Realization of primitives (requests and indications) in MAL Java API.
  • Representation of the MAL data types.
  • …

Structure of the MAL Java API

The MAL Java API is organized in several packages:

  • org.ccsds.moims.mo.mal
    Interfaces and classes representing operations, services, standard errors and exceptions, and top-level MAL functions. Encoding and decoding interfaces are also defined here.
  • org.ccsds.moims.mo.mal.structures
    Representation of data types defined by the MAL.
  • org.ccsds.moims.mo.mal.consumer
    This part of the API is dedicated to the MAL clients initiating interactions as service consumers.
  • org.ccsds.moims.mo.mal.provider
    This part of the API is dedicated to the MAL clients handling interactions as service providers.
  • org.ccsds.moims.mo.mal.broker
    This part of the API is dedicated to the MAL clients handling PUBLISH-SUBSCRIBE interactions as brokers.
  • org.ccsds.moims.mo.mal.transport
    The transport API defines the interfaces and classes required by the MAL transport interface.
  • org.ccsds.moims.mo.mal.accesscontrol
    The access control API defines the interfaces and classes required by the MAL access control interface.
  • org.ccsds.moims.mo.mal.encoding
    The encoding API should be used by the transport modules that need to externalise the encoding behaviour of a body element.

For the sake of simplicity the package name prefixes will be omitted from here on.

Using the MAL Java API

This tutorial provides guidelines on how to use the MAL Java API in the following situations:

  • Implementation of a transport binding. It will make use of the following MAL Java API packages:
    • transport - transport API
    • encoding - (transport) encoding API
  • Implementation of an MO service (TODO for a future revision). It will make use of the following MAL Java API packages:
    • consumer
    • provider

The packages mal and structures are always needed. Of course not all classes and interfaces in these packages are implemented in the reference MAL implementation. Unfortunately, there does not seem to be any information about the implementation responsibilities available, so this tutorial will try to give an overview of what has been or needs to be implemented by a MAL implementation, a transport binding implementation and a service implementation.

The implementation of access controls, brokers or extensions to the MAL data model are not part of this tutorial.

How to implement a new transport binding

In order to implement a transport binding most of the transport interfaces and classes (the transport API) have to be implemented. These are:

  • class MALTransportFactory
  • interface MALTransport
  • interface MALEndpoint
  • interface MALMessage
  • interface MALMessageHeader
  • interface MALMessageBody
  • interface MALErrorBody
  • interface MALRegisterBody
  • interface MALDeregisterBody
  • interface MALPublishRegisterBody
  • interface MALPublishBody
  • interface MALNotifyBody

The MAL exclusively uses this API for binding to a transport. However, in face of modularity and reusability it is recommended to use the encoding API provided by the MAL Java API if applicable to the transport. Usage of the encoding API is optional, but provides a means to separate the actual encoding from the transport interface. Because some API parts responsible for encoding reside outside the encoding package but inside the mal package, a more precise wording will be employed throughout this document: The API contained in the encoding package is called transport encoding API from now on (the MAL Java API calls this just encoding API). The API parts responsible for encoding in the mal package are now called data encoding API. Together they form the encoding API, but we try to avoid this term because it collides with the different meaning of what the MAL Java API defines as encoding API.

Data/transport encoding here means the on-the-wire encoding of the message body contents of a MAL message, while transport stands for encoding everything else which is necessary to transport a MAL message (MAL message header, protocol specific information). The transport component uses the encoded MAL message body created by the data/transport encoding component as a Blob or byte array. The MAL Java API does not enforce this modularization and you are free to implement a monolithic transport binding, especially if the distinction between data/transport encoding and transport does not work for your transport binding.

If you decide to make use of this optional API, these are the classes and interfaces that need to be implemented:

  • transport encoding API (package encoding)
    • class MALElementStreamFactory
    • interface MALElementInputStream
    • interface MALElementOutputStream
  • data encoding API (package mal)
    • interface MALEncoder
    • interface MALDecoder
    • interface MALListEncoder
    • interface MALListDecoder

So how do you know, if your transport is compatible with the data/transport encoding API? If you can split the encoding of a MAL message such that it can be encoded element-wise using a stream of elements, chances are high your transport is compatible with this API. You might even consider to reuse an already existing encoding implementation.

How are messages encoded or decoded?

In order to implement a transport binding you should know how the transport API is supposed to be used by the MAL. This does not become very clear from the MAL Java API, so we try to give an overview here.

In order to use a transport the MAL creates an object that implements MALTransportFactory. It is by configuration that the MAL knows which MALTransportFactory implementations are able to handle which protocol (all available transport factories are held in a registry). This logic resides in the static methods of MALTransportFactory, which are already implemented. Because one factory class may handle multiple protocols, the protocol name is passed in during factory construction. There will be one transport factory instance for each protocol. When your specific transport protocol gets used for the first time the MAL calls createTransport(), which is the only method besides the constructor you have to implement.

createTransport() is then responsible for creating an instance of MALTransport. Thus, there should be one instance of a MALTransport implementation for each protocol. Now, if the MAL wants service instances to communicate with each other, it assigns a so called endpoint to each of them (actually, there is a one-to-many relationship between endpoint and service instances). Each endpoint gets assigned a URI unique to this specific protocol. It is your responsibility as transport binding implementor to allocate this URI. The endpoint is represented by the MALEndpoint interface and created by the MAL calling createEndpoint() on the transport object. After the endpoint has been created, it can be used by the MAL to send and receive messages, which will be described in the following sections.

Message sending

When the MAL has to send a message it does so by first constructing it with a call to the endpoint’s createMessage() method. This method exists in four overloaded definitions with the only difference being how the MAL operation and the message contents, the body, are passed in. While the representation of the MAL operation is only a technical detail, message content passing is more interesting. The MAL either passes an Object array or a MALEncodedBody. While the MALEncodedBody represents a message body that is already encoded (e.g. because it has been received by a broker and does not need to be decoded in order to relay it to subscribers), the Object array represents the message body elements. Unfortunately, the MAL Java API does not get concrete about the possible types in the Object array, so it is best to assume to get anything, from a Java mapping type, a Union type, null or even a MALEncodedBodyElement. The MAL expects a MALMessage as return type of createMessage(). If available, you should provide a specialized class of MALMessage depending on the operation being executed. The framework provides you with these specialized interfaces:

  • interface MALErrorBody
  • interface MALRegisterBody
  • interface MALDeregisterBody
  • interface MALPublishRegisterBody
  • interface MALPublishBody
  • interface MALNotifyBody

At some later point the MAL may decide to actually send a message and will call an endpoint’s sendMessage() or sendMessages() method. Now, it is up to the binding implementation to properly encode the message and send it out. If your binding uses splitting into transport and data/transport encoding, then this is all orchestrated by the transport component and you have to know how to use the encoding component properly. The idea behind the encoding API is that you can read and write elements from and to a stream. You extend MALElementStreamFactory to create these input and output streams (MALElementInputStream, MALElementOutputStream). Your implementation of MALElementOutputStream contains the method writeElement() that takes a MAL element, encodes it and writes it to the stream. Encoding of an element works by calling its encode() method. Because the element implementation is part of the MAL, it does not know how to encode itself. Therefore you pass in an implementation of MALEncoder, which contains encoding methods for all basic MAL element types. The element’s encode() method then uses these methods to encode itself. It is up to you as transport binding implementor to implement MALEncoder. If lists are about to be encoded, you should use an implementation of MALListEncoder in order to encode the list the way you want (e.g. with a length descriptor at the beginning or with a stop symbol or whatever you can think of or are required to do in order to stay compliant with your transport). To start encoding of the message body elements you can use MALElementStreamFactory.encode() that takes an array of message body elements.

Message reception

It is the responsibility of the transport binding to ensure messages sent to it are received properly. This depends very much on the nature of the transport and you should know best how to do this (e.g. by creating threads waiting for input on some communication socket). The MAL is informed about new messages by calling the onMessage() method of a MALMessageListener. The object implementing the MALMessageListener interface has been previously set by the MAL on a specific endpoint using the enpoint’s setMessageListener() method. It is the MAL’s responsibility to provide a MALMessageListener implementation.

Decoding elements works similar to encoding them: You create an implementation of MALElementInputStream, call the readElement() method to read in an encoded element from the input stream and return the decoded element. However, you will soon realize that you need to pass in an element before you can read an element. This is on purpose, because in order to choose the right decoding method for the following element you need to know the type of the element to decode. The MAL passes in some kind of prototype element with a throw-away value, but whose decode() method you can use to properly decode the element in the input stream. This will not work if polymorphism is employed for the last message body element (it is only allowed there). In this case the passed in element is null and you have to determine the type by decoding type information that your encoder should have written to the stream. As for obtaining type information you can also query the MAL service that triggered the message to be generated. The service description defines the operations and data types and the MAL provides an API to retrieve this information. The following code snippets shows an example on how to use a MALEncodingContext to find out information about the current interaction and possible types:

MALEncodingContext ctx = ...;
MALMessageHeader header = ctx.getHeader();
UOctet stage = header.getInteractionStage();
InteractionType interaction = header.getInteractionType();
Boolean isError = header.getIsErrorMessage();
MALOperationStage operationStage = ctx.getOperation().getOperationStage(stage);
int bodyElementIndex = ctx.getBodyElementIndex();
Long shortForm = (Long) operationStage.getElementShortForms()[bodyElementIndex];
Object[] allowedShortForms = operationStage.getLastElementShortForms();

This can be useful if your encoding scheme makes a distinction between a general abstract element and an abstract attribute. Because the passed in prototype element is null in both cases, you have to retrieve type information from the service (the allowed short forms) to decide if the element is an abstract attribute or not. You can then create your own prototype element using

MALContextFactory.getElementFactoryRegistry().lookupElementFactory(Long.valueOf(shortForm)).createElement();

Unfortunately, these convenience methods are missing from the MAL Java API. Conceptually, decoding then works like encoding: The decode() method uses the decoding methods of the passed in implementation of MALDecoder or MALListDecoder that you provide.

TODO: How to implement a new service

future revision, ideas:

  • How to run code auto-generator if service XML is available?
  • Explain the structure of the auto-generated code and how to go on from there (in fact this should already become clear from the MAL Java API, but wrap it up here in a way suitable as tutorial).
  • Make clear how services are used from user applications.

TODO: Pitfalls and gotchas

elaborate on:

  • Meaning of MAL element types.
  • Union types and native Java types.
  • Durations, Time, FineTime: Configuration and epoch information.
  • Somewhat convoluted configuration system.
  • Missing convenience functions.

The MAL test bed

What is the test bed and what is it for?

The MO test bed provides a common ground for testing new implementations of MO interfaces. At the time of writing tests exist covering the Message Abstraction Layer (MAL) and the Space Packet Protocol binding (MAL/SPP binding). Any implementation of these interfaces should pass the tests defined in this test bed in order to be considered MO compliant. The MAL/SPP binding tests are still work in progress and not yet usable as a reference test system.

Where do I get the test bed and how do I install it?

The test bed is organized as a collection of several Maven artifacts. Therefore the described usage of the test bed heavily depends on Maven. Depending on your or your organization’s needs a repository manager such as Nexus is in place. Here it is assumed that you have a working infrastructure for getting and deploying Java artifacts.

Prerequisites

The following prerequisites have to be met in order to get the test bed up and running:

  • Java 7 JDK
  • local Maven setup (with working repository manager if necessary)
  • the actual test bed (not yet publicly accessible), extracted into a directory without spaces (will be the root directory for all subsequent directory references)

What is included in the test bed

At the time of writing the test bed consists of these five components:

  • MOIMS_TESTBED_MAL
    Contains test definition FitNesse wiki pages for testing a MAL implementation. The fixture code in Java connecting the wiki pages to the components under test is also included here.
  • MOIMS_TESTBED_POM
    The parent POM used by all test bed components. It is advisable to declare your dependencies in its <dependencyManagement> section.
  • MOIMS_TESTBED_SPP
    Contains test definition FitNesse wiki pages for testing a Space Packet Protocol binding implementation. In future this might serve as a template for testing other protocol bindings as well.
  • MOIMS_TESTBED_SPP_FRAMEWORK
    Contains the Space Packet Protocol binding fixture code in Java connecting the wiki pages to the components under test. In future this might serve as a template for other protocol bindings as well, provided they are structured similarly to the Space Packet Protocol binding implementation.
  • MOIMS_TESTBED_UTIL
    Contains the test bed components responsible for setting up the test framework, i.e. starting the FitNesse server and setting up providers and consumers. The exact setup is controlled by configuration files.

How to use the test bed

The test bed provides a means to set up two complete MO stacks communicating with each other through some underlying technology. The stacks are called remote and local and are executed in their own processes. Each stack consists of (from top to bottom):

  • test application
  • service provider/consumer
  • MAL implementation
  • transport implementation

The test bed is run as a dummy JUnit test contained in MOIMS_TESTBED_MAL or MOIMS_TESTBED_SPP using the FitNesse JUnit test runner. This test is contained in src/test/java/org/ccsds/moims/mo/mal[spp]/test/suite/JUnitSuiteTest.java and configured with annotations. The @FitnesseDir annotation denotes the root directory of FitNesse, the @Name annotation the test or suite name to be executed. Thus, in order to execute acceptance tests involving the whole MO stack simply execute this JUnit test. FitNesse then controls the test execution using the wiki pages and the fixture code.

It is your responsibility to define a sensible MO stack. First you need to define the dependencies you need in order to execute your tests. You do this by defining your own profile in the POM files of the tests and adding your artifacts there. In a second step you need to tell the FitNesse system how to wire together these components, i.e. you need to specify which implementation of the MO interfaces you want to use. This is done with two sets of configuration files, one for the local stack, another one for the remote stack. Your stack should consist of tested components except for the one you want to test. Otherwise you will have a hard time finding the culprit in case tests fail.

Configuration files

Configuration of the test bed is split across several files. The format is that of a standard Java property file. These files are loaded during start-up of the test bed in the order they are listed here. Their content is put in the global System properties (?). Not all of these files need to be present, as long as you properly set all configuration parameters required by your deployment. The file structure assists you in keeping like parameters together in one file. File location for the local stack is ./src/main/resources/deployment/[profile]/, where [profile] is a profile for your configurations that should be present in the POM and selected with Maven’s -P option for executing the tests. The remote stack’s configuration files reside in ./.

File name Description
SuiteManagementEnv.properties deployment specific properties (?)
SuiteManagement.properties test specific properties (?)
BaseLocalMALInstanceEnv.properties
BaseLocalMALInstanceMAL.properties
LocalMALInstanceEnv.properties
LocalMALInstanceMAL.properties
RemoteMALInstance.properties (test specific??)
BaseTestServiceProviderEnv.properties??
BaseTestServiceProviderMAL.properties??
TestServiceProviderEnv.properties
TestServiceProviderMAL.properties

Configuration properties

The configuration properties currently understood by the test bed are listed here, together with a description. Names in the following tables’ first columns have to be prepended with org.ccsds. to yield the proper property name. Each component (including your own) may add its own list of configuration properties, which are not listed here of course.

Transport properties

Property name Description
moims.mo.mal.transport.default.protocol
moims.mo.testbed.transport.factory fully qualified class name of the test bed transport factory
moims.mo.testbed.transport.protocol transport protocol used for the test bed (e.g. malspp or rmi)

Local MAL properties

Property name Description
moims.mo.testbed.local.class local MAL class; the local MAL instance, brokers and service providers and consumers are brought into life here; for testing the MAL this would be org.ccsds.moims.mo.mal.test.suite.LocalMALInstance
moims.mo.testbed.local.configuration.dir local configuration directory (?)

Remote MAL properties

Property name Description
remote.cmdline.{OS_NAME}.{#} remote command line prefix; used to start the remote process; replace {OS_NAME} with your operating system name and replace spaces with underscores; replace {#} with the position of the command line argument, starting at 1 (e.g. ...Windows_XP.1=cmd, ...Windows_XP.2=/c)
moims.mo.testbed.remote.class fully qualified class name of the test service provider; for testing the MAL this would be org.ccsds.moims.mo.mal.test.suite.TestServiceProvider
moims.mo.testbed.remote.output.dir
moims.mo.testbed.remote.configuration.dir
moims.mo.testbed.remote.extra.args extra arguments for starting the remote JVM; this could be useful if you want to connect a debugger to the remote JVM; in this case use something like -agentlib:jdwp=transport=dt_socket,server=y,address=54320,suspend=n
moims.mo.testbed.remote.classpath.extra
moims.mo.testbed.remote.classpath.maven
moims.mo.testbed.remote.classpath.filter

Other properties

Property name Description
moims.mo.testbed.wait.timeout
moims.mo.testbed.transport.level.shared.broker
moims.mo.mal.security.factory.class

Even more properties

Property name Description
moims.mo.mal.accesscontrol.factory.class fully qualified class name of the access control factory class; for a test access control factory class use org.ccsds.moims.mo.mal.test.accesscontrol.TestAccessControlFactory
moims.mo.mal.factory.class fully qualified class name of the MAL factory class; for the ESA MAL implementation this is esa.mo.mal.impl.MALContextFactoryImpl

In addition to those properties each component of the MAL stack may provide its own configuration parameters. Refer to the component’s documentation for more information. It is good practice to prepend the configuration parameters of your own components with their package names.

Home

Getting Started

Tutorials

Documentation

Software

FAQ

Clone this wiki locally