Application Template

Jeremy Barlow edited this page Mar 27, 2018 · 291 revisions

Table of Contents

Overview

The "Application" template generates an OpenDXL "application". An application runs persistently and can listen for DXL events and/or register DXL services.

This template is typically used to accomplish the following:

  • Execute a set of orchestration steps in response to receiving particular DXL event messages
  • Expose a DXL service that responds to incoming DXL request messages (request/response)

The name used to select the application template when invoking the OpenDXL Bootstrap application is application-template.

For example:

python -m dxlbootstrap application-template config\application-template.config myapp-output

Configuration File

The configuration file required to generate an "application" using the application-template is detailed below.

Example

The following is an example of a valid application-template configuration file.

[Application]
name=myapp
fullName=My DXL Application
appClassName=MyDxlApplication
copyright=Copyright 2018
eventHandlers=eventhandler1
services=service1

[eventhandler1]
topic=/some/dxl/event/topic
className=Event1Callback

[service1]
serviceType=/mycompany/service/myservice
requestHandlers=service1_requesthandler1

[service1_requesthandler1]
topic=/mycompany/service/myservice/request1
className=Service1Request1Callback

Application Section

The [Application] section is used to specify information about the application that is going to be generated.

Name Required Description
name yes The name of the application (will be used for the Python package name)
fullName yes The full name of the application
appClassName yes The name of the application class (Python class name)
copyright no Copyright information for the application
eventHandlers no A comma delimited list of event handler names. For each handler specified, a corresponding section must be defined within this configuration file (See "Event Handler" section below).
services no A comma delimited list of service names. For each service specified, a corresponding section must be defined within this configuration file (See "Service" section below).
installRequires no A comma delimited list of Python packages the application requires. These packages will be added to the appropriate files (setup.py, etc.)
languageVersion no The Python language version that the application requires. This is used to control the version information in the generated packaging and documentation files. Specify "2" (for Python 2.x only), "3" (for Python 3.x only), or "universal" (for both Python 2.x and Python 3.x). (Defaults to "2").

Event Handler Section

Each event handler specified in the eventHandlers property of the [Application] section must have a section defined which contains details about the particular event handler.

The section name must match the name of the event handler in the eventHandlers property (for example: [eventhandler1]).

Name Required Description
topic yes The DXL topic associated with the event handler
className yes The name for the event handler callback class (Python class name)
separateThread no Whether the event handler should be invoked on a thread other than the incoming message thread. A separate thread must be used if synchronous DXL requests are made in the callback (which could cause deadlock). (defaults to "yes")

Service Section

Each service specified in the services property of the [Application] section must have a section defined which contains details about the particular service.

The section name must match the name of the service in the services property (for example: [service1]).

Name Required Description
serviceType yes The "type" to associate with the service (a string used to uniquely identify the service type). Allows for multiple instances of the same service to be identified via their "type".
requestHandlers no A comma delimited list of request handler names for the service. For each handler specified, a corresponding section must be defined within this configuration file (See "Request Handler" section below).

Request Handler Section

Each request handler specified in the requestHandlers property of the service section must have a section defined which contains details about the particular request handler.

The section name must match the name of the request handler in the requestHandlers property (for example: [service1requesthandler1]).

Name Required Description
topic yes The DXL topic associated with the request handler
className yes The name for the request handler callback class (Python class name)
separateThread no Whether the request handler should be invoked on a thread other than the incoming message thread. A separate thread must be used if synchronous DXL requests are made in the callback (which could cause deadlock). (defaults to "yes")

Output

The following file structure will be created in the specified output directory when the OpenDXL Bootstrap application is executed with the application-template.

<app_name>\
    __init__.py
    __main__.py
    _version.py
    app.py
    eventhandlers.py (if event handlers are defined in configuration)
    requesthandlers.py (if request handlers are defined in configuration)
    _config\
        __init__.py
        app\
            __init__.py
            <app_name>.config    
            dxlclient.config
            logging.config
        sample\
            __init__.py
            dxlclient.config    
config\
    <app_name>.config    
    dxlclient.config
    logging.config
doc\
    sdk\
        README.html
        index.rst
        ...
    conf.py
sample\
    basic\
        basic_sample.py
    common.py
    dxlclient.config    
clean.py
dist.py
Dockerfile
LICENSE
MANIFEST.in
README
README.md
setup.py

Each of the files and directories listed above are described in detail below.

Directory Name Description
\ The root directory containing all of the files and directories that were generated by running the OpenDXL Bootstrap application with the application-template template.
clean.py The clean.py script cleans the application directory and file structure. This will result in any changes to the configuration and sample configuration files being reverted to the values in the associated "default" configuration files (see \<app_name>\_config\ directory below). Also, any extra files found in the configuration and sample configuration directories will be removed. See the Clean Script section for detailed information regarding this script.
dist.py The dist.py script creates redistributable packages for the application. The first step this script performs is invoking the clean.py script (see Clean Script). See the Distribution Script section for detailed information regarding this script.
Dockerfile Dockerfile that builds an image containing the application. The contents of this file also includes instructions for passing application configuration information to a container based on the image. See the Docker Support section for detailed information regarding this file.
LICENSE License information for the application. This file should be updated to include license information related to the application.
MANIFEST.in This file is used to generate the MANIFEST file that contains the list of files that will be included in the Python application package. By default the manifest includes the LICENSE file.
README README information that is included with the Python application package. This file should be updated to include application-specific information.
README.md README information that is typically used to describe a repository that contains the application source code such as GitHub. This file should be updated to include application-specific information.
setup.py The setup script that is primarily used for building, distributing, and installing the application.
\<app_name>\ This directory is the application package that contains the generated application modules.
__init__.py Initialization module for the application package.
__main__.py This module is the "main" entry point for the generated application. When the application is executed, the code in this file is responsible for configuring logging, parsing the command line arguments, and launching the application.
_version.py This file contains the version number for the application. The __version__ value should be updated to reflect the current version of the application.
app.py The application module that contains the generated main application class. The application class contains several callbacks that will be invoked during the application lifecycle (on_load_configuration, on_register_event_handlers, on_register_services, etc.). The main application class derives the Application base class which is a part of the OpenDXL Bootstrap library. For complete information regarding the Application base class see this page.
eventhandlers.py The eventhandlers module contains the generated event handler callback classes. A unique callback class will be generated for each of the event handlers defined via the eventHandlers property of the application-template configuration file. By default, each event handler will log the event messages that are received.
requesthandlers.py The requesthandlers module contains the generated request handler callback classes. A unique callback class will be generated for each of the request handlers defined via the requestHandlers property for each service defined in the application-template configuration file.
\<app_name>\_config\ This directory (and its sub-directories) contain the "default" configuration files for the application and its related samples.
__init__.py Initialization module for the configuration package.
\<app_name>\_config\app\ This directory contain the "default" configuration files for the application.
__init__.py Initialization module for the application configuration package.
<app_name>.config Configuration file that contains settings specific to the application. This file should be updated to include settings that allow users to configure the application. This is the application configuration file that will be included in the redistributable packages generated via the distribution script.
dxlclient.config Configuration file that contains settings for configuring the DXL client to connect to the DXL fabric. This is the DXL client configuration file that will be included in the redistributable packages generated via the distribution script.
logging.config Configuration file that contains logging settings for the application. This is the logging configuration file that will be included in the redistributable packages generated via the distribution script.
\<app_name>\_config\sample\ This directory contains the "default" configuration files for the samples.
__init__.py Initialization module for the sample configuration package.
dxlclient.config Configuration file that contains settings for configuring the DXL client to connect to the DXL fabric in the samples. This is the DXL client configuration file for the samples that will be included in the application distribution file (<app_name>-python-dist-<version>.zip) generated via the distribution script.
\config\ The directory containing the configuration files for the application.
<app_name>.config "Working" version of the application configuration file. This file may be edited to allow for in-place testing of the application (see use in Tutorial). However, running the clean or distribution scripts will revert the values in this file to those contained in the "default" configuration file (see \<app_name>\_config\app\<app_name>.config file above).
dxlclient.config "Working" version of the DXL client configuration file. This file may be edited to allow for in-place testing of the application (see use in Tutorial). However, running the clean or distribution scripts will revert the values in this file to those contained in the "default" configuration file (see \<app_name>\_config\app\ dxlclient.config file above).
logging.config "Working" version of the logging configuration file. This file may be edited to allow for in-place testing of the application. However, running the clean or distribution scripts will revert the values in this file to those contained in the "default" configuration file (see \app_name>\_config\app\ logging.config file above).
\doc\ The root documentation directory containing the source files for end-user application documentation. The documentation source files are written using reStructuredText and converted to final documentation using the Sphinx documentation generator.
conf.py The Sphinx build configuration file (contains configuration needed to customize Sphinx input and output behavior).
\doc\sdk\ Directory containing source files for end-user application documentation.
README.html HTML-based README file that will be copied to the root of the application distribution file (<app_name>-python-dist-<version>.zip) when the distribution script is executed.
index.rst The root (or main) documentation file written in reStructuredText. This file should provide links to the other documentation files.
... Several other documentation files are provided in the generated application including an overview of the application, installation and usage instructions, instructions for running samples, etc. All of these files are written in reStructuredText.
\sample\ This directory contains the samples for the application. These samples will be included in the application distribution file (<app_name>-python-dist-<version>.zip) when the distribution script is executed.
common.py The common module that is shared between the samples for the application. This module contains code to initialize logging and the location of the DXL client configuration to use in the samples.
dxlclient.config "Working" version of the DXL client configuration file for the samples. This file may be edited to allow for in-place testing of the application samples (see use in Tutorial). However, running the clean or distribution scripts will revert the values in this file to those contained in the "default" configuration file (see \<app_name>\_config\sample\ dxlclient.config file above).
\sample\basic\ Directory containing "basic" samples for the application.
basic_sample.py The "basic" sample that is generated for the application. By default, this sample will send an event that will be received by each event handler defined via the eventHandlers property of the application-template configuration file. The sample will also invoke (via synchronous request) each request handler defined via the requestHandlers property for each service defined in the application-template configuration file. The response received for each of the requests will be displayed.

Clean Script (clean.py)

This script cleans the application directory and file structure. This will result in any changes to the configuration and sample configuration files being reverted to the values in the associated "default" configuration files (see \<app_name>\_config\ directory above). Also, any extra files found in the configuration and sample configuration directories will be removed. Finally, any compiled Python files (.pyc) found in the application directory structure will be removed.

To run the script, execute the following command from the root of the generated application:

python clean.py

The purpose of this script is to remove any private configuration settings and files from the application structure. Thus, this script should be executed every time the application is about to be checked into a source code repository.

Distribution Script (dist.py)

This script generates redistributable packages of the application for use by others.

NOTE: The distribution script requires the Sphinx Python Document Generator to be installed.

To install Sphinx, run the following command:

pip install sphinx

To run the distribution script, execute the following command from the root of the generated application:

python dist.py

The distribution script for the application performs the following actions:

  • The application directory and file structure is cleaned (see clean script). This will result in any changes to the "working" configuration and sample configuration files being reverted to the values in the associated "default" configuration files. Also, any extra files found in the configuration and sample configuration directories will be removed. This cleaning is done to prevent private configuration settings and files from being included in the distribution.
  • The distribution directory is created (app\dist).
  • The documentation for the application is generated (from app\doc) and placed in the doc sub-directory of the distribution directory.
  • The sample and its associated configuration files (from app\sample) are copied into the sample sub-directory of the distribution directory.
  • The application libraries are created (.zip, .whl) and placed into the lib sub-directory of the distribution directory.
  • The configuration files for the application (from app\config) are copied into the config sub-directory of the distribution directory.
  • The README.html file from the app\doc\sdk directory is copied into the root of the distribution directory.
  • The distribution file <app_name>-python-dist-<version>.zip is created and includes the generated documentation, samples, libraries, configuration files, and README files for the application. This distribution file contains all of the components necessary to install and run the application along with documentation providing installation and usage instructions.
  • The distribution file <app_name>-python-dist-config-<version>.zip is created and only includes the configuration files for the application. This distribution file is useful in scenarios where the application is installed by means other than the full distribution (via a Docker image, PyPI, etc.).

At this point, the distribution file <app_name>-python-dist-<version>.zip can be distributed which contains the application libraries and documentation including installation and usage instructions.

Docker Support

The generated application includes a Dockerfile that can be used to build a Docker image of the application.

Build Image

To build the Docker image execute a command similar to the following from the root directory of the generated application:

docker build -t myapp:0.0.0 .

The arguments are as follows:

  • <image-name>:<image-tag>: The name for the created image and an associated tag (in the example above the image will be named myapp and have an associated tag of 0.0.0 which represents its version).
  • location of the Dockerfile: The location of the Dockerfile (in the example above . is specified since the command is executed from the root directory of the generated application which contains the Dockerfile).

Create Container

Execute a command similar to the following to create a Docker container based on the image created above:

docker run -d -v /home/myuser/myapp/config:/opt/myapp-config --name myapp myapp:0.0.0

The arguments are as follows:

  • -v <host-directory>:<container-data-volume>: Specifies a directory on the host that contains the configuration files (application, dxlclient, logging) for the application. The directory on the host is mapped to a virtual location in the container. The location in the container will take the form of /opt/<app_name>-config based on the name of the application specified in the configuration file used to generate the application.
  • --name <container-name>: A name for the container that is being created (in the example above myapp is specified as the container name).
  • <image-name>:<image-tag>: The image name and associated tag to create the container from (in the example above myapp:0.0.0 is specified which refers to the image that was created in the previous step).

View Container Log Output

Execute a command similar to the following to view the container log output:

docker logs myapp

The arguments are as follows:

  • <container-name>: The name of the container to view log information for (in the example above myapp was specified which refers to the container that was created in the previous step).

The following is an example of the log output:

Running application ...
On 'run' callback.
On 'load configuration' callback.
Incoming message configuration: queueSize=1000, threadCount=10
Message callback configuration: queueSize=1000, threadCount=10
Attempting to connect to DXL fabric ...
Connected to DXL fabric.
Registering event callback: eventhandler1
Registering service: service1
Registering request callback: service1_requesthandler1
On 'DXL connect' callback.
Event received on topic: '/some/dxl/event/topic' 
    with payload: 'eventhandler1 event payload'
Request received on topic: '/mycompany/service/myservice/request1' 
    with payload: 'service1_requesthandler1 request payload'

The output above shows that the application is running, has received a DXL event, and responded to a request that was received via an exposed DXL service.

NOTE: The structure of the generated application and associated Dockerfile is compatible with the automated builds feature of Docker Hub when used in conjunction with source repositories such as GitHub.

As an example of an automated build, see the Open DXL ePO Service which is automatically built via Docker Hub based on source code residing in GitHub.

Example Usage

The OpenDXL Bootstrap distribution contains an example "application" template configuration (application-template.config) which is located in the config directory. The example application registers a single DXL event listener and exposes a DXL service with a single request handler.

Install OpenDXL Bootstrap

The first step is to install the OpenDXL Bootstrap application.

  1. Ensure the OpenDXL Bootstrap application prerequisites have been met.

  2. Download the latest release of the OpenDXL Bootstrap application (dxlbootstrap-python-dist-<version>.zip).

  3. Extract the distribution file (.zip).

  4. Use pip to install the Python wheel (.whl) file located in the lib directory of the extracted distribution.

    pip install dxlbootstrap-<version>-py2.py3-none-any.whl

Generate Application

To generate the example application execute the following command:

python -m dxlbootstrap application-template config\application-template.config myapp-output

The arguments are as follows:

  • application-template: The type of template to use (in this case "application")
  • config\application-template.config: The location of the configuration file that provides information about the application that is to be generated
  • myapp-output: The location to output the generated application

The directory structure and files generated in the output directory (myapp-output) are described in detail in the output section, above.

Run Generated Application

Prior to running the generated application, the dxlclient.config file located in the myapp-output\config directory must be modified to include the information necessary to connect the application to the DXL fabric.

The steps to populate this configuration file are the same as those documented in the OpenDXL Python SDK, see the OpenDXL Python SDK Samples Configuration page for more information.

To run the application execute the following command from within the myapp-output directory:

python -m myapp config

The config argument specifies the directory that contains the application configuration which includes the dxlclient.config file that provides the information necessary to connect to the DXL fabric.

The output from running this command should appear similar to the following:

Running application ...
On 'run' callback.
On 'load configuration' callback.
Incoming message configuration: queueSize=1000, threadCount=10
Message callback configuration: queueSize=1000, threadCount=10
Attempting to connect to DXL fabric ...
Connected to DXL fabric.
Registering event callback: eventhandler1
Registering service: service1
Registering request callback: service1_requesthandler1
On 'DXL connect' callback.

As shown in the output above, a DXL event callback is registered (eventhandler1) and a DXL service (service1) is exposed that has an associated request handler (service1_requesthandler1).

At this point, leave the application running as it is required for the next step.

Run Generated Sample

A default sample is generated by the application template that will send DXL events to the topics the generated application is listening for. It will also invoke each service method that is exposed by the generated application. In this particular case it will send a DXL event to the topic associated with eventhandler1 and invoke the service's service1_requesthandler1 method.

Prior to running the generated sample, the dxlclient.config file located in the myapp-output\sample directory must be modified to include the information necessary to connect the sample to the DXL fabric.

The steps to populate this configuration file are the same as those documented in the OpenDXL Python SDK, see the OpenDXL Python SDK Samples Configuration page for more information.

To run the sample execute the following command from within the myapp-output directory:

python sample\basic\basic_sample.py

The output from running this command should appear similar to the following:

Response for service1_requesthandler1: 'service1_requesthandler1 response payload'

The output above shows that a response message was received for the request that was sent by the sample to the service that is exposed by the generated application.

The output for the application should appear similar to the following:

Event received on topic: '/some/dxl/event/topic' 
    with payload: 'eventhandler1 event payload'
Request received on topic: '/mycompany/service/myservice/request1' 
    with payload: 'service1_requesthandler1 request payload'

The output above shows that the event sent from the sample was received by the application's event handler. Further, it shows that the request sent by the sample was received by the application service's request handler.

You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.