CI Plugin SDK for ALM Octane

Introduction

Octane's CI Plugin Development Kit provides an easy and robust way to integrate CI Environment with ALM Octane application server. In vast majority of cases, SDK will be used as a part of a bigger integration unit - CI server's plugin/addon, therefore we'll be referring to the whole integration component as CI Plugin henceforth.

The SDK exposes several services allowing to push CI artifacts to ALM Octane (test results as of now, coverage, build logs and more in the future), where this data will undergo further analysis and linkage to the relevant application management context.

SDK's services are accessible via the methods exposed on OctaneSDK instance. For example, accessing EventsService would be: OctaneSDK.getInstance().getEventsService().

From the other side, SDK knows to establish steady connectivity (by means of long polling over HTTP/S protocol) with ALM Octane, thus being able to handle tasks submitted from ALM Octane and return results if/where relevant. There is a predefined set of tasks that are supported by such a flow (discover projects/jobs or job's/project's structure, trigger build etc).

In order to inter-operate with the 'hosting' CI environment, SDK employs a Service Provider Interface pattern (SPI). Currently SPI consists of a single interface, namely CIPluginServices, that is required to be implemented by an SDK's consumer and instance of which is a must parameter of SDK's initialization method. Data provided by CIPluginServices implementation will be used by SDK in order to connect to the ALM Octane, notify about server/plugin info and rest of functionalities.

Typical SDK-based ALM Octane integrated CI Plugin development would walk along the following path:

1. Including the SDK into the CI Plugin (be it new or an existing one) as maven dependency
2. Implementing an SPI interface/s (or extending CIPluginServicesBase abstract class, convenient for partial implementation)
3. Adding SDK initialization to the initialization flow of the CI Plugin
4. Hooking into the specific CI environment's event mechanism in order:
   1. to push builds' events to ALM Octane
   2. to push builds' artifacts (tests results etc) to ALM Octane

This Java SDK project has two sub-projects:

• integrations-sdk, which is the main source of the CI Plugin SDK.
• integrations-dto, which contains the definition and building factory of all DTO objects used for communication with ALM Octane.

See the Javadoc for more information about the CI Plugin SDK API.

Compiling the Project

The easiest way to compile the project is to use Maven and run the following command from the root directory:

mvn clean install

Creating JavaDoc

To create Javadoc, run the following Maven command from the project root directory:

mvn javadoc:aggregate

This creates a javadoc site in the /target/site/apidocs/index.html

Include in Your Project

Add the following dependency to the pom.xml to use this SDK in your project:

<dependency>
    <artifactId>integrations-sdk</artifactId>
    <groupId>com.hpe.adm.octane.plugins</groupId>
    <version>1.0</version>
</dependency>

Usage Examples

The following CI Plugins are already using CI Plugin SDK for ALM Octane to connect and communicate with ALM Octane:
Octane Bamboo Plugin
Octane TeamCity Plugin

Initialization

To start using the CI Plugin SDK, first initialize an OctaneSDK instance. This class provides the main entry point of interaction between the SDK and its services, and interaction between the concrete CI Plugin and its services:

OctaneSDK.init(new MyPluginServices(), true);

The init() method expects to get a valid implementation of the CIPluginServices SPI. This object is actually a composite API of all the endpoints to be implemented by a hosting CI Plugin for an ALM Octane use cases.

Second parameter, boolean, reads: 'establish connectivity with ALM Octane'; it always should be set to true unless there are very specific need to make an offline integration (which may be achieved by providing empty OctaneConfiguration from SPI as well, so still no excuses for false here).

Communicating with ALM Octane using Data Transfer Objects (DTO)

Data transfer object (DTO) is a design pattern used to transfer data between software application subsystems. We use DTO objects to communicate data to ALM Octane.

Any DTO in the system should be created using DTOFactory:

T dto = DTOFactory.getInstance().newDTO(Class<T> targetType);

Updating ALM Octane with CI Events

Most/all of the CI servers provide a means to hook into their own events system, thus providing convenient extensibility points for any addon/plugin willing to leverage the CI data further. Upon a CI event, the CI Plugin should update ALM Octane using the EventsService. The steps are:

1. Subscribe to CI events by means of underlying/hosting CI system
2. Create ALM Octane's internal CIEvent DTO from the data provided by the CI system
3. Provide this DTO to the publishEvent method of EventsService:

CIEvent ciEvent = DTOFactory.getInstance().newDTO(CIEvent.class)
      .setEventType(eventType)
      .setCauses(causes)
      .setProject(project)
      .setProjectDisplayName(displayName)
      .setBuildCiId(buildCiId)
      .setEstimatedDuration(estimatedDuration)
      .setStartTime(startTime)
      .setPhaseType(phaseType);

OctaneSDK.getInstance().getEventsService().publishEvent(ciEvent);

One of the capabilities of CIEvent is to transfer SCM data as part of it. SCM data provided by most/all of the CI servers taking it directly from the Source Control Management tool (Git, Mercurial etc) and mostly comprises from all of the commits that was introduced into the code and built in the inspected build (aka diff from the last commit of the previous build)/ SCM data can greatly leverage integration with ALM Octane, which knows to analyze it, link it's contents to relevant Work Items (by commit patterns), provide statistical insights, failure analysis and more of it.

Pay attention, that although SCM data may be posted in the later stages of the build/pipeline run, it is always better to push the data to ALM Octane as soon as it available. Add an SCM data to the event in the following way:

ciEvent.setScmData(scmData);

If at the point of time, where some CI event is being pushed to ALM Octane, CI Plugin already knows that tests results are/will surely be available for this build, notifying ALM Octane about this would help to improve the performance and responsiveness of the flow. To do so, please use the following API of CIEvent DTO:

ciEvent.setTestResultExpected(true);

ALM Octane Pipeline Structure

ALM Octane pipelines represent the flow of the CI server jobs and steps. Pipeline provides a clear, multi-level, analytic view of specific CI process, CI runs and their status so you can track product quality and progress.

ALM Octane pipeline supports a hierarchical structure of pipeline nodes. Any PipelineNode can contain a list of internal phases and list of post-build phases. Each PipelinePhase in turn contains a list of child pipeline nodes.

Pipeline nodes in internal phases represent CI nodes that complete their execution before post-build phases start to execute. Each internal phase should contain nodes that are running in parallel.

This structure is used for correct pipeline representation in ALM Octane. This way in ALM Octane, it is possible to see the flow of steps in the CI server, including which steps run in sequence and which steps run as parts of other steps.

Providing CI Build Information

The ALM Octane server may ask for a specific build information of some pipeline. To provide the information, implement two CIPluginServices methods:

SnapshotNode getSnapshotByNumber(String ciJobId, String buildCiId, boolean subTree)

This provides a snapshot of the specified build of the specified job.

SnapshotNode getSnapshotLatest(String ciJobId, boolean subTree)

This provides a snapshot of the latest build of the specified job.

The main DTOs for CI build snapshots are SnapshotNode and SnapshotPhase. These provide the same hierarchical structure described in the Pipeline Structure section and allow the ALM Octane user to see the build number, the last run date, the run status, and the duration of the run.

Providing Test Results

We are about CI events hooking again. Once SDK driven integration got some CI event where it can safely retrieve tests results of a build - usually that would be a finished/completed event of any build, thus we are talking on the level of a single build - it should push tests results to ALM Octane.

The preferred flow is somewhat asynchronous and it is crucial to understand it, so here is the detailed description:

1. When tests result are fully accessible, CI Plugin should ensure, that those may be identified and accessed later on; for identification we use a notion of jobCiId and buildCiId which are a simple strings; pay attention, that in some/most CI systems tests results (with their full data, like duration, exceptions etc) are not available for a prolonged period of time, in such a cases it is required to store this data somewhere for later retrieval
2. Next, CI Plugin should 'notify' the SDK that tests results are available for such a buildCiId of such a jobCiId, whatever they are, using the following API:

   OctaneSDK.getInstance().getTestsService().enqueuePushTestsResult(someJobCiId, someBuildCiId);

3. SDK will enqueue this info internally (not persisted, as of now) and somewhat later, usually almost immediately, but it depends on the system load of course, turn back to an SPI and will ask for a full report of tests results of the said above jobCiId/buildCiId; CI Plugin is expected to be able to retrieve the correct tests results (probably stored somewhere, as we mentioned in punkt 1 above) and return them; to be sure, relevant SPI method is:

   TestsResult getTestsResult(String jobCiId, String buildCiId);

   TestResult DTO should contain a list of all test runs (TestRun) that were executed in the specified build of the specified job. Each TestRun object represents a run of a single test. It contains all information about the test, run's result (TestRunResult enum) and the error information if failed. URL to the test run report page can be provided with the setExternalReportUrl() method.

4. SDK will further perform the actual push to the ALM Octane

Another, less preferred way to push tests results to ALM Octane is a synchronous push, which may be done by calling the below API somewhere within punkt 1 above:

OctaneSDK.getInstance().getTestsService().pushTestsResult(pushTestsResult);

Pay attention, that usually executing time consuming actions within the CI event call effectively means holding the main CI system execution thread, since most of the CI system's events are executing on the main thread. Don't do that.

Disclamer update

Certain versions of software accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company.  As of September 1, 2017, the software is now offered by Micro Focus, a separately owned and operated company.  Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.