Skip to content

Commit

Permalink
OCE -> ALEC for docs.
Browse files Browse the repository at this point in the history
  • Loading branch information
Jesse White committed Apr 12, 2019
1 parent 9c21874 commit 1db26e3
Show file tree
Hide file tree
Showing 21 changed files with 90 additions and 84 deletions.
4 changes: 2 additions & 2 deletions docs/antora.yml
@@ -1,5 +1,5 @@
name: oce
name: alec
version: '1.0.0-SNAPSHOT'
title: OpenNMS Correlation Engine
title: ALEC
nav:
- modules/ROOT/nav.adoc
Binary file removed docs/modules/ROOT/assets/images/model.png
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
18 changes: 12 additions & 6 deletions docs/modules/ROOT/pages/index.adoc
@@ -1,16 +1,22 @@
= User's Guide to the _OpenNMS Correlation Engine_
= User's Guide to the _Architecture for Learning Enabled Correlation_
:page-layout: home
:!sectids:

Welcome to the OpenNMS Correlation Engine! This guide is intended to provide users and administrators with everything that is needed to conceptually understand and deploy some of the most advanced software concepts and components ever created for OpenNMS.
Welcome to the Architecture for Learning Enabled Correlation (ALEC)!
This guide is intended to provide users and administrators with everything that is needed to conceptually understand and deploy some of the most advanced software concepts and components ever created for OpenNMS.

These new components were built from several years of research and development. Additionally, they have been written to take full advantage of being deployed on top of a great many new architectural improvements that have recently been made to the platform. In the tradition of OpenNMS development, these architectural changes have been created following a continuous improvement (CI) strategy; as opposed to starting from scratch with a complete rewrite (oh, so tempting!). This technique has proved itself well for continuity within the community as well as providing a very stable management platform; two decades in the making.
These new components were built from several years of research and development.
Additionally, they have been written to take full advantage of being deployed on top of a great many new architectural improvements that have recently been made to the platform.
In the tradition of OpenNMS development, these architectural changes have been created following a continuous improvement (CI) strategy; as opposed to starting from scratch with a complete rewrite (oh, so tempting!).
This technique has proved itself well for continuity within the community as well as providing a very stable management platform; two decades in the making.

The developer community has spent several years in R&D as well as working with a few large corporate sponsors to realize these required architecture improvements. They have also played a significant role vetting the machine learning (ML) algorithms and deep learning (DL) technologies that best fit OpenNMS and that best correlated problems detected by their OpenNMS management systems.
The developer community has spent several years in R&D as well as working with a few large corporate sponsors to realize these required architecture improvements.
They have also played a significant role vetting the machine learning (ML) algorithms and deep learning (DL) technologies that best fit OpenNMS and that best correlated problems detected by their OpenNMS management systems.

Following this guide, the user will gain sufficient knowledge about the correlation engine (CE) and the algorithms it uses to get started with transitioning from resolving Alarms (problems) to better managing the actual more complex Situations that they often represent.

As a caution to the "new to OpenNMS" readers, many assumptions have been made about the reader having a basic understanding of OpenNMS as some of these CE concepts and supporting technologies are quite complex in their own right. However, this is a living document and every effort is being made to enable organizations to benefit from the many values provided by the CE.
As a caution to the "new to OpenNMS" readers, many assumptions have been made about the reader having a basic understanding of OpenNMS as some of these CE concepts and supporting technologies are quite complex in their own right.
However, this is a living document and every effort is being made to enable organizations to benefit from the many values provided by the CE.


For an overview of OCE start with: xref:about:welcome.adoc[What is OCE?]
For an overview of ALEC start with: xref:about:welcome.adoc[What is ALEC?]
8 changes: 4 additions & 4 deletions docs/modules/about/pages/welcome.adoc
@@ -1,9 +1,9 @@
= OpenNMS Correlation Engine (OCE) Documentation
= Architecture for Learning Enabled Correlation (ALEC) Documentation
:imagesdir: ../assets/images

== What is OCE?
== What is ALEC?

The OpenNMS Correlation Engine (OCE) is a framework for logically grouping related faults (alarms) into higher level objects (situations) with OpenNMS.
The Architecture for Learning Enabled Correlation (ALEC) is a framework for logically grouping related faults (alarms) into higher level objects (situations) with OpenNMS.

== How does it work?

Expand All @@ -14,7 +14,7 @@ Nodes, their components and their relations are used to build a graph.
image::model.png[Model example,600]

The layout of the graph, and the different types of elements on the graph are what compose "the meta-model".
In OCE, the vertices on the graph are referred to as "inventory objects" and edges are used to represent "relationships".
In ALEC, the vertices on the graph are referred to as "inventory objects" and edges are used to represent "relationships".

=== Alarms are attached to the graph

Expand Down
10 changes: 5 additions & 5 deletions docs/modules/architecture/pages/opennms_integration.adoc
@@ -1,7 +1,7 @@
= OpenNMS Integration
:imagesdir: ../assets/images

The following diagram depicts the various components involved in the life-cycle of an alarm and correlation with OpenNMS 24.0.0 and OCE:
The following diagram depicts the various components involved in the life-cycle of an alarm and correlation with OpenNMS 24.0.0 and ALEC:

image::opennms_alarm_lifecycle.png[Alarm lifecyle,600]

Expand All @@ -15,12 +15,12 @@ OCE leverages these hooks to perform additional processing and *tag* the alarms
See the <<managed_object_tagging>> section bellow for more details.

OpenNMS also provides facilities to be notified when alarms are in some way modified.
The <<direct_datasource>> leverages API hooks directly into the run-time, whereas the <<kafka_datasource>> leverages the Kafka integration for interfacing externally.
The xref:datasources:direct.adoc[direct datasource] leverages API hooks directly into the run-time, whereas the xref:datasources:kafka.adoc[Kafka datasource] leverages the Kafka integration for interfacing externally.

[[managed_object_tagging]]
== Managed Object Tagging

Managed object tagging allows alarms to be further contextualized by associating them with a specific component instance (or inventory object in OCE).
Managed object tagging allows alarms to be further contextualized by associating them with a specific component instance (or inventory object in ALEC).

The process is flexible and configurable, allowing alarms to be associated with components beyond the existing node/interface/service model built-in to OpenNMS.

Expand Down Expand Up @@ -68,10 +68,10 @@ The xref:reference:model.adoc#snmp-interface-link[Model reference] shows that th
3. zHostname

Referring to the syslog match definition above, we can see that we extract all of these tokens from the syslog message and convert them to event parameters.
When the alarm is created, the link:https://github.com/OpenNMS/oce/blob/4e5ae2cd6271541421c3af1b4b4aafd49ace9df3/integrations/opennms/extension/src/main/java/org/opennms/oce/opennms/extension/ManagedObjectAlarmExt.java[ManagedObjectAlarmExt] extension gets triggered, and will go through the process of resolving and canonicalizing the managed object.
When the alarm is created, the link:https://github.com/OpenNMS/alec/blob/4e5ae2cd6271541421c3af1b4b4aafd49ace9df3/integrations/opennms/extension/src/main/java/org/opennms/oce/opennms/extension/ManagedObjectAlarmExt.java[ManagedObjectAlarmExt] extension gets triggered, and will go through the process of resolving and canonicalizing the managed object.

In this case, we search for a node with the same label as given in the `zHostname` attribute, and attempt to the ifIndex for SNMP interfaces with the given ifDescrs.
We then update the managed object instance on the alarm to include the resolved attributes.
When we need to store multiple attributes in managed object instance field we encode these in JSON so that they can be represented as a single string.

When the alarm is received by OCE, it now contains sufficient information in the MO type and instance fields so that it can be attached to a corresponding inventory object.
When the alarm is received by ALEC, it now contains sufficient information in the MO type and instance fields so that it can be attached to a corresponding inventory object.
4 changes: 2 additions & 2 deletions docs/modules/architecture/pages/overview.adoc
@@ -1,9 +1,9 @@
= OCE Architecture
= ALEC Architecture
:imagesdir: ../assets/images

== Overview

The OCE framework provides building blocks for developing correlation systems:
ALEC provides building blocks for developing correlation systems:

image::architecture_components.png[Components,600]

Expand Down
10 changes: 5 additions & 5 deletions docs/modules/datasources/pages/direct.adoc
Expand Up @@ -3,14 +3,14 @@

== Overview

The OpenNMS Direct Datasource is used when OCE is deployed in a *monolithic* installation.
The OpenNMS Direct Datasource is used when ALEC is deployed in a *monolithic* installation.
It runs in the same JVM as OpenNMS and acts upon new *Alarms* and *Nodes* using the APIs provided by the link:https://github.com/OpenNMS/opennms-integration-api[OIA].

== Scripted extensions

=== Config

To update the bundle configuration and point to a script on the file system, you must update the _scriptFile_ property of the _org.opennms.oce.datasource.opennms.direct_ PID.
To update the bundle configuration and point to a script on the file system, you must update the _scriptFile_ property of the _org.opennms.alec.datasource.opennms.direct_ PID.

```
config:edit org.opennms.oce.datasource.opennms.direct
Expand All @@ -24,7 +24,7 @@ There are three methods exposed to the InventoryObject script: One is Alarm focu

==== Alarms

When an Alarm from OpenNMS arrives via the DirectAlarmDatasource, it is directly mapped from its OpenNMS type to the corresponding OCE type.
When an Alarm from OpenNMS arrives via the DirectAlarmDatasource, it is directly mapped from its OpenNMS type to the corresponding ALEC type.
During this mapping, you can modify the ModelObjectType or the ModelObjectIdentifier of the InventoryObject that the Alarm will be associated with.

===== overrideTypeAndInstance
Expand All @@ -35,7 +35,7 @@ void overrideTypeAndInstance(ImmutableAlarm.Builder alarmBuilder, Alarm alarm);

==== Inventory

As Nodes arrive via the DirectInventoryDatasource and are Mapped from their OpenNMS type to the internal OCE type, there is an opportunity to derive InventoryObjects.
As Nodes arrive via the DirectInventoryDatasource and are Mapped from their OpenNMS type to the internal ALEC type, there is an opportunity to derive InventoryObjects.

===== createInventoryObjects - Node

Expand All @@ -50,7 +50,7 @@ If there is a *Node* in the node attribute of the alarm, the above method will b

===== createInventoryObjects - Alarm

As each *Alarm* from OpenNMS is received by OCE, there is an opportunity to derive InventoryObjects.
As each *Alarm* from OpenNMS is received by ALEC, there is an opportunity to derive InventoryObjects.

```
List<InventoryObject> alarmToInventory(Alarm alarm);
Expand Down
8 changes: 4 additions & 4 deletions docs/modules/datasources/pages/kafka.adoc
Expand Up @@ -3,21 +3,21 @@

== Overview

The OpenNMS Kafka Datasource is used when OCE is deployed in a *distributed* installation.
The OpenNMS Kafka Datasource is used when ALEC is deployed in a *distributed* installation.

The Kafka data source integrates with OpenNMS by leveraging the Kafka Producer and Kafka Event Consumer features.

The Kafka Producer feature is used to forward alarms, nodes and topology data encoded in link:https://developers.google.com/protocol-buffers/[protocol buffers] to several topics.
The Kafka Datasource consumes from these topics and provides OCE with a feed of alarms and inventory.
The Kafka Datasource consumes from these topics and provides ALEC with a feed of alarms and inventory.

== Scripted extensions

=== Config

To update the bundle configuration and point to a script on the file system, you must update the _scriptFile_ property of the _org.opennms.oce.datasource.opennms.kafka_ PID.
To update the bundle configuration and point to a script on the file system, you must update the _scriptFile_ property of the _org.opennms.alec.datasource.opennms.kafka_ PID.

```
config:edit org.opennms.oce.datasource.opennms.kafka
config:edit org.opennms.alec.datasource.opennms.kafka
config:property-set scriptFile /tmp/inventory.groovy
config:update
```
Expand Down
6 changes: 3 additions & 3 deletions docs/modules/datasources/pages/overview.adoc
Expand Up @@ -16,13 +16,13 @@ The mapping of *Alarms*, *Nodes* and *Links* to *Inventory Objects* is done usin

The default implementation of this mapping can be found in the _inventory.groovy_ script of each bundle:

* link:https://github.com/OpenNMS/oce/tree/master/datasource/opennms-direct/src/main/resources/inventory.groovy[OpenNMS Direct]
* link:https://github.com/OpenNMS/alec/tree/master/datasource/opennms-direct/src/main/resources/inventory.groovy[OpenNMS Direct]

* link:https://github.com/OpenNMS/oce/tree/master/datasource/opennms-kafka/src/main/resources/inventory.groovy[OpenNMS Kafka]
* link:https://github.com/OpenNMS/alec/tree/master/datasource/opennms-kafka/src/main/resources/inventory.groovy[OpenNMS Kafka]

The included implementation uses the link:http://groovy-lang.org/[Groovy] scripting language.

IMPORANT: Ensure to reference the script that corresponds to the same version of OCE that you are running.
IMPORANT: Ensure to reference the script that corresponds to the same version of ALEC that you are running.

To modify the behaviour of the current implementation, you may begin with one of the scripts above.
See the appropriate datasource page for specific configuration and API information.
Expand Down
8 changes: 4 additions & 4 deletions docs/modules/install/pages/common_install.adoc
@@ -1,7 +1,7 @@
= Common Install
:imagesdir: ../assets/images

This guide contains instructions for installing _OCE_ that are common to both the monolithic and distributed installs.
This guide contains instructions for installing _ALEC_ that are common to both the monolithic and distributed installs.

== OpenNMS Configuration

Expand Down Expand Up @@ -53,7 +53,7 @@ Everything is awesome.

=== Enable alarm history storage for Elasticsearch

When using OCE, we strongly recommend that leverage our integration for maintaining alarm history in Elasticsearch.
When using ALEC, we strongly recommend that leverage our integration for maintaining alarm history in Elasticsearch.
This data can be used for analysis and to help train the correlation engine.

From the OpenNMS Karaf shell, run:
Expand Down Expand Up @@ -100,11 +100,11 @@ to:

This change will come in effect next time OpenNMS is restarted - we don't need to do this yet.

== OCE Installation
== ALEC Installation

=== Setup YUM repository

OCE packages are currently available via a YUM repository hosted on link:https://packagecloud.io/opennms/plugin-snapshot[packagecloud].
ALEC packages are currently available via a YUM repository hosted on link:https://packagecloud.io/opennms/plugin-snapshot[packagecloud].

You can install the repository using instructions available on the link above, or by running:
```
Expand Down
22 changes: 11 additions & 11 deletions docs/modules/install/pages/distributed_install.adoc
@@ -1,7 +1,7 @@
= Distributed Install
:imagesdir: ../assets/images

This installation guide describes a distributed install where we run _OCE_ as a service on top of the _Sentinel_ runtime:
This installation guide describes a distributed install where we run _ALEC_ as a service on top of the _Sentinel_ runtime:

image::distributed_deployment.png[Distributed,700]

Expand Down Expand Up @@ -46,7 +46,7 @@ echo 'org.opennms.netmgt.eventd.sink.enable=true' > "$OPENNMS_HOME/etc/opennms.p

IMPORTANT: We assume that you're already using the Kafka strategy for the Sink API. See <<prequisites>>.

== Install OCE in OpenNMS
== Install ALEC in OpenNMS

On the OpenNMS server, install the package:
```
Expand All @@ -55,10 +55,10 @@ yum -y install opennms-alec-plugin

Configure the feature set to be installed when OpenNMS starts:
```
echo 'oce-opennms-distributed wait-for-kar=opennms-alec-plugin' > "$OPENNMS_HOME/etc/featuresBoot.d/oce.boot"
echo 'alec-opennms-distributed wait-for-kar=opennms-alec-plugin' > "$OPENNMS_HOME/etc/featuresBoot.d/alec.boot"
```

== Install OCE in Sentinel
== Install ALEC in Sentinel

On the Sentinel systems, install the package:
```
Expand All @@ -68,19 +68,19 @@ yum -y install sentinel-alec-plugin
Configure the Kafka Stream (consumer):
```
echo 'bootstrap.servers=127.0.0.1:9092
commit.interval.ms=5000' > "$SENTINEL_HOME/etc/org.opennms.oce.datasource.opennms.kafka.streams.cfg"
commit.interval.ms=5000' > "$SENTINEL_HOME/etc/org.opennms.alec.datasource.opennms.kafka.streams.cfg"
```

Configure the Kafka producer:
```
echo 'bootstrap.servers=127.0.0.1:9092' > "$SENTINEL_HOME/etc/org.opennms.oce.datasource.opennms.kafka.producer.cfg"
echo 'bootstrap.servers=127.0.0.1:9092' > "$SENTINEL_HOME/etc/org.opennms.alec.datasource.opennms.kafka.producer.cfg"
```

TIP: The consumer is used to read alarms & inventory, whereas the producer is used to send events.

Configure the feature set to be installed when Sentinel starts:
```
echo 'oce-sentinel-distributed wait-for-kar=opennms-alec-plugin' >> "$SENTINEL_HOME/etc/featuresBoot.d/oce.boot"
echo 'alec-sentinel-distributed wait-for-kar=opennms-alec-plugin' >> "$SENTINEL_HOME/etc/featuresBoot.d/alec.boot"
```

== Verify
Expand All @@ -91,13 +91,13 @@ Run the `health:check` commands in both OpenNMS and Sentinel.

From a Sentinel Karaf shell, enumerate the available graphs using:
```
oce:list-graphs
opennms-alec:list-graphs
```

At this point we expect the command output a single graph called "cluster":
At this point we expect the command output a single graph called "dbscan":
```
admin@opennms> oce:list-graphs
cluster: 0 situations on 524 vertices and 4 edges.
admin@opennms> opennms-alec:list-graphs
dbscan: 0 situations on 524 vertices and 4 edges.
```

It's okay if there are no situations, vertices or edges on the graph at this point.
Expand Down
16 changes: 8 additions & 8 deletions docs/modules/install/pages/monolithic_install.adoc
@@ -1,15 +1,15 @@
= Monolithic Install
:imagesdir: ../assets/images

This installation guide describes a monolithic install where we run _OCE_ as another service within the _OpenNMS_ core:
This installation guide describes a monolithic install where we run _ALEC_ as another service within the _OpenNMS_ core:

image::monolithic_deployment.png[Monolithic,400]

== Common Setup

Start by following the instructions in the xref:common_install.adoc[common install] section if you have not already done so.

== Install OCE
== Install ALEC

Install the package:
```
Expand All @@ -18,12 +18,12 @@ yum -y install opennms-alec-plugin

Configure the feature set to be installed when OpenNMS starts:
```
echo 'oce-opennms-standalone wait-for-kar=opennms-alec-plugin' > "$OPENNMS_HOME/etc/featuresBoot.d/oce.boot"
echo 'alec-opennms-standalone wait-for-kar=opennms-alec-plugin' > "$OPENNMS_HOME/etc/featuresBoot.d/alec.boot"
```

To install the feature set immediately without restarting, login to the OpenNMS Karaf shell and run:
```
feature:install oce-opennms-standalone
feature:install alec-opennms-standalone
```

NOTE: We bundle many common features up together for ease of use, but you can also install these individually if for example, you would like to run using a different _engine_.
Expand All @@ -34,13 +34,13 @@ Run the health check again to verify that everything was loaded correctly using

You can enumerate the available graphs using:
```
oce:list-graphs
opennms-alec:list-graphs
```

At this point we expect the command output a single graph called "cluster":
At this point we expect the command output a single graph called "dbscan":
```
admin@opennms> oce:list-graphs
cluster: 0 situations on 524 vertices and 4 edges.
admin@opennms> opennms-alec:list-graphs
dbscan: 0 situations on 524 vertices and 4 edges.
```

It's okay of there are no situations, vertices or edges on the graph at this point.
Expand Down

0 comments on commit 1db26e3

Please sign in to comment.