Backend Plugins

Oliver Gorwits edited this page Nov 27, 2017 · 11 revisions

Introduction

This document is a technical description of the operation and design of Netdisco’s backend plugin system. As there are new terms to learn, and subtleties in the logic, we have a Cookbook, and you can also look at Netdisco’s core plugins themselves.

Overview

Netdisco’s plugin system allows you to alter, add to, remove, or override, components of the backend daemon’s activity. Plugins can be distributed independently from Netdisco to add local functionality. They also integrate fully with the scheduler and command-line netdisco-do app.

Typically, plugin workers gather information from network devices using transports (such as SNMP, SSH, or HTTPS) and store results in the database. Workers combine transports with relevant application protocols such as SNMP, NETCONF (OpenConfig with XML), RESTCONF (OpenConfig with JSON), eAPI, or even CLI scraping.

The combination of transport and protocol is known as a driver. With familar ACL syntax, workers can be restricted to certain vendor platforms, particular drivers, and specific actions in Netdisco’s backend operation.

When a Netdisco action is run (discover, macsuck, etc), all the relevant plugins are loaded and their workers registered, and then run in a particular order according to simple conventions.

Registering a Worker

A worker is Perl code that is registered to Netdisco from within a plugin package. The worker can do anything you like, and will be run when its parent action (determined by the package name) is invoked by Netdisco’s scheduler or the netdisco-do command-line app.

App::Netdisco plugins must load the App::Netdisco::Worker::Plugin module. This exports a helper subroutine to register your worker(s). Here’s the boilerplate code for our example plugin module:

package App::Netdisco::Worker::Plugin::Discover::Wireless::UniFi;

use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';

# worker registration code goes here, ** see below **

true;

Use the register_worker helper from App::Netdisco::Worker::Plugin to register a worker:

register_worker( $coderef );
# or
register_worker( \%workerconf, $coderef );

For example (using the second form):

register_worker({
  driver => 'unifiapi',
}, sub { "worker code here" });

The %workerconf hashref is optional, and described below. The $coderef is the main body of your worker. You can register more than one worker in a packge, each is run within a Try::Tiny statement to catch errors, and is passed the following arguments:

$coderef->($job, \%workerconf);

The $job is an instance of App::Netdisco::Backend::Job. Note that this class has a device slot which may be filled, depending on the action, and if the device is not yet discovered then the row will not yet be in storage. The \%workerconf hashref is what was passed in the registration of the worker (if anything), plus some other useful data, and is documented below.

Package Naming Convention

The package name used where the worker is declared is significant. Let’s look at the boilerplate example again:

package App::Netdisco::Worker::Plugin::Discover::Wireless::UniFi;

The package name must contain Plugin:: and the namespace component after that becomes the action. For example, workers registered in the above package will be run during a discover job. You can replace Discover with other actions such as Macsuck, Arpnip, Expire, and Nbtstat, or create your own.

The package namespace component following the action (Wireless in this example) is one stage of the action. The worker’s code is registered within this stage, and you can override, or add to, any of the core stages in Netdisco.

Looking at the core arpnip action, we have two stages: nodes for gathering ARP tables and subnets to gather directly connected prefixes on the router:

App::Netdisco::Worker::Plugin::Arpnip;
App::Netdisco::Worker::Plugin::Arpnip::Nodes;
App::Netdisco::Worker::Plugin::Arpnip::Subnets;

Workers may also be registered directly to the parent action. This namespace is typically reserved for "checking" code that will abort the whole action if certain conditions are not met (such as user configuration blocking the action for the target device), or for very simple single-stage actions such as psql or expire.

%workerconf Settings

Along with your worker $coderef can be a %workerconf HASH reference. All settings are optional:

  • Access Control Lists

Workers may have only and no parameters configured which use the standard ACL syntax described in Access Control Lists. The only directive is especially useful as it can restrict a worker to a given device platform or operating system. For example:

register_worker({
  only => ['vendor:cisco', 'os:ios-xr'], # Cisco IOS-XR
}, sub { "worker code here" });
  • phase (string)

Running an action is separated by Netdisco into four phases: check, early, main, and user (in that order). Worker code is registered to a phase, or to the user phase if none is specified.

register_worker({
  phase => 'main',
}, sub { "worker code here" });

For example the check phase would abort a whole action if it is blocked by user configuration. The early phase allows initial setup and data gathering, followed by most work taking place in the main phase. The user phase is not used by Netdisco and is reserved for your own post-processing workers.

  • driver (string)

The driver is a label associated with a group of workers and typically refers to the combination of transport and application protocol. Examples include snmp, netconf, restconf, eapi, and cli. The convention is for driver names to be lowercase.

register_worker({
  driver => 'snmp',
}, sub { "worker code here" });

Users will bind authentication configuration settings to drivers in their configuration. If no driver is specified when registering a worker, it will be run for every device (such as during Expire jobs).

  • priority (integer)

Workers get assigned a priority value so that worker code with the same action, stage (package name), driver, and phase, can be overridden or added to. The priority is also used to pick the best driver if more than one is available.

register_worker({
  priority => 120,
}, sub { "worker code here" });

The priority is set from the driver (Netdisco knows what the best drivers are), or else defaults to zero. The Worker Cookbook gives examples of how to use the priority to add, chain, or override Netdisco actions.

Writing Workers

You can register more than one worker subroutine in a packge, and each is run within a Try::Tiny statement to catch errors.

Worker Return Status

The return value of the worker is significant, as it indicates to Netdisco whether to continue, branch, or abort the running of an action. You should either return nothing, or an instance of the aliased App::Netdisco::Worker::Status helper (loaded as in the boilerplate above):

return Status->done('a success that represents the complete action');
# or
return Status->info('a success but not the primary goal of the action');
# or
return Status->defer('could not connect to device for any reason');
# or
return Status->error('something went wrong');

Accessing Transports

From your worker you will want to connect to a device to gather data. This is done using a transport protocol session (SNMP, SSH, etc). Transports are singleton objects instantiated on demand, so they can be shared among a set of workers that are accessing the same device.

See the documentation for each transport to find out how to access it:

Database Connections

The Netdisco database is available via the netdisco schema key, as below. You can also use the external_databases configuration item to set up connections to other databases.

# plugin package
use Dancer::Plugin::DBIC;
my $set = schema('netdisco')->resultset('Devices')
                            ->search({vendor => 'cisco'});

Worker Execution Order

Given all of the above (action, stage, phase, driver, and priority), Netdisco builds an ordered list of workers to be run.

Perl Packages matching the action name in the extra_worker_plugins setting and then the worker_plugins setting are loaded. The order does not affect stages, which are sorted alphabetically, but does affect workers of equal priority within a stage. This is a useful subtlety which is described more in the Worker Cookbook.

The four phases (check, early, main, and user) are run in turn, with workers from each stage and in descending order of priority. The priority is either given in the worker’s configuration or determined from the driver (each driver maps to a priority value), or is zero.

If there are check phase workers but none of them returns a done() status, then the whole action is aborted. Similarly, if higher priority workers return done(), then lower ones are not run. The outcome of the action overall is then the best return value (Status instance) from all the workers that have run.

You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.