Skip to content
This repository has been archived by the owner on Nov 8, 2022. It is now read-only.

update documentation content #26

Merged
merged 1 commit into from
Oct 12, 2016
Merged

update documentation content #26

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
95 changes: 12 additions & 83 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,97 +1,26 @@
# Snap Plugin Library for Go
## Snap Plugin Library for Go

This is a library for writing plugins for the [snap](https://github.com/intelsdi-x/snap) telemetry framework. The goal for this library is to make it super simple to write a plugin.
This is a library for writing plugins in Go for the [Snap telemetry framework](https://github.com/intelsdi-x/snap).

# Writing a plugin
Snap has three different plugin types and for instructions on how to write a plugin check out the [collector](examples/collector/README.md), [processor](examples/processor/README.md), and [publisher](examples/publisher/README.md) plugin docs.

In order to write a plugin for snap, it is necessary to define some methods to satisfy the appropriate interfaces. The interface is slightly different depending on what type (collector, processor, or publisher) of plugin is being written. The interfaces are:
Before writing a Snap plugin:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It'd be best to give people the choice of writing a plugin right away. This additional context is excellent, though too much at this point. Some of it may go well under a ##a brief explanation of snap architecture or something like that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, added a separate sub-section


* See if one already exists in the [Plugin Catalog](https://github.com/intelsdi-x/snap/blob/master/docs/PLUGIN_CATALOG.md)
* See if someone mentioned it in the [plugin wishlist](https://github.com/intelsdi-x/snap/labels/plugin-wishlist)

```go
type Plugin interface {
GetConfigPolicy() (ConfigPolicy, error)
}
If you do decide to write a plugin, open a new issue following the plugin [wishlist guidelines](https://github.com/intelsdi-x/snap/blob/master/docs/PLUGIN_CATALOG.md#wish-list) and let us know you are working on one!

// Collector is a plugin which is the source of new data in the Snap pipeline.
type Collector interface {
Plugin

GetMetricTypes(Config) ([]Metric, error)
CollectMetrics([]Metric) ([]Metric, error)
}
## Brief Overview of Snap Architecture

// Processor is a plugin which filters, agregates, or decorates data in the
// Snap pipeline.
type Processor interface {
Plugin
Snap is an open and modular telemetry framework designed to simplify the collection, processing and publishing of data through a single HTTP based API. Plugins provide the functionality of collection, processing and publishing and can be loaded/unloaded, upgraded and swapped without requiring a restart of the Snap daemon.

Process([]Metric, Config) ([]Metric, error)
}
A Snap plugin is a program that responds to a set of well defined [gRPC](http://www.grpc.io/) services with parameters and return types specified as protocol buffer messages (see [plugin.proto](https://github.com/intelsdi-x/snap/blob/master/control/plugin/rpc/plugin.proto)). The Snap daemon handshakes with the plugin over stdout and then communicates over gRPC.

// Publisher is a sink in the Snap pipeline. It publishes data into another
// System, completing a Workflow path.
type Publisher interface {
Plugin

Publish([]Metric, Config) error
}
```
## Snap Plugin Go Library Examples
You will find [example plugins](examples) that cover the basics for writing collector, processor, and publisher plugins in the examples folder.

# Starting a plugin

After implementing a type that satisfies one of {collector, processor, publisher} interfaces, all that is left to do is a call the appropriate plugin.StartX() with your plugin specific options. That could be as simple as:

```go
plugin.StartCollector(rand.RandCollector{}, pluginName, pluginVersion)
```

## Meta options

The available options are defined in [plugin/meta.go](https://github.com/intelsdi-x/snap-plugin-lib-go/tree/master/v/1/plugin/meta.go). You can use some or none of the options. The options with definitions/explanations are below:

```go
// ConcurrencyCount is the max number of concurrent calls the plugin
// should take. For example:
// If there are 5 tasks using the plugin and its concurrency count is 2,
// snapd will keep 3 plugin instances running.
// ConcurrencyCount overwrites the default (5) for a Meta's ConcurrencyCount.
func ConcurrencyCount(cc int) MetaOpt {
}

// Exclusive == true results in a single instance of the plugin running
// regardless of the number of tasks using the plugin.
// Exclusive overwrites the default (false) for a Meta's Exclusive key.
func Exclusive(e bool) MetaOpt {
}

// Unsecure results in unencrypted communication with this plugin.
// Unsecure overwrites the default (false) for a Meta's Unsecure key.
func Unsecure(e bool) MetaOpt {
}

// RoutingStrategy will override the routing strategy this plugin requires.
// The default routing strategy is Least Recently Used.
// RoutingStrategy overwrites the default (LRU) for a Meta's RoutingStrategy.
func RoutingStrategy(r router) MetaOpt {
}

// CacheTTL will override the default cache TTL for the this plugin. snapd
// caches metrics on the daemon side for a default of 500ms.
// CacheTTL overwrites the default (500ms) for a Meta's CacheTTL.
func CacheTTL(t time.Duration) MetaOpt {
}
```

An example of what using all of them would look like:

```go
plugin.StartCollector(mypackage.Mytype{},
pluginName,
pluginVersion,
plugin.ConcurrencyCount(a),
plugin.Exclusive(b),
plugin.Unsecure(c),
plugin.RoutingStrategy(d),
plugin.CacheTTL(e))
```

135 changes: 135 additions & 0 deletions examples/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,135 @@
## Snap Plugin Go Library Examples
Here you will find example plugins that cover the basics for writing collector, processor, and publisher plugins.

## Build Plugins & Use with Snap

To get these example collector, processor, and publisher plugins to build properly and work with Snap you will need to have [glide](https://glide.sh/) installed in your $PATH. You should also add snapctl and snapd in your $PATH.

To test these plugins with Snap, you will need to have [Snap](https://github.com/intelsdi-x/snap) installed, check out these docs for [Snap setup details](https://github.com/intelsdi-x/snap/blob/master/docs/BUILD_AND_TEST.md#getting-started).

###1. Get the plugin library repo:
`go get github.com/intelsdi-x/snap-plugin-lib-go/...` will add the repo to your $GOPATH

###2. Go to the snap-plugin-lib-go folder and update to the newest versions of the package with `glide up`:

```
$ cd snap-plugin-lib-go
$ glide up
[INFO] Downloading dependencies. Please wait...
[INFO] --> Fetching updates for github.com/jtolds/gls.
[INFO] --> Fetching updates for github.com/golang/protobuf.
[INFO] --> Fetching updates for github.com/smartystreets/assertions.
[INFO] --> Fetching updates for github.com/smartystreets/goconvey.
[INFO] --> Fetching updates for google.golang.org/grpc.
[INFO] --> Fetching updates for github.com/gopherjs/gopherjs.
[INFO] --> Fetching updates for golang.org/x/net.
[INFO] --> Setting version for github.com/golang/protobuf to 888eb0692c857ec880338addf316bd662d5e630e.
[INFO] --> Setting version for github.com/smartystreets/assertions to 443d812296a84445c202c085f19e18fc238f8250.
[INFO] --> Setting version for github.com/smartystreets/goconvey to 995f5b2e021c69b8b028ba6d0b05c1dd500783db.
[INFO] --> Setting version for github.com/gopherjs/gopherjs to 4b53e1bddba0e2f734514aeb6c02db652f4c6fe8.
[INFO] --> Setting version for github.com/jtolds/gls to 8ddce2a84170772b95dd5d576c48d517b22cac63.
[INFO] --> Setting version for google.golang.org/grpc to 0032a855ba5c8a3c8e0d71c2deef354b70af1584.
[INFO] --> Setting version for golang.org/x/net to 154d9f9ea81208afed560f4cf27b4860c8ed1904.
[INFO] Resolving imports
[INFO] Downloading dependencies. Please wait...
[INFO] Setting references for remaining imports
[INFO] Exporting resolved dependencies...
[INFO] --> Exporting github.com/gopherjs/gopherjs
[INFO] --> Exporting github.com/golang/protobuf
[INFO] --> Exporting github.com/jtolds/gls
[INFO] --> Exporting github.com/smartystreets/assertions
[INFO] --> Exporting github.com/smartystreets/goconvey
[INFO] --> Exporting google.golang.org/grpc
[INFO] --> Exporting golang.org/x/net
[INFO] Replacing existing vendor dependencies
[INFO] Project relies on 7 dependencies.
```

###3. Build the collector, processor, and/or publisher plugins in the examples folder.
Use the `go build` command to generate the example binary files for the collector, processor, and publisher.
option -o outputs the binary to the specified name

```
$ go build -o example-collector examples/collector/main.go
$ go build -o example-processor examples/processor/main.go
$ go build -o example-publisher examples/publisher/main.go
```

###4. Run Snap, Load Plugins, and Run Tasks
You can now try [running Snap](https://github.com/intelsdi-x/snap#running-snap), [loading plugins](https://github.com/intelsdi-x/snap#load-plugins), and [running tasks](https://github.com/intelsdi-x/snap#running-tasks). You'll find a sample task.yml file for these examples at the bottom of this README.md.

Below are some sample commands to try:

```
$ snapctl plugin load example-collector
Plugin loaded
Name: test-rand-collector
Version: 1
Type: collector
Signed: false
Loaded Time: Fri, 23 Sep 2016 17:41:44 PDT

$ snapctl plugin list
NAME VERSION TYPE SIGNED STATUS LOADED TIME
test-rand-collector 1 collector false loaded Fri, 23 Sep 2016 17:41:44 PDT

$ snapctl metric list
NAMESPACE VERSIONS
/random/float 1
/random/integer 1
/random/string 1

$ snapctl plugin load example-processor
Plugin loaded
Name: test-reverse-processor
Version: 1
Type: processor
Signed: false
Loaded Time: Fri, 23 Sep 2016 17:44:12 PDT

$ snapctl plugin load example-publisher
Plugin loaded
Name: test-file-publisher
Version: 1
Type: publisher
Signed: false
Loaded Time: Fri, 23 Sep 2016 17:44:23 PDT

$ snapctl plugin list
NAME VERSION TYPE SIGNED STATUS LOADED TIME
test-rand-collector 1 collector false loaded Fri, 23 Sep 2016 17:41:44 PDT
test-reverse-processor 1 processor false loaded Fri, 23 Sep 2016 17:44:12 PDT
test-file-publisher 1 publisher false loaded Fri, 23 Sep 2016 17:44:23 PDT

```

### Create a task.
You can create a task.yml file for these examples using the following code.
Discover [how tasks work](https://github.com/intelsdi-x/snap/blob/master/docs/TASKS.md).

task.yml

```---
version: 1
schedule:
type: "simple"
interval: "1s"
max-failures: 10
workflow:
collect:
metrics:
/random/float: {}
/random/integer: {}
/random/string: {}
config:
process:
-
plugin_name: "test-reverse-processor"
process: null
publish:
-
plugin_name: "test-file-publisher"
config:
file: "/tmp/snap_published_grpc_file.log"

```
145 changes: 145 additions & 0 deletions examples/collector/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,145 @@

## Snap Plugin Go Library: Collector Plugin Example
Here you will find an example plugin that covers the basics for writing a collector plugin.

## Plugin Naming, Files, and Directory
For your collector plugin, create a new repository and name your plugin project using the following format:

>snap-plugin-[plugin-type]-[plugin-name]

For example:
>snap-plugin-collector-rand


Example files and directory structure:
```
snap-plugin-[plugin-type]-[plugin-name]
|--[plugin-name]
|--[plugin-name].go
|--[plugin-name]_test.go
|--main.go
```

For example:
```
snap-plugin-collector-rand
|--rand
|--rand.go
|--rand_test.go
|--main.go
```

* The [plugin-name] folder (for example `rand`) will include all files to implement the appropriate interface methods
* Your [plugin-name] folder will also include your test files.



## Interface Methods

In order to write a plugin for Snap, it is necessary to define a few methods to satisfy the appropriate interfaces. These interfaces must be defined for a collector plugin:


```go
// Plugin is an interface shared by all plugins and must implemented by each plugin to communicate with Snap.
type Plugin interface {
GetConfigPolicy() (ConfigPolicy, error)
}

// Collector is a plugin which is the source of new data in the Snap pipeline.
type Collector interface {
Plugin

GetMetricTypes(Config) ([]Metric, error)
CollectMetrics([]Metric) ([]Metric, error)
}
```
The interface is slightly different depending on what type (collector, processor, or publisher) of plugin is being written. Please see other plugin types for more details.



## Starting a plugin

After implementing a type that satisfies one of {collector, processor, publisher} interfaces, all that is left to do is to call the appropriate plugin.StartX() with your plugin specific meta options. For example with no meta options specified:

```go
plugin.StartCollector(rand.RandCollector{}, pluginName, pluginVersion)
```

### Meta options

The available options are defined in [plugin/meta.go](https://github.com/intelsdi-x/snap-plugin-lib-go/tree/master/v/1/plugin/meta.go). You can use some or none of the options. The options with definitions/explanations are below:

```go
// ConcurrencyCount is the max number of concurrent calls the plugin
// should take. For example:
// If there are 5 tasks using the plugin and its concurrency count is 2,
// snapd will keep 3 plugin instances running.
// ConcurrencyCount overwrites the default (5) for a Meta's ConcurrencyCount.
func ConcurrencyCount(cc int) MetaOpt {
}

// Exclusive == true results in a single instance of the plugin running
// regardless of the number of tasks using the plugin.
// Exclusive overwrites the default (false) for a Meta's Exclusive key.
func Exclusive(e bool) MetaOpt {
}

// Unsecure results in unencrypted communication with this plugin.
// Unsecure overwrites the default (false) for a Meta's Unsecure key.
func Unsecure(e bool) MetaOpt {
}

// RoutingStrategy will override the routing strategy this plugin requires.
// The default routing strategy is Least Recently Used.
// RoutingStrategy overwrites the default (LRU) for a Meta's RoutingStrategy.
func RoutingStrategy(r router) MetaOpt {
}

// CacheTTL will override the default cache TTL for the this plugin. snapd
// caches metrics on the daemon side for a default of 500ms.
// CacheTTL overwrites the default (500ms) for a Meta's CacheTTL.
func CacheTTL(t time.Duration) MetaOpt {
}
```

An example using some arbitrary values::

```go
plugin.StartCollector(
mypackage.Mytype{},
pluginName,
pluginVersion,
plugin.ConcurrencyCount(2),
plugin.Exclusive(true),
plugin.Unsecure(true),
plugin.RoutingStrategy(StickyRouter),
plugin.CacheTTL(time.Second))
```

## Testing
For testing reference the [Snap Testing Guidelines](https://github.com/intelsdi-x/snap/blob/master/CONTRIBUTING.md#testing-guidelines). To test your plugin with Snap you will need to have [Snap](https://github.com/intelsdi-x/snap) installed, check out these docs for [Snap setup details](https://github.com/intelsdi-x/snap/blob/master/docs/BUILD_AND_TEST.md#getting-started).

Each test file should specify the appropriate build tag such as "small", "medium", "large" (e.g. // +build small).


For example if you want to run only small tests:
```
// +build small
// you must include at least one line between the build tag and the package name.
package rand
```

For example if you want to run small and medium tests:
```
// +build small medium

package rand
```

## Ready to Share
You've made a plugin! Now it's time to share it. Create a release by following these [steps](https://help.github.com/articles/creating-releases/). We recommend that your release version match your plugin version, see example [here](https://github.com/intelsdi-x/snap-plugin-lib-go/blob/master/examples/collector/main.go#L29).

Don't forget to announce your plugin release on [slack](https://intelsdi-x.herokuapp.com/) and get your plugin added to the [Plugin Catalog](https://github.com/intelsdi-x/snap/blob/master/docs/PLUGIN_CATALOG.md)!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mjbrender-updated this last section

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! @sarahjhh