Added {select, Expr} pattern for predefined subscriptions.

Also, added a start phase for predefined reporters. The reporters must start *after* the predefined entries have been registered, in order to avoid a race condition.
Feuerlabs · Apr 7, 2014 · 530fb4a · 530fb4a
1 parent 5fc9ec6
commit 530fb4a
Show file tree

Hide file tree

Showing 8 changed files with 201 additions and 36 deletions.
diff --git a/README.md b/README.md
@@ -4,7 +4,7 @@
 
 Copyright (c) 2014 Basho Technologies, Inc.  All Rights Reserved.
 
-__Version:__ Mar 19 2014 23:14:26
+__Version:__ Mar 20 2014 20:42:23
 
 __Authors:__ Ulf Wiger ([`ulf.wiger@feuerlabs.com`](mailto:ulf.wiger@feuerlabs.com)), Magnus Feuer ([`magnus.feuer@feuerlabs.com`](mailto:magnus.feuer@feuerlabs.com)).
 
@@ -676,14 +676,24 @@ The `subscribers` sub-section contains all static subscriptions to be
 setup att exometer applications start. Each tuple in the prop list
 should be of one of the following formats:
 
-`{Reporter, Metric, DataPoint, Interval}`
-`{Reporter, Metric, DataPoint, Interval, RetryFailedMetrics}`
-`{Reporter, Metric, DataPoint, Interval, RetryFailedMetrics, Extra}`
-`{apply, {M, F, A}}`
+* `{Reporter, Metric, DataPoint, Interval}`
+
+* `{Reporter, Metric, DataPoint, Interval, RetryFailedMetrics}`
+
+* `{Reporter, Metric, DataPoint, Interval, RetryFailedMetrics, Extra}`
+
+* `{apply, {M, F, A}}`
+
+* `{select, {MatchPattern, DataPoint, Interval [, Retry [, Extra] ]}}`
 
 In the case of `{apply, M, F, A}`, the result of `apply(M, F, A)` must
 be a list of `subscribers` tuples.
 
+In the case of `{select, Expr}`, a list of metrics is fetched using
+`exometer:select(MatchPattern)`, where the result must be on the form
+`{Key, Type, Status}` (i.e. what corresponds to `'$_'`).
+The rest of the items will be applied to each of the matching entries.
+
 The meaning of the above tuple elements is:
 
 + `Reporter :: module()`
@@ -722,6 +732,33 @@ Provides a means to pass along extra information for a given
    subscription. An example is the `syntax` option for the SNMP reporter,
    in which case `Extra` needs to be a property list.
 
+Example configuration in sys.config, using the `{select, Expr}` pattern:
+
+```erlang
+
+[
+ {exometer, [
+             {predefined,
+              [{[a,1], counter, []},
+               {[a,2], counter, []},
+               {[b,1], counter, []},
+               {[c,1], counter, []}]},
+             {report,
+              [
+               {reporters,
+                [{exometer_report_tty, []}]},
+               {subscribers,
+                [{select, {[{ {[a,'_'],'_','_'}, [], ['$_']}],
+                           exometer_report_tty, value, 1000}}]}
+              ]}
+            ]}
+].
+
+```
+
+This will activate a subscription on `[a,1]` and `[a,2]` in the
+`exometer_report_tty` reporter, firing once per second.
+
 
 #### <a name="Configuring_reporter_plugins">Configuring reporter plugins</a> ####
 

diff --git a/doc/README.md b/doc/README.md
@@ -4,7 +4,7 @@
 
 Copyright (c) 2014 Basho Technologies, Inc.  All Rights Reserved.
 
-__Version:__ Mar 5 2014 11:56:38
+__Version:__ Mar 20 2014 20:42:23
 
 __Authors:__ Ulf Wiger ([`ulf.wiger@feuerlabs.com`](mailto:ulf.wiger@feuerlabs.com)), Magnus Feuer ([`magnus.feuer@feuerlabs.com`](mailto:magnus.feuer@feuerlabs.com)).
 
@@ -659,8 +659,8 @@ Below is an example, from `exometer/priv/app.config`:
 ```erlang
 
 {exometer, [
-    {report, [ 
-        {subscribers, [ 
+    {report, [
+        {subscribers, [
             {exometer_report_collectd, [db, cache, hits], mean, 2000, true},
             {exometer_report_collectd, [db, cache, hits], max, 5000, false}
         ]}
@@ -674,36 +674,90 @@ how to configure individual plugins.
 
 The `subscribers` sub-section contains all static subscriptions to be
 setup att exometer applications start. Each tuple in the prop list
-contains five elements:
+should be of one of the following formats:
+
+* `{Reporter, Metric, DataPoint, Interval}`
+
+* `{Reporter, Metric, DataPoint, Interval, RetryFailedMetrics}`
+
+* `{Reporter, Metric, DataPoint, Interval, RetryFailedMetrics, Extra}`
+
+* `{apply, {M, F, A}}`
 
-+ `receiver` (module name atom)
+* `{select, {MatchPattern, DataPoint, Interval [, Retry [, Extra] ]}}`
+
+In the case of `{apply, M, F, A}`, the result of `apply(M, F, A)` must
+be a list of `subscribers` tuples.
+
+In the case of `{select, Expr}`, a list of metrics is fetched using
+`exometer:select(MatchPattern)`, where the result must be on the form
+`{Key, Type, Status}` (i.e. what corresponds to `'$_'`).
+The rest of the items will be applied to each of the matching entries.
+
+The meaning of the above tuple elements is:
+
++ `Reporter :: module()`
 <br></br>
 Specifies the reporter plugin module, such as`exometer_report_collectd` that is to receive updated metric's data
 points.
 
-+ `name` (list of atoms)
++ `Metric :: [atoms()]`
 <br></br>
 Specifies the path to a metric previously created with an`exometer:new()` call.
 
-+ `datapoint` (atom)
++ `DataPoint` ::  atom() | [atom()]'
 <br></br>
 Specifies the data point within the given metric to send to the
     receiver. The data point must match one of the data points returned by`exometer:info(Name, datapoints)` for the given metrics name.
 
-+ `interval` (milliseconds)
++ `Interval` :: integer()' (milliseconds)
 <br></br>
 Specifies the interval, in milliseconds, between each update of the
 given metric's data point. At the given interval, the data point will
 be samples, and the result will be sent to the receiver.
 
-+ `retry_failed_metrics (true | false)`
++ `RetryFailedMetrics :: boolean()`
 <br></br>
 Specifies if the metric should be continued to be reported
-even if it is not found during a reporting cycle. This would be
-the case if a metric is not created by the time it is reported for
-the first time. If the metric will be created at a later time,
-this value should be set to true. Set this value to false if all
-attempts to report the metric should stop if when is not found.
+    even if it is not found during a reporting cycle. This would be
+    the case if a metric is not created by the time it is reported for
+    the first time. If the metric will be created at a later time,
+    this value should be set to true. Set this value to false if all
+    attempts to report the metric should stop if when is not found.
+    The default value is `true`.
+
++ `Extra :: any()`
+<br></br>
+Provides a means to pass along extra information for a given
+   subscription. An example is the `syntax` option for the SNMP reporter,
+   in which case `Extra` needs to be a property list.
+
+Example configuration in sys.config, using the `{select, Expr}` pattern:
+
+```erlang
+
+[
+ {exometer, [
+             {predefined,
+              [{[a,1], counter, []},
+               {[a,2], counter, []},
+               {[b,1], counter, []},
+               {[c,1], counter, []}]},
+             {report,
+              [
+               {reporters,
+                [{exometer_report_tty, []}]},
+               {subscribers,
+                [{select, {[{ {[a,'_'],'_','_'}, [], ['$_']}],
+                           exometer_report_tty, value, 1000}}]}
+              ]}
+            ]}
+].
+
+```
+
+This will activate a subscription on `[a,1]` and `[a,2]` in the
+`exometer_report_tty` reporter, firing once per second.
 
 
 #### <a name="Configuring_reporter_plugins">Configuring reporter plugins</a> ####

diff --git a/doc/edoc-info b/doc/edoc-info
@@ -1,3 +1,4 @@
+%% encoding: UTF-8
 {application,exometer}.
 {packages,[]}.
 {modules,[exometer,exometer_admin,exometer_cache,exometer_cpu,

diff --git a/doc/exometer_report.md b/doc/exometer_report.md
@@ -313,7 +313,7 @@ metric() = [atom(), ...]
 
 
 <table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#add_reporter-2">add_reporter/2</a></td><td></td></tr><tr><td valign="top"><a href="#call_reporter-2">call_reporter/2</a></td><td></td></tr><tr><td valign="top"><a href="#cast_reporter-2">cast_reporter/2</a></td><td></td></tr><tr><td valign="top"><a href="#list_metrics-0">list_metrics/0</a></td><td></td></tr><tr><td valign="top"><a href="#list_metrics-1">list_metrics/1</a></td><td></td></tr><tr><td valign="top"><a href="#list_reporters-0">list_reporters/0</a></td><td></td></tr><tr><td valign="top"><a href="#list_subscriptions-1">list_subscriptions/1</a></td><td></td></tr><tr><td valign="top"><a href="#new_entry-1">new_entry/1</a></td><td></td></tr><tr><td valign="top"><a href="#remove_reporter-1">remove_reporter/1</a></td><td></td></tr><tr><td valign="top"><a href="#remove_reporter-2">remove_reporter/2</a></td><td></td></tr><tr><td valign="top"><a href="#setopts-3">setopts/3</a></td><td></td></tr><tr><td valign="top"><a href="#start_link-0">start_link/0</a></td><td>Starts the server
---------------------------------------------------------------------.</td></tr><tr><td valign="top"><a href="#subscribe-4">subscribe/4</a></td><td>Equivalent to <a href="#subscribe-5"><tt>subscribe(Reporter, Metric, DataPoint, Interval, [])</tt></a>.</td></tr><tr><td valign="top"><a href="#subscribe-5">subscribe/5</a></td><td>Add a subscription to an existing reporter.</td></tr><tr><td valign="top"><a href="#terminate_reporter-1">terminate_reporter/1</a></td><td></td></tr><tr><td valign="top"><a href="#unsubscribe-3">unsubscribe/3</a></td><td>Equivalent to <a href="#unsubscribe-4"><tt>unsubscribe(Reporter, Metric, DataPoint, [])</tt></a>.</td></tr><tr><td valign="top"><a href="#unsubscribe-4">unsubscribe/4</a></td><td>Removes a subscription.</td></tr><tr><td valign="top"><a href="#unsubscribe_all-2">unsubscribe_all/2</a></td><td>Removes all subscriptions related to Metric in Reporter.</td></tr></table>
+--------------------------------------------------------------------.</td></tr><tr><td valign="top"><a href="#start_reporters-0">start_reporters/0</a></td><td></td></tr><tr><td valign="top"><a href="#subscribe-4">subscribe/4</a></td><td>Equivalent to <a href="#subscribe-5"><tt>subscribe(Reporter, Metric, DataPoint, Interval, [])</tt></a>.</td></tr><tr><td valign="top"><a href="#subscribe-5">subscribe/5</a></td><td>Add a subscription to an existing reporter.</td></tr><tr><td valign="top"><a href="#terminate_reporter-1">terminate_reporter/1</a></td><td></td></tr><tr><td valign="top"><a href="#unsubscribe-3">unsubscribe/3</a></td><td>Equivalent to <a href="#unsubscribe-4"><tt>unsubscribe(Reporter, Metric, DataPoint, [])</tt></a>.</td></tr><tr><td valign="top"><a href="#unsubscribe-4">unsubscribe/4</a></td><td>Removes a subscription.</td></tr><tr><td valign="top"><a href="#unsubscribe_all-2">unsubscribe_all/2</a></td><td>Removes all subscriptions related to Metric in Reporter.</td></tr></table>
 
 
 <a name="functions"></a>
@@ -435,6 +435,13 @@ start_link() -&gt; {ok, pid()} | ignore | {error, any()}
 
 Starts the server
 --------------------------------------------------------------------
+<a name="start_reporters-0"></a>
+
+### start_reporters/0 ###
+
+`start_reporters() -> any()`
+
+
 <a name="subscribe-4"></a>
 
 ### subscribe/4 ###

diff --git a/doc/overview.edoc b/doc/overview.edoc
@@ -587,14 +587,24 @@ The `subscribers' sub-section contains all static subscriptions to be
 setup att exometer applications start. Each tuple in the prop list
 should be of one of the following formats:
 
-`{Reporter, Metric, DataPoint, Interval}'
-`{Reporter, Metric, DataPoint, Interval, RetryFailedMetrics}'
-`{Reporter, Metric, DataPoint, Interval, RetryFailedMetrics, Extra}'
-`{apply, {M, F, A}}'
+* `{Reporter, Metric, DataPoint, Interval}'
+
+* `{Reporter, Metric, DataPoint, Interval, RetryFailedMetrics}'
+
+* `{Reporter, Metric, DataPoint, Interval, RetryFailedMetrics, Extra}'
+
+* `{apply, {M, F, A}}'
+
+* `{select, {MatchPattern, DataPoint, Interval [, Retry [, Extra] ]}}'
 
 In the case of `{apply, M, F, A}', the result of `apply(M, F, A)' must
 be a list of `subscribers' tuples.
 
+In the case of `{select, Expr}', a list of metrics is fetched using
+`exometer:select(MatchPattern)', where the result must be on the form
+`{Key, Type, Status}' (i.e. what corresponds to <code>'$_'</code>).
+The rest of the items will be applied to each of the matching entries.
+
 The meaning of the above tuple elements is:
 
 + `Reporter :: module()'
@@ -630,6 +640,31 @@ The meaning of the above tuple elements is:
    subscription. An example is the `syntax' option for the SNMP reporter,
    in which case `Extra' needs to be a property list.
 
+Example configuration in sys.config, using the `{select, Expr}' pattern:
+
+<pre lang="erlang">
+[
+ {exometer, [
+             {predefined,
+              [{[a,1], counter, []},
+               {[a,2], counter, []},
+               {[b,1], counter, []},
+               {[c,1], counter, []}]},
+             {report,
+              [
+               {reporters,
+                [{exometer_report_tty, []}]},
+               {subscribers,
+                [{select, {[{ {[a,'_'],'_','_'}, [], ['$_']}],
+                           exometer_report_tty, value, 1000}}]}
+              ]}
+            ]}
+].
+</pre>
+
+This will activate a subscription on `[a,1]' and `[a,2]' in the
+`exometer_report_tty' reporter, firing once per second.
+
 
 === Configuring reporter plugins ===
 The various reporter plugins to be loaded by exometer are configured

diff --git a/src/exometer.app.src b/src/exometer.app.src
@@ -14,7 +14,8 @@
    [
    ]},
   {mod, {exometer_app, []}},
-  {start_phases, [{preset_defaults, []}]},
+  {start_phases, [{preset_defaults, []},
+                  {start_reporters, []}]},
   {env, 
    [
    ]}

diff --git a/src/exometer_app.erl b/src/exometer_app.erl
@@ -26,7 +26,10 @@ start(_StartType, _StartArgs) ->
     exometer_sup:start_link().
 
 start_phase(preset_defaults, _Type, []) ->
-    exometer_admin:preset_defaults().
+    exometer_admin:preset_defaults();
+start_phase(start_reporters, _Type, []) ->
+    exometer_report:start_reporters().
+
 
 stop(_State) ->
     ok.