Merge pull request #286 from guilhemmarchand/testing

Version 1.2.37
guilhemmarchand · Mar 14, 2021 · 31aed6e · 31aed6e
2 parents 9019a39 + 9dc8c88
commit 31aed6e
Show file tree

Hide file tree

Showing 42 changed files with 1,726 additions and 2,414 deletions.
diff --git a/docs/configuration.rst b/docs/configuration.rst
diff --git a/docs/img/ootb_alerts.png b/docs/img/ootb_alerts.png
diff --git a/docs/img/ops_parsing_issues.png b/docs/img/ops_parsing_issues.png
diff --git a/docs/img/splunk_forwarding/allow_internal.png b/docs/img/splunk_forwarding/allow_internal.png
diff --git a/docs/img/splunk_forwarding/allow_metrics.png b/docs/img/splunk_forwarding/allow_metrics.png
diff --git a/docs/img/splunk_forwarding/data_host_details.png b/docs/img/splunk_forwarding/data_host_details.png
diff --git a/docs/img/splunk_forwarding/data_host_per_host_policy.png b/docs/img/splunk_forwarding/data_host_per_host_policy.png
diff --git a/docs/img/splunk_forwarding/data_hosts.png b/docs/img/splunk_forwarding/data_hosts.png
diff --git a/docs/img/splunk_forwarding/heavyforwarder1.png b/docs/img/splunk_forwarding/heavyforwarder1.png
diff --git a/docs/img/splunk_forwarding/heavyforwarder2.png b/docs/img/splunk_forwarding/heavyforwarder2.png
diff --git a/docs/img/splunk_forwarding/metric_host_details.png b/docs/img/splunk_forwarding/metric_host_details.png
diff --git a/docs/img/splunk_forwarding/metric_host_details2.png b/docs/img/splunk_forwarding/metric_host_details2.png
diff --git a/docs/img/splunk_forwarding/metric_hosts.png b/docs/img/splunk_forwarding/metric_hosts.png
diff --git a/docs/img/step_by_step_configure/ui_data_sources_mode.png b/docs/img/step_by_step_configure/ui_data_sources_mode.png
diff --git a/docs/img/step_by_step_configure/ui_update_indexes.png b/docs/img/step_by_step_configure/ui_update_indexes.png
diff --git a/docs/img/trackme_idx_filter.png b/docs/img/trackme_idx_filter.png
diff --git a/docs/img/trackme_roles.png b/docs/img/trackme_roles.png
diff --git a/docs/index.rst b/docs/index.rst
@@ -102,6 +102,7 @@ User guide:
    userguide
    itsi_integration
    cribl_integration
+   monitor_forwarders
    rest_api_reference
 
 Troubleshoot:

diff --git a/docs/monitor_forwarders.rst b/docs/monitor_forwarders.rst
@@ -0,0 +1,131 @@
+Monitor Splunk instances forwarding
+===================================
+
+**TrackMe monitors by default any Splunk instance forwarding to the Splunk indexing layer, this includes:**
+
+- Universal Forwarder instances
+- Heavy Forwarder instances
+- All other types of instances from your Splunk infrastructure
+
+**Forwarding is monitored via:**
+
+- ``data hosts`` by tracking the ``index=_internal sourcetype=splunkd``
+- ``metric hosts`` by trackking the ``spl`` metrics stored in the ``_metrics`` index
+
+.. image:: img/splunk_forwarding/data_hosts.png
+   :alt: data_hosts.png
+   :align: center
+   :width: 1200px
+
+.. image:: img/splunk_forwarding/metric_hosts.png
+   :alt: metric_hosts.png
+   :align: center
+   :width: 1200px
+
+Requirements
+------------
+
+Splunk forwarding good practices configuration
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+**Splunk good configuration practices implies that you are systematically forwarding the Splunk internals (and metrics) to the indexing layer in your outputs.conf configuration, see the Splunk PS base config apps:**
+
+- `Configurations Base Apps <https://drive.google.com/open?id=107qWrfsv17j5bLxc21ymTagjtHG0AobF>`_
+
+- `Configurations Cluster Apps <https://drive.google.com/open?id=10aVQXjbgQC99b9InTvncrLFWUrXci3gz>`_
+
+*See: org_all_forwarder_outputs / org_cluster_forwarder_outputs*
+
+Concretely, this implies that you configure the Splunk instances (all but indexers) to have an outputs.conf sanza similar to:
+
+::
+
+    [tcpout]
+    defaultGroup = primary_indexers
+
+    forwardedindex.2.whitelist = (_audit|_introspection|_internal|_metrics)
+
+TrackMe allow lists and block lists
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+**The default configuration of TrackMe implies monitoring every single index including the _internal (limited to sourcetype=splunkd) and the _metrics, if you use allowlisting & blocklisting, you need to make sure to include these items accordingly:**
+
+*For data hosts, allow the _internal*
+
+.. image:: img/splunk_forwarding/allow_internal.png
+   :alt: allow_internal.png
+   :align: center
+   :width: 600px
+
+*For metric hosts, allow the _metrics*
+
+.. image:: img/splunk_forwarding/allow_metrics.png
+   :alt: allow_metrics.png
+   :align: center
+   :width: 600px
+
+Usage
+-----
+
+**A Splunk instance that does not generate any data out of the internal will appear with the single _internal / splunkd combination in data hosts, and the spl metrics in metrics hosts:**
+
+.. image:: img/splunk_forwarding/data_host_details.png
+   :alt: data_host_details.png
+   :align: center
+   :width: 1200px
+
+.. image:: img/splunk_forwarding/metric_host_details.png
+   :alt: metric_host_details.png
+   :align: center
+   :width: 1200px
+
+**A green status basically indicates that:**
+
+- Splunk service is up and running
+- The instance is able to reach the indexing layer and properly ingest data as it is forwarding effectively its own data and metrics (which validates configuration and network layers theoritically)
+- The instance is expected to be acting in a normal and sane state
+
+Data hosts tracking
+^^^^^^^^^^^^^^^^^^^
+
+**When a Splunk instance does more than just indexing its own data and the host Metadata is refering to itself, the Splunk internal data and metrics appear as part of the indexing flow:**
+
+.. image:: img/splunk_forwarding/heavyforwarder1.png
+   :alt: heavyforwarder1.png
+   :align: center
+   :width: 1200px
+
+The default behaviour driven by the global host policy implies that as long as the Splunk instance is forwarding data, the host will remain green even if the sources monitored by and as this host runs into troubles, you can on a global basis change the :ref:`Data Hosts alerting policy<TrackMe Data Hosts - Define what works for you>` or selectively on a per host basis:
+
+*Global policy in TrackMe manage and configure:*
+
+.. image:: img/data_hosts_allerting_policy_config.png
+   :alt: data_hosts_allerting_policy_config.png
+   :align: center
+   :width: 1200px
+
+*Per data host policy:*
+
+.. image:: img/splunk_forwarding/data_host_per_host_policy.png
+   :alt: data_host_per_host_policy.png
+   :align: center
+   :width: 1200px
+
+**When the global policy, or the per host policy, is set to track per sourcetype, the data host will appear in a non green status if at least one sourcetype is red (for example even if Splunk internal is still going through):**
+
+.. image:: img/splunk_forwarding/heavyforwarder2.png
+   :alt: heavyforwarder2.png
+   :align: center
+   :width: 1200px
+
+Metric hosts tracking
+^^^^^^^^^^^^^^^^^^^^^
+
+**Metrics tracking acts differently, if any of the metric categories does not comply with monitoring rules (including the spl metrics), the host will turn into a red state:**
+
+.. image:: img/splunk_forwarding/metric_host_details2.png
+   :alt: metric_host_details2.png
+   :align: center
+   :width: 1200px
+
+Congratulations, you have now a builtin, easy and efficient monitoring of your Splunk instances availability, enable and configure up to your preferences the :ref:`Out of the box alerts` and the job is done!
diff --git a/docs/releasenotes.rst b/docs/releasenotes.rst
@@ -1,6 +1,30 @@
 Release notes
 #############
 
+Version 1.2.37
+==============
+
+**CAUTION:**
+
+This is a new main release branch, TrackMe 1.2.x requires the deployment of the following dependencies:
+
+- Semicircle Donut Chart Viz, Splunk Base: https://splunkbase.splunk.com/app/4378
+- Splunk Machine Learning Toolkit, Splunk Base: https://splunkbase.splunk.com/app/2890
+- Splunk Timeline - Custom Visualization, Splunk Base: https://splunkbase.splunk.com/app/3120
+
+TrackMe requires a summary index (defaults to trackme_summary) and a metric index (defaults to trackme_metrics):
+https://trackme.readthedocs.io/en/latest/configuration.html
+
+- Enhancement - Issue #279 - Decomission of the getlistdef custom command in favor of a simpler and cleaner pure SPL approach
+- Enhancement - Issue #280 - Add new REST endpoint to manage logical group associations
+- Enhancement - Issue #285 - Flipping statuses workflow improvements
+- Change - Issue #275 - permissions - provides a builtin trackme_user role to handle the minimal non admin access for TrackMe
+- Change - Issue #276 - User Interface - Migration of Ajax javascript REST calls made within the UI from splunkd to TrackMe based API endpoints
+- Change - Issue #278 - Upgrade of splunklib Python SDK to latest release 1.6.15
+- Fix - Issue #273 - User Interfaces - Several searches should not kick off start at TrackMe main UI loading time
+- Fix - Issue #274 - Data Sources - tags dropdown can render unwanted results when no tags are defined
+- Fix - Issue #277 - REST endpoint - the endpoint ds_update_min_dcount_host should allow any as the input
+
 Version 1.2.36
 ==============
 

diff --git a/docs/rest_api_reference.rst b/docs/rest_api_reference.rst
@@ -671,7 +671,7 @@ ds_update_min_dcount_host / Update minimal host dcount
 **This endpoint configures the minimal number of distinct hosts count for an existing data source, it requires a POST call with the following information:**
 
 - ``"data_name": name of the data source``
-- ``"min_dcount_host": minimal accepted number of distinct count hosts, must be an integer``
+- ``"min_dcount_host": minimal accepted number of distinct count hosts, must be an integer or any to disable the feature``
 - ``"update_comment": OPTIONAL: a comment for the update, comments are added to the audit record, if unset will be defined to: API update``
 
 *External:*
@@ -3615,17 +3615,21 @@ Logical Groups endpoints
 
 **Resources summary:**
 
-+---------------------------------------------------------------------------------------------------+-----------------------------------------------------------------+
-| Resource                                                                                          | API Path                                                        | 
-+===================================================================================================+=================================================================+
-| :ref:`logical_groups_collection / Get entire logical groups collection`                           | /services/trackme/v1/logical_groups/logical_groups_collection   |
-+---------------------------------------------------------------------------------------------------+-----------------------------------------------------------------+
-| :ref:`logical_groups_get_grp / Get a logical group`                                               | /services/trackme/v1/logical_groups/logical_groups_get_grp      |
-+---------------------------------------------------------------------------------------------------+-----------------------------------------------------------------+
-| :ref:`logical_groups_add_grp / Add a new or update a logical group`                               | /services/trackme/v1/logical_groups/logical_groups_add_grp      |
-+---------------------------------------------------------------------------------------------------+-----------------------------------------------------------------+
-| :ref:`logical_groups_del_grp / Delete a logical group`                                            | /services/trackme/v1/logical_groups/logical_groups_del_grp      |
-+---------------------------------------------------------------------------------------------------+-----------------------------------------------------------------+
++---------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+
+| Resource                                                                                          | API Path                                                           |
++===================================================================================================+====================================================================+
+| :ref:`logical_groups_collection / Get entire logical groups collection`                           | /services/trackme/v1/logical_groups/logical_groups_collection      |
++---------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+
+| :ref:`logical_groups_get_grp / Get a logical group`                                               | /services/trackme/v1/logical_groups/logical_groups_get_grp         |
++---------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+
+| :ref:`logical_groups_add_grp / Add a new or update a logical group`                               | /services/trackme/v1/logical_groups/logical_groups_add_grp         |
++---------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+
+| :ref:`logical_groups_del_grp / Delete a logical group`                                            | /services/trackme/v1/logical_groups/logical_groups_del_grp         |
++---------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+
+| :ref:`logical_groups_associate_group /  Associate an object with an existing logical group`       | /services/trackme/v1/logical_groups/logical_groups_associate_group |
++---------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+
+| :ref:`logical_groups_unassociate / Unassociate an object from any logical group it is member of`  | /services/trackme/v1/logical_groups/logical_groups_unassociate     |
++---------------------------------------------------------------------------------------------------+--------------------------------------------------------------------+
 
 logical_groups_collection / Get entire logical groups collection
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -3778,6 +3782,73 @@ logical_groups_del_grp / Delete a logical group
 
     Record with _key 5fdf7aa55af72855ab693b47 was deleted from the logical groups collection.
 
+logical_groups_associate_group /  Associate an object with an existing logical group
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+**This endpoint associates an object (data host or metric host) with an existing logical group (existing members of the logical groups are preserved and this object membership will be removed), it requires a POST call with the following data required:**
+
+- ``"object": the name of the data host or the metric host``
+- ``"key": the KVstore unique key of the logical group``
+- ``"update_comment": OPTIONAL: a comment for the update, comments are added to the audit record, if unset will be defined to: API update``
+
+*External:*
+
+::
+
+    curl -k -u admin:'ch@ngeM3' -X POST https://localhost:8089/services/trackme/v1/logical_groups/logical_groups_associate_group -d '{"object": "telegraf-node3", "key": "604356885ea0f10084356707", "comment_update": "Automated API driven logical group creation."}'
+
+*SPL query:*
+
+::
+
+    | trackme url="/services/trackme/v1/logical_groups/logical_groups_associate_group" mode="post" body="{\"object\": \"telegraf-node3\", \"key\": \"604356885ea0f10084356707\", \"comment_update\": \"Automated API driven logical group creation.\"}"
+
+*response:*
+
+::
+
+    {
+      "object_group_name": "logical group example",
+      "object_group_members": [
+      "telegraf-node1",
+      "telegraf-node2",
+      "telegraf-node3"
+    ],
+      "object_group_min_green_percent": "50",
+      "object_group_mtime": "1615025866.585574",
+      "_user": "nobody",
+      "_key": "604356885ea0f10084356707"
+    }
+
+logical_groups_unassociate / Unassociate an object from any logical group it is member of
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+**This endpoint unassociates an object (data host or metric host) from a logical group it is member of (existing associations of the logical groups are preserved), it requires a POST call with the following data required:**
+
+- ``"object": the object name (data host or metric host) to remove association for``
+- ``"key": the KVstore unique key of the logical group``
+- ``"update_comment": OPTIONAL: a comment for the update, comments are added to the audit record, if unset will be defined to: API update``
+
+*External:*
+
+::
+
+    curl -k -u admin:'ch@ngeM3' -X POST https://localhost:8089/services/trackme/v1/logical_groups/logical_groups_unassociate -d '{"object": "telegraf-node3", "key": "6043a23b33d53e70d86fc091", "comment_update": "Automated API driven logical group update."}'
+
+*SPL query:*
+
+::
+
+    | trackme url="/services/trackme/v1/logical_groups/logical_groups_unassociate" mode="post" body="{\"object\": \"telegraf-node3\", \"key\": \"6043a23b33d53e70d86fc091\", \"comment_update\": \"Automated API driven logical group update.\"}"
+
+*response:*
+
+::
+
+    {
+        "response": "object telegraf-node3 has been unassociated from logical group record key: 604356885ea0f10084356707"
+    }
+
 Data Sampling endpoints
 -----------------------
 

diff --git a/docs/userguide.rst b/docs/userguide.rst
@@ -2412,7 +2412,7 @@ Alerting policy for data hosts
    - The global alternative mode named "track per sourcetype" instructs TrackMe to consider sourcetypes and their monitoring rules individually on a per host basis, to finally define the overall state of the host
    - This global mode can optionally be overriden on a per host basis via the configuration screen of the data host
 
-See :ref:`Data hosts global alerting policy` to control the global policy settings.
+See :ref:`Data Hosts alerting policy<TrackMe Data Hosts - Define what works for you>` to control the global policy settings.
 
 **An host emitting multiple sourcetypes will appear in the UI with a multi value summary field describing the state and main information of sourcetypes:**
 
@@ -2449,7 +2449,7 @@ See :ref:`Data hosts global alerting policy` to control the global policy settin
 
 **To configure sourcetypes to be taken into account individually, you can either:**
 
-- Define the global policy accordingly (note: this applies by default to all hosts), See :ref:`Data hosts global alerting policy`
+- Define the global policy accordingly (note: this applies by default to all hosts), See :ref:`Data Hosts alerting policy<TrackMe Data Hosts - Define what works for you>`
 - Define the alerting policy for that host especially in the data host configuration screen
 
 **Defining a policy per host:**

diff --git a/trackme/app.manifest b/trackme/app.manifest
@@ -5,7 +5,7 @@
     "id": {
       "group": null,
       "name": "trackme",
-      "version": "1.2.36"
+      "version": "1.2.37"
     },
     "author": [
       {