(PIE-1363) Parameter name changes and removals

coreymbe · Mar 15, 2024 · 4c80908 · 4c80908
1 parent 2ca22e2
commit 4c80908
Show file tree

Hide file tree

Showing 8 changed files with 88 additions and 32 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -6,6 +6,8 @@ All notable changes to this project will be documented in this file. The format
 
 ### Added
 
+- New parameters `token_events` and `url_events` can now be used to store events from `pe_event_forwarding` in a different index. [#212](https://github.com/puppetlabs/puppetlabs-splunk_hec/pull/212)
+
 - The parameter `ignore_system_cert_store` is now named `include_system_cert_store` and defaults to **false**. [#208](https://github.com/puppetlabs/puppetlabs-splunk_hec/pull/208)
 
 - Credential data provided to this module is now written to a separate configuration file utilizing the Sensitive data type to ensure redaction from Puppet logs and reports. [#204](https://github.com/puppetlabs/puppetlabs-splunk_hec/pull/204)
@@ -18,8 +20,14 @@ All notable changes to this project will be documented in this file. The format
 
 - Add support for Puppet 8. [#200](https://github.com/puppetlabs/puppetlabs-splunk_hec/pull/200)
 
+### Removed
+
+- The deprecated `reports` parameter has been removed in favor of having the module automatically add the **splunk_hec** setting to `puppet.conf`. [#212](https://github.com/puppetlabs/puppetlabs-splunk_hec/pull/212)
+
 ### Fixed
 
+- The `collect_facts` parameter has been renamed to `facts_allowlist` to align with the `facts_blocklist` parameter. [#212](https://github.com/puppetlabs/puppetlabs-splunk_hec/pull/212)
+
 - No longer utilizing `parse_legacy_metrics` function for metrics collected with older versions of `puppet_metrics_collector`. [#211](https://github.com/puppetlabs/puppetlabs-splunk_hec/pull/211)
 
 - False positive when attempting to rescue required facts from an unconfigured  `splunk_hec::facts_blocklist`. [#210](https://github.com/puppetlabs/puppetlabs-splunk_hec/pull/210)

diff --git a/README.md b/README.md
@@ -89,7 +89,7 @@ Instructions assume you are using Puppet Enterprise. For Open Source Puppet inst
   * Commit the changes.
   * Run Puppet on the node group; this will cause a restart of the `pe-puppetserver` service.
 
-4. Log into the Splunk console and search `index=* sourcetype=puppet:summary`, if everything was done properly you should see the reports (and soon facts) from the systems in your Puppet environment.
+4. Log into the Splunk console and search `index=* sourcetype=puppet:summary`, if everything was done properly you should see the reports from the systems in your Puppet environment.
 
 ## Source Types
 
@@ -217,12 +217,12 @@ class profile::splunk_hec {
 
 The following parameters are utilized to configure which facts (including custom facts) you would like to send to Splunk:
 
-  * `collect_facts`
+  * `facts_allowlist`
   * `facts_blocklist` (**Optional**)
 
-To configure which facts to collect add the `collect_facts` parameter to the `splunk_hec` class and modify the array of facts presented.
+To configure which facts to collect add the `facts_allowlist` parameter to the `splunk_hec` class and modify the array of facts presented.
 
-  * To collect **all facts** available at the time of the Puppet run, add the special value `all.facts` to the `collect_facts` array.
+  * To collect **all facts** available at the time of the Puppet run, add the special value `all.facts` to the `facts_allowlist` array.
   * When collecting **all facts**, you can configure the optional parameter `facts_blocklist` with an array of facts that should not be collected.
 
 ## PE Event Forwarding

diff --git a/REFERENCE.md b/REFERENCE.md
@@ -71,7 +71,7 @@ The following parameters are available in the `splunk_hec` class:
       - [Parameters](#parameters)
         - [`url`](#url)
         - [`token`](#token)
-        - [`collect_facts`](#collect_facts)
+        - [`facts_allowlist`](#facts_allowlist)
         - [`enable_reports`](#enable_reports)
         - [`record_event`](#record_event)
         - [`disabled`](#disabled)
@@ -91,9 +91,11 @@ The following parameters are available in the `splunk_hec` class:
         - [`token_summary`](#token_summary)
         - [`token_facts`](#token_facts)
         - [`token_metrics`](#token_metrics)
+        - [`token_events`](#token_events)
         - [`url_summary`](#url_summary)
         - [`url_facts`](#url_facts)
         - [`url_metrics`](#url_metrics)
+        - [`url_events`](#url_events)
         - [`include_logs_status`](#include_logs_status)
         - [`include_logs_catalog_failure`](#include_logs_catalog_failure)
         - [`include_logs_corrective_change`](#include_logs_corrective_change)
@@ -115,7 +117,7 @@ The following parameters are available in the `splunk_hec` class:
 
 ##### <a name="-splunk_hec--url"></a>`url`
 
-Data type: `String`
+Data type: `Optional[String]`
 
 The url of the server that PE is running on
 
@@ -126,7 +128,7 @@ Data type: `Optional[String]`
 The default Splunk HEC token
 Note: The value of the token is converted to Puppet's Sensitive data type during catalog application.
 
-##### <a name="-splunk_hec--collect_facts"></a>`collect_facts`
+##### <a name="-splunk_hec--facts_allowlist"></a>`facts_allowlist`
 
 Data type: `Array`
 
@@ -297,6 +299,15 @@ Note: The value of the token is converted to Puppet's Sensitive data type during
 
 Default value: `undef`
 
+##### <a name="-splunk_hec--token_events"></a>`token_events`
+
+Data type: `Optional[String]`
+
+When storing events from pe_event_forwarding in a different index than the default token
+Note: The value of the token is converted to Puppet's Sensitive data type during catalog application.
+
+Default value: `undef`
+
 ##### <a name="-splunk_hec--url_summary"></a>`url_summary`
 
 Data type: `Optional[String]`
@@ -321,6 +332,14 @@ Similar to token_metrics; used to store metrics in a different index than the de
 
 Default value: `undef`
 
+##### <a name="-splunk_hec--url_events"></a>`url_events`
+
+Data type: `Optional[String]`
+
+Similar to token_events; used to store events from pe_event_forwarding in a different index than the default url
+
+Default value: `undef`
+
 ##### <a name="-splunk_hec--include_logs_status"></a>`include_logs_status`
 
 Data type: `Optional[Array]`

diff --git a/files/hec_secrets.yaml.epp b/files/hec_secrets.yaml.epp
@@ -1,7 +1,8 @@
 <%- | Optional[Sensitive[String]] $token = undef, 
       Optional[Sensitive[String]] $token_summary = undef,
       Optional[Sensitive[String]] $token_facts = undef,
-      Optional[Sensitive[String]] $token_metrics = undef
+      Optional[Sensitive[String]] $token_metrics = undef,
+      Optional[Sensitive[String]] $token_events = undef
 | -%>
 # managed by splunk_hec module
 ---
@@ -17,3 +18,6 @@
 <% if $token_metrics { -%>
 "token_metrics" : "<%= $token_metrics %>"
 <% } -%>
+<% if $token_events { -%>
+"token_events" : "<%= $token_events %>"
+<% } -%>
diff --git a/lib/puppet/indirector/facts/splunk_hec.rb b/lib/puppet/indirector/facts/splunk_hec.rb
@@ -48,7 +48,6 @@ def save(request)
       ]
 
       # lets ensure user provided fact names are downcased
-      # settings['facts.allowlist'] is populated by the splunk_hec::collect_facts param
       allow_list  = (settings['facts.allowlist'].map(&:downcase) + hardcoded).uniq
       block_list  = settings['facts.blocklist'].nil? ? [] : settings['facts.blocklist'].map(&:downcase)
       # lets rescue any hardcoded facts that have been added to the blocklist

diff --git a/manifests/init.pp b/manifests/init.pp
@@ -8,12 +8,12 @@
 # @example
 #   include splunk_hec
 #
-# @param [String] url
+# @param [Optional][String] url
 #   The url of the server that PE is running on
 # @param [Optional[String]] token
 #   The default Splunk HEC token
 #   Note: The value of the token is converted to Puppet's Sensitive data type during catalog application.
-# @param [Array] collect_facts
+# @param [Array] facts_allowlist
 #   The list of facts that will be collected in the report. To collect all facts available add the special value 'all.facts'.
 # @param [Boolean] enable_reports
 #   Adds splunk_hec to the list of report processors
@@ -33,9 +33,6 @@
 #   Makes sure that the facts get sent to splunk_hec
 # @param [Optional[Array]] facts_blocklist
 #   The list of facts that will not be collected in the report
-# @param [Optional[String]] reports
-#   Can specify report processors (other than puppetdb which is default)
-#   Deprecated; should not use (will give warning).
 # @param [String] pe_console
 #   The FQDN for the PE console
 # @param [Optional[Integer]] timeout
@@ -64,12 +61,17 @@
 #   Corresponds to puppet:metrics in the Puppet Report Viewer
 #   When storing metrics in a different index than the default token
 #   Note: The value of the token is converted to Puppet's Sensitive data type during catalog application.
+# @param [Optional[String]] token_events
+#   When storing events from pe_event_forwarding in a different index than the default token
+#   Note: The value of the token is converted to Puppet's Sensitive data type during catalog application.
 # @param [Optional[String]] url_summary
 #   Similar to token_summary; used to store summary in a different index than the default url
 # @param [Optional[String]] url_facts
 #   Similar to token_facts; used to store facts in a different index than the default url
 # @param [Optional[String]] url_metrics
 #   Similar to token_metrics; used to store metrics in a different index than the default url
+# @param [Optional[String]] url_events
+#   Similar to token_events; used to store events from pe_event_forwarding in a different index than the default url
 # @param [Optional[Array]] include_logs_status
 #   Determines if puppet logs should be included based on the return status of the puppet agent run
 #   Can be none, one, or any of the following: failed, changed, unchanged
@@ -103,9 +105,9 @@
 # @param [Optional[Array]] code_manager_data_filter
 #   Filters the code_manager event data
 class splunk_hec (
-  String $url,
+  Optional[String] $url                                  = undef,
   Optional[String] $token                                = undef,
-  Array $collect_facts                                   = ['dmi','disks','partitions','processors','networking'],
+  Array $facts_allowlist                                 = ['dmi','disks','partitions','processors','networking'],
   Boolean $enable_reports                                = false,
   Boolean $record_event                                  = false,
   Boolean $disabled                                      = false,
@@ -115,7 +117,6 @@
   String $facts_terminus                                 = 'puppetdb',
   String $facts_cache_terminus                           = 'splunk_hec',
   Optional[Array] $facts_blocklist                       = undef,
-  Optional[String] $reports                              = undef,
   String $pe_console                                     = $settings::report_server,
   Optional[Integer] $timeout                             = undef,
   Optional[String] $ssl_ca                               = undef,
@@ -125,9 +126,11 @@
   Optional[String] $token_summary                        = undef,
   Optional[String] $token_facts                          = undef,
   Optional[String] $token_metrics                        = undef,
+  Optional[String] $token_events                         = undef,
   Optional[String] $url_summary                          = undef,
   Optional[String] $url_facts                            = undef,
   Optional[String] $url_metrics                          = undef,
+  Optional[String] $url_events                           = undef,
   Optional[Array] $include_logs_status                   = undef,
   Boolean $include_logs_catalog_failure                  = false,
   Boolean $include_logs_corrective_change                = false,
@@ -145,7 +148,7 @@
   if (
     ($token == undef)
     and
-    ($token_summary or $token_facts or $token_metrics == undef)
+    (($token_summary == undef) or ($token_facts == undef) or ($token_metrics == undef))
   ) {
     $authorization_failure_message = @(MESSAGE/L)
       Splunk HEC: Unless you are utilizing individual HEC tokens \
@@ -155,6 +158,19 @@
     fail($authorization_failure_message)
   }
 
+  if (
+    ($url == undef)
+    and
+    (($url_summary == undef) or ($url_facts == undef) or ($url_metrics == undef))
+  ) {
+    $authorization_failure_message = @(MESSAGE/L)
+      Splunk HEC: Unless you are utilizing individual URLs \
+      ('url_summary', 'url_facts', 'url_metrics') \
+      you must configure the 'url' parameter.
+      |-MESSAGE
+    fail($authorization_failure_message)
+  }
+
   # Account for the differences in Puppet Enterprise and Open Source and Agent
   $agent_node = $facts['splunk_hec_agent_only_node']
 
@@ -189,22 +205,15 @@
 
   # Secure credential data
   $hec_secrets = {
-    'token' => $token,
+    'token'         => $token,
     'token_summary' => $token_summary,
-    'token_facts' => $token_facts,
+    'token_facts'   => $token_facts,
     'token_metrics' => $token_metrics,
+    'token_events'  => $token_events,
   }
   $secrets = Deferred('splunk_hec::secure', [$hec_secrets])
 
   if $enable_reports {
-    # lint:ignore:140chars
-    if $reports != undef {
-      notify { 'reports param deprecation warning':
-        message  => 'The reports parameter has been deprecated in favor of having the module automatically add the splunk_hec setting to puppet.conf. Update the reports param to undef or remove it entirely. Please note that the reports parameter is currently ignored and will be removed in a future release of this module.',
-        loglevel => 'warning',
-      }
-    }
-    # lint:endignore
     # The subsetting resource automatically adds the 'splunk_hec' report
     # processor to the reports setting if it hasn't yet been added there.  
     Resource[$ini_subsetting] { 'enable splunk_hec':
@@ -263,7 +272,7 @@
       group   => $group,
       mode    => '0600',
       require => File["${settings::confdir}/splunk_hec"],
-      content => Deferred('inline_epp', [file('splunk_hec/hec_secrets.yaml.epp'), $secrets]),
+      content => Sensitive(Deferred('inline_epp', [file('splunk_hec/hec_secrets.yaml.epp'), $secrets])),
       notify  => Service[$service],
     }
   } elsif $agent_node and $events_reporting_enabled {
@@ -281,11 +290,23 @@
       group   => $group,
       mode    => '0600',
       require => File["${settings::confdir}/splunk_hec"],
-      content => Deferred('inline_epp', [file('splunk_hec/hec_secrets.yaml.epp'), $secrets]),
+      content => Sensitive(Deferred('inline_epp', [file('splunk_hec/hec_secrets.yaml.epp'), $secrets])),
     }
   }
 
   if $events_reporting_enabled {
+    if (
+      (($token == undef) and ($token_events == undef))
+      or
+      (($url == undef) and ($url_events == undef))
+    ) {
+      $authorization_failure_message = @(MESSAGE/L)
+        Splunk HEC: Unless you are utilizing a specific URL \
+        and token for events ('url_events', 'token_events') \
+        you must configure the 'url' and 'token' parameters.
+        |-MESSAGE
+      fail($authorization_failure_message)
+    }
     if $pe_event_forwarding::confdir != undef {
       $confdir_base_path = $pe_event_forwarding::confdir
     }

diff --git a/templates/settings.yaml.epp b/templates/settings.yaml.epp
@@ -1,8 +1,10 @@
 # managed by splunk_hec module
 ---
+<% if $splunk_hec::url { -%>
 "url" : "<%= $splunk_hec::url %>"
+<% } -%>
 "facts.allowlist" :
-<% $splunk_hec::collect_facts.each |$fact| {-%>
+<% $splunk_hec::facts_allowlist.each |$fact| {-%>
         - <%= $fact %>
 <% } -%>
 <% unless $splunk_hec::facts_blocklist == undef { -%>
@@ -35,6 +37,9 @@
 <% if $splunk_hec::url_metrics { -%>
 "url_metrics" : "<%= $splunk_hec::url_metrics %>"
 <% } -%>
+<% if $splunk_hec::url_events { -%>
+"url_events" : "<%= $splunk_hec::url_events %>"
+<% } -%>
 <% if $splunk_hec::include_logs_status { -%>
 "include_logs_status" :
 <% $splunk_hec::include_logs_status.each |$status| {-%>

diff --git a/templates/util_splunk_hec.erb b/templates/util_splunk_hec.erb
@@ -90,7 +90,7 @@ end
 
 def submit_request(body)
   # we want users to be able to provide different tokens per sourcetype if they want
-  source_type = 'pe_event_forwarding'
+  source_type = 'events'
   token_name = "token_#{source_type}"
   http = create_http(source_type)
   token = secrets[token_name] || secrets['token'] || raise('Must provide token parameter to splunk class')