CHANGELOG.md

@@ -1,5 +1,13 @@
 # Changelog
 
+## [v4.12.0](https://github.com/sensu/sensu-puppet/tree/v4.12.0) (2020-07-05)
+
+[Full Changelog](https://github.com/sensu/sensu-puppet/compare/v4.11.0...v4.12.0)
+
+### Added
+
+- Allow disabling namespace validation for large environments [\#1254](https://github.com/sensu/sensu-puppet/pull/1254) ([treydock](https://github.com/treydock))
+
 ## [v4.11.0](https://github.com/sensu/sensu-puppet/tree/v4.11.0) (2020-06-29)
 
 [Full Changelog](https://github.com/sensu/sensu-puppet/compare/v4.10.0...v4.11.0)
@@ -178,13 +186,13 @@
 - Add several parameters to sensu::agent class [\#1185](https://github.com/sensu/sensu-puppet/pull/1185) ([treydock](https://github.com/treydock))
 - Document upcoming breaking changes [\#1167](https://github.com/sensu/sensu-puppet/pull/1167) ([treydock](https://github.com/treydock))
 - BREAKING: Move cli resources to sensu::cli class [\#1164](https://github.com/sensu/sensu-puppet/pull/1164) ([treydock](https://github.com/treydock))
-- BREAKING: Update type properties to map to Sensu Go specifications [\#1154](https://github.com/sensu/sensu-puppet/pull/1154) ([treydock](https://github.com/treydock))
 
 ### Added
 
 - Document contact routing and bonsai asset bugfix [\#1194](https://github.com/sensu/sensu-puppet/pull/1194) ([treydock](https://github.com/treydock))
 - Support 'sensu-backend init' added in Sensu Go 5.16 [\#1192](https://github.com/sensu/sensu-puppet/pull/1192) ([treydock](https://github.com/treydock))
 - Misc test fixes [\#1189](https://github.com/sensu/sensu-puppet/pull/1189) ([treydock](https://github.com/treydock))
+- Initial work at design document [\#1161](https://github.com/sensu/sensu-puppet/pull/1161) ([treydock](https://github.com/treydock))
 
 ### Merged Pull Requests
 
@@ -248,11 +256,6 @@
 
 [Full Changelog](https://github.com/sensu/sensu-puppet/compare/v3.9.0...v3.10.0)
 
-### Added
-
-- Initial work at design document [\#1161](https://github.com/sensu/sensu-puppet/pull/1161) ([treydock](https://github.com/treydock))
-- Add bolt tasks [\#1153](https://github.com/sensu/sensu-puppet/pull/1153) ([treydock](https://github.com/treydock))
-
 ## [v3.9.0](https://github.com/sensu/sensu-puppet/tree/v3.9.0) (2019-10-10)
 
 [Full Changelog](https://github.com/sensu/sensu-puppet/compare/v3.8.0...v3.9.0)

README.md

@@ -975,6 +975,19 @@ class { 'sensu::backend':
 }
 ```
 
+If many thousands of resources such as `sensu_check` are defined there will be an execution of `sensuctl namespace list` for each check to validate
+the namespace exists if the namespace is not defined in Puppet.
+A similar validation is performed with `sensu_api` provider.  To avoid this extra overhead it may be necessary to disable this validation if you
+are defining namespaces outside of Puppet.
+
+**NOTE**: If namespace validation is disabled it's necessary to ensure a namespace is defined in Puppet in order to assign resources to that namespace.
+
+```puppet
+class { 'sensu':
+  validate_namespaces => false,
+}
+```
+
 ### Composite Names for Namespaces
 
 All resources that support having a `namespace` also support a composite name to define the namespace.

REFERENCE.md

@@ -252,6 +252,14 @@ DEPRECATED - The sensu agent old password needed when changing agent_password
 
 Default value: `undef`
 
+##### `validate_namespaces`
+
+Data type: `Boolean`
+
+Determines if sensuctl and sensu_api types will validate their namespace exists
+
+Default value: `true`
+
 ### sensu::agent
 
 Class to manage the Sensu agent.
@@ -1401,6 +1409,14 @@ Sensu API username
 
 Sensu API password
 
+##### `validate_namespaces`
+
+Valid values: `true`, `false`
+
+Determines of namespaces should be validated with Sensu API
+
+Default value: `true`
+
 ### sensu_api_validator
 
 **NOTE** This is a private type not intended to be used directly.
@@ -3807,6 +3823,14 @@ The name of the resource.
 
 sensuctl chunk-size
 
+##### `validate_namespaces`
+
+Valid values: `true`, `false`
+
+Determines of namespaces should be validated with sensuctl
+
+Default value: `true`
+
 ##### `path`
 
 path to sensuctl

lib/puppet/provider/sensu_api.rb

@@ -11,6 +11,11 @@ class << self
     attr_accessor :password
     attr_accessor :access_token
     attr_accessor :refresh_token
+    attr_accessor :validate_namespaces
+  end
+
+  def validate_namespaces
+    self.class.validate_namespaces
   end
 
   def self.update_access_token

lib/puppet/provider/sensuctl.rb

@@ -10,6 +10,11 @@ class Puppet::Provider::Sensuctl < Puppet::Provider
   class << self
     attr_accessor :chunk_size
     attr_accessor :path
+    attr_accessor :validate_namespaces
+  end
+
+  def validate_namespaces
+    self.class.validate_namespaces
   end
 
   def self.config_path

lib/puppet/type/sensu_api_config.rb

@@ -25,6 +25,12 @@
     desc "Sensu API password"
   end
 
+  newparam(:validate_namespaces, :boolean => true) do
+    desc "Determines of namespaces should be validated with Sensu API"
+    newvalues(:true, :false)
+    defaultto(:true)
+  end
+
   # First collect all types with sensu_api provider that come from this module
   # For each sensu_api type, set the class variable 'chunk_size' used by
   # each provider to list resources
@@ -40,6 +46,7 @@ def generate
       provider_class.url = self[:url]
       provider_class.username = self[:username]
       provider_class.password = self[:password]
+      provider_class.validate_namespaces = self[:validate_namespaces]
     end
     []
   end

lib/puppet/type/sensuctl_config.rb

@@ -17,6 +17,12 @@
     desc "sensuctl chunk-size"
   end
 
+  newparam(:validate_namespaces, :boolean => true) do
+    desc "Determines of namespaces should be validated with sensuctl"
+    newvalues(:true, :false)
+    defaultto(:true)
+  end
+
   newparam(:path) do
     desc "path to sensuctl"
   end
@@ -34,6 +40,7 @@ def generate
     sensuctl_types.each do |type|
       provider_class = Puppet::Type.type(type).provider(:sensuctl)
       provider_class.chunk_size = self[:chunk_size]
+      provider_class.validate_namespaces = self[:validate_namespaces]
       provider_class.path = self[:path]
     end
     []

lib/puppet_x/sensu/type.rb

@@ -39,14 +39,28 @@ def add_autorequires(namespace=true, require_configure=true, require_admin=true)
       end
 
       def self.validate_namespace(resource)
-        catalog_namespaces = []
-        resource.catalog.resources.each do |catalog_resource|
-          if catalog_resource.class.to_s == 'Puppet::Type::Sensu_namespace'
-            catalog_namespaces << catalog_resource.name
+        if ! @catalog_namespaces.nil?
+          catalog_namespaces = @catalog_namespaces
+        else
+          catalog_namespaces = []
+          resource.catalog.resources.each do |catalog_resource|
+            if catalog_resource.class.to_s == 'Puppet::Type::Sensu_namespace'
+              catalog_namespaces << catalog_resource.name
+            end
           end
+          @catalog_namespaces = catalog_namespaces
         end
-        namespaces = resource.provider.namespaces()
-        if (resource[:ensure] && resource[:ensure].to_sym != :absent) && !( catalog_namespaces.include?(resource[:namespace]) || namespaces.include?(resource[:namespace]) )
+        # Check if namespace is in catalog
+        if (resource[:ensure] && resource[:ensure].to_sym != :absent) && catalog_namespaces.include?(resource[:namespace])
+          return true
+        end
+        if ! resource.provider.validate_namespaces.nil? && resource.provider.validate_namespaces.to_s.to_sym == :false
+          namespaces = catalog_namespaces
+        else
+          namespaces = resource.provider.namespaces()
+        end
+        # Check if namespace exists on system (not defined in catalog)
+        if (resource[:ensure] && resource[:ensure].to_sym != :absent) && ! namespaces.include?(resource[:namespace])
           raise Puppet::Error, "Sensu namespace '#{resource[:namespace]}' must be defined or exist"
         end
       end

manifests/api.pp

@@ -9,9 +9,10 @@
   include sensu
 
   sensu_api_config { 'sensu':
-    url      => $sensu::api_url,
-    username => 'admin',
-    password => $sensu::password,
+    url                 => $sensu::api_url,
+    username            => 'admin',
+    password            => $sensu::password,
+    validate_namespaces => $sensu::validate_namespaces,
   }
 
   sensu_api_validator { 'sensu':

manifests/cli.pp

@@ -82,8 +82,9 @@
 
   if $configure {
     sensuctl_config { 'sensu':
-      chunk_size => $sensuctl_chunk_size,
-      path       => $sensuctl_path,
+      chunk_size          => $sensuctl_chunk_size,
+      path                => $sensuctl_path,
+      validate_namespaces => $sensu::validate_namespaces,
     }
 
     sensuctl_configure { 'puppet':

manifests/init.pp

@@ -59,6 +59,8 @@
 #   The sensu agent password
 # @param agent_old_password
 #   DEPRECATED - The sensu agent old password needed when changing agent_password
+# @param validate_namespaces
+#   Determines if sensuctl and sensu_api types will validate their namespace exists
 class sensu (
   String $version = 'installed',
   Stdlib::Absolutepath $etc_dir = '/etc/sensu',
@@ -79,6 +81,7 @@
   Optional[String] $old_password = undef,
   String $agent_password = 'P@ssw0rd!',
   Optional[String] $agent_old_password = undef,
+  Boolean $validate_namespaces = true,
 ) {
 
   if $old_password {

metadata.json

@@ -1,6 +1,6 @@
 {
   "name": "sensu-sensu",
-  "version": "4.11.0",
+  "version": "4.12.0",
   "author": "sensu",
   "summary": "A module to install the Sensu monitoring framework",
   "license": "MIT",

spec/classes/api_spec.rb

@@ -19,9 +19,10 @@
 
         it {
           should contain_sensu_api_config('sensu').with({
-            'url'      => 'https://test.example.com:8080',
-            'username' => 'admin',
-            'password' => 'P@ssw0rd!',
+            'url'                 => 'https://test.example.com:8080',
+            'username'            => 'admin',
+            'password'            => 'P@ssw0rd!',
+            'validate_namespaces' => 'true',
           })
         }
 

spec/classes/cli_spec.rb

@@ -66,6 +66,7 @@
         end
 
         it { should contain_sensuctl_config('sensu').without_chunk_size }
+        it { should contain_sensuctl_config('sensu').with_validate_namespaces('true') }
 
         it {
           should contain_sensuctl_configure('puppet').with({
@@ -99,6 +100,13 @@
         end
       end
 
+      context 'with validate_namespaces => false' do
+        let(:pre_condition) do
+          "class { 'sensu': validate_namespaces => false }"
+        end
+        it { should contain_sensuctl_config('sensu').with_validate_namespaces('false') }
+      end
+
       context 'with use_ssl => false' do
         let(:pre_condition) do
           "class { 'sensu': use_ssl => false }"

spec/shared_examples/namespace.rb

@@ -2,15 +2,16 @@
 
 RSpec.shared_examples 'namespace' do
   it 'should not fail if namespace defined' do
-    config[:namespace] = 'devs'
-    namespace = Puppet::Type.type(:sensu_namespace).new(:name => 'devs')
+    config[:namespace] = 'default'
+    namespace = Puppet::Type.type(:sensu_namespace).new(:name => 'default')
     catalog = Puppet::Resource::Catalog.new
     catalog.add_resource res
     catalog.add_resource namespace
     expect { res.pre_run_check }.not_to raise_error
   end
 
   it 'should not fail if namespace exists' do
+    allow(res.provider).to receive(:validate_namespaces).and_return(true)
     allow(res.provider).to receive(:namespaces).and_return(['devs','default'])
     config[:namespace] = 'devs'
     catalog = Puppet::Resource::Catalog.new