From d45e2814218c1ff2437dde43cb77d5c3e00e2bdc Mon Sep 17 00:00:00 2001 From: Pete Soloway Date: Tue, 7 Apr 2015 14:59:23 -0700 Subject: [PATCH 01/76] Updates README per 936dc29d9b3a1d7432089962ecdbbaefd36f0d8b * Removes references to the ruby script and holding space. * Mentions the `file_concat` dependency. * Removes the `ensure_newline`, `force`, and `ensure` parameter definitions. * Renames the `warn` parameter to `warn_header`. * Removes all deprecation listings. * Adds a list of removed features. * Cleans up some stray line breaks. Implements edits from @jbondpdx * Updates table of contents * Clarifies requirement * Specifies the module version where functionality was removed. * Clarifies changes to the `warn` parameter * Additional tweaks for clarity Final copy tweak * Further clarify the nature of boolean values in `warn_header`. --- README.md | 91 ++++++++++++++----------------------------------------- 1 file changed, 23 insertions(+), 68 deletions(-) diff --git a/README.md b/README.md index 873f6478..aba73b42 100644 --- a/README.md +++ b/README.md @@ -9,9 +9,9 @@ * [Beginning with concat](#beginning-with-concat) 4. [Usage - Configuration options and additional functionality](#usage) 5. [Reference - An under-the-hood peek at what the module is doing and how](#reference) - * [Public Defines](#public-defines) + * [Defines](#defines) * [Parameters](#parameters) - * [Deprecations](#deprecations) + * [Removed functionality](#removed-functionality) 6. [Limitations - OS compatibility, etc.](#limitations) 7. [Development - Guide for contributing to the module](#development) @@ -21,14 +21,13 @@ The concat module lets you construct files from multiple ordered fragments of te ##Module Description -The concat module lets you gather `concat::fragment` resources from your other modules and order them through a single `concat` resource into a coherent file. It does this through a Ruby script and a temporary holding space for the fragments. +The concat module lets you gather `concat::fragment` resources from your other modules and order them into a coherent file through a single `concat` resource. ##Setup ###What concat affects -* Installs `concatfragments.rb`. -* Adds a `concat/` directory into Puppet's `vardir`. +The concat module requires the [file_concat module](https://forge.puppetlabs.com/ispavailability/file_concat). If you don't have file_concat installed, concat installs it for you. ###Beginning with concat @@ -123,9 +122,7 @@ When you're finished, the motd file will look something like this: ##Reference -**Note**: Several of this module's parameters and features have been deprecated. See the [Deprecations](#deprecations) section below. - -###Public defines +###Defines * `concat`: Manages a file, compiled from one or more text fragments. * `concat::fragment`: Manages a fragment of text to be compiled into a file. @@ -143,15 +140,6 @@ Specifies whether (and how) to back up the destination file before overwriting i Specifies whether the destination file should exist. Setting to 'absent' tells Puppet to delete the destination file if it exists, and negates the effect of any other parameters. Valid options: 'present' and 'absent'. Default value: 'present'. - -#####`ensure_newline` - -Specifies whether to ensure there's a new line at the end of each fragment. Valid options: 'true' and 'false'. Default value: 'false'. - -#####`force` - -In case no fragments have been added, this parameter specifies whether to go ahead and create a potentially empty file. Valid options: 'true' and 'false'. Default value: 'false'. - #####`group` Specifies a permissions group for the destination file. Valid options: a string containing a group name. Default value: undefined. @@ -168,12 +156,10 @@ You can override this setting for individual fragments by adjusting the `order` #####`owner` - Specifies the owner of the destination file. Valid options: a string containing a username. Default value: undefined. #####`path` - Specifies a destination file for the combined fragments. Valid options: a string containing an absolute path. Default value: the title of your declared resource. #####`replace` @@ -184,12 +170,11 @@ Specifies whether to overwrite the destination file if it already exists. Valid Specifies a validation command to apply to the destination file. Requires Puppet version 3.5 or newer. Valid options: a string to be passed to a file resource. Default value: undefined. -#####`warn` - -Specifies whether to add a warning message at the top of the destination file so users know it was autogenerated by Puppet. Valid options: 'true', 'false', or a string to be delivered as a warning message. Default value: 'false'. +#####`warn_header` +Specifies whether to add a header message at the top of the destination file so users know it was autogenerated by Puppet. In earlier versions of the concat module, this parameter was called `warn`. Valid options: the booleans 'true' and 'false', or a string to serve as the header. Default value: 'false'. -If you set this parameter to 'true', Puppet adds the following message: +If you set 'warn_header' to 'true', `concat` adds the following message: ~~~ # This file is managed by Puppet. DO NOT EDIT. @@ -197,17 +182,12 @@ If you set this parameter to 'true', Puppet adds the following message: ####`concat::fragment` - Except where noted, all the below parameters are optional. #####`content` Supplies the content of the fragment. **Note**: You must supply either a `content` parameter or a `source` parameter. Valid options: a string. Default value: undef. -#####`ensure` - -Specifies whether the fragment should be included in the destination file or discarded. Valid options: 'present' and 'absent'. Default value: 'present'. - #####`order` Reorders your fragments within the destination file. Fragments that share the same order number are ordered by name. Valid options: a string (recommended) or an integer. Default value: '10'. @@ -220,51 +200,26 @@ Specifies a file to read into the content of the fragment. **Note**: You must su *Required.* Specifies the destination file of the fragment. Valid options: a string containing an absolute path. -###Deprecations - -**`concat` has the following deprecations** - -#####`gnu` - -Generates a catalog compile time warning and has no effect. This parameter was silently ignored in version `1.0.0` and will be removed in a future release. - -#####stringified 'true'/'false' values deprecated in `warn` - -Passing stringified boolean values (strings of 'true' and 'false') to the `warn` parameter of `concat` is deprecated. Generates a catalog compile time warning, and will be silently treated as the concatenated file header/warning message in a future release. - -Please migrate to using the Puppet DSL's native [Boolean data -type](http://docs.puppetlabs.com/puppet/3/reference/lang_datatypes.html#booleans). - - -**`concat::fragment` has the following deprecations** - -#####`backup` - -Generates a catalog compile time warning and has no effect. In the `1.0.0` release this parameter controlled file bucketing of the file fragment. Bucketing the fragment(s) is redundant with bucketing the final concatenated file and this feature has been removed. - - -#####`group` - -Generates a catalog compile time warning and has no effect. Had no user-visible effect in version `1.0.0` and will be removed in a future release. - -#####`mode` - -Generates a catalog compile time warning and has no effect. Had no user-visible effect in version `1.0.0` and will be removed in a future release. - - -#####`owner` - -Generates a catalog compile time warning and has no effect. Had no user-visible effect in version `1.0.0` and will be removed in a future release. +###Removed functionality -#####file paths are deprecated in `ensure` +The following functionality existed in previous versions of the concat module, but were removed in version 2.0.0: -Passing a value other than 'present' or 'absent' in the `ensure` parameter of `concat::fragment` is **deprecated**, and generates a catalog compile time warning. The warning will become a catalog compilation failure in a future release. +Parameters removed from `concat`: +* `ensure_newline` +* `force` +* `warn` (replaced by `warn_header`) -If you want to use the content of a file as a fragment please use the [`source`](#source) parameter. +Parameters removed from `concat::fragment`: +* `ensure` +* `gnu` +* `backup` +* `group` +* `mode` +* `owner` -####`concat::setup` +The `concat::setup` class has also been removed. -The `concat::setup` class should no longer be directly included in the manifest. It will be removed in a future release. +Prior to concat version 2.0.0, if you set the `warn` parameter to a string value of 'true', 'false', 'yes', 'no', 'on', or 'off', the module translated the string to the corresponding boolean value. In concat version 2.0.0 and newer, the `warn_header` parameter treats those values the same as other strings and uses them as the content of your header message. To avoid that, pass the 'true' and 'false' values as booleans instead of strings. ##Limitations From 51bd49fcc3773a8268e192c4c02d6a56418e4bce Mon Sep 17 00:00:00 2001 From: Bryan Jen Date: Wed, 25 Mar 2015 11:48:51 -0700 Subject: [PATCH 02/76] refactor concat to wrap electrical/file_concat --- .fixtures.yml | 5 +- .gitignore | 1 + Gemfile | 5 +- files/concatfragments.rb | 153 ----------- lib/facter/concat_basedir.rb | 11 - .../parser/functions/concat_getparam.rb | 35 --- lib/puppet/parser/functions/concat_is_bool.rb | 22 -- manifests/fragment.pp | 100 +------ manifests/init.pp | 208 +++------------ manifests/setup.pp | 64 ----- metadata.json | 13 +- spec/acceptance/backup_spec.rb | 2 +- spec/acceptance/concat_spec.rb | 64 ----- spec/acceptance/deprecation_warnings_spec.rb | 238 ----------------- spec/acceptance/empty_spec.rb | 23 -- spec/acceptance/fragment_source_spec.rb | 2 +- spec/acceptance/newline_spec.rb | 67 ----- spec/acceptance/nodesets/default.yml | 8 +- spec/acceptance/order_spec.rb | 4 +- spec/acceptance/replace_spec.rb | 3 +- .../{warn_spec.rb => warn_header_spec.rb} | 8 +- spec/spec_helper_acceptance.rb | 14 +- spec/unit/classes/concat_setup_spec.rb | 98 ------- spec/unit/defines/concat_fragment_spec.rb | 233 +---------------- spec/unit/defines/concat_spec.rb | 247 ++++-------------- spec/unit/facts/concat_basedir_spec.rb | 18 -- 26 files changed, 143 insertions(+), 1503 deletions(-) delete mode 100644 files/concatfragments.rb delete mode 100644 lib/facter/concat_basedir.rb delete mode 100644 lib/puppet/parser/functions/concat_getparam.rb delete mode 100644 lib/puppet/parser/functions/concat_is_bool.rb delete mode 100644 manifests/setup.pp delete mode 100644 spec/acceptance/deprecation_warnings_spec.rb delete mode 100644 spec/acceptance/empty_spec.rb delete mode 100644 spec/acceptance/newline_spec.rb rename spec/acceptance/{warn_spec.rb => warn_header_spec.rb} (94%) delete mode 100644 spec/unit/classes/concat_setup_spec.rb delete mode 100644 spec/unit/facts/concat_basedir_spec.rb diff --git a/.fixtures.yml b/.fixtures.yml index 67added0..fb0f76a6 100644 --- a/.fixtures.yml +++ b/.fixtures.yml @@ -2,6 +2,9 @@ fixtures: repositories: 'stdlib': repo: 'git://github.com/puppetlabs/puppetlabs-stdlib.git' - ref: '4.2.0' + ref: '4.5.1' + 'file_concat': + repo: 'git://github.com/electrical/puppet-lib-file_concat.git' + branch: '1.0.0' symlinks: 'concat': '#{source_dir}' diff --git a/.gitignore b/.gitignore index b5db85e0..ef01482d 100644 --- a/.gitignore +++ b/.gitignore @@ -7,3 +7,4 @@ spec/fixtures/ coverage/ .idea/ *.iml +*.swp diff --git a/Gemfile b/Gemfile index cc4048ae..f7319c22 100644 --- a/Gemfile +++ b/Gemfile @@ -12,13 +12,14 @@ end group :development, :unit_tests do gem 'rake', :require => false - gem 'rspec-core', '3.1.7', :require => false - gem 'rspec-puppet', '~> 1.0', :require => false + gem 'rspec-core', '~> 3.1.7', :require => false + gem 'rspec-puppet', '~> 2.0', :require => false gem 'puppetlabs_spec_helper', :require => false gem 'puppet-lint', :require => false gem 'simplecov', :require => false gem 'puppet_facts', :require => false gem 'json', :require => false + gem 'pry' end beaker_version = ENV['BEAKER_VERSION'] diff --git a/files/concatfragments.rb b/files/concatfragments.rb deleted file mode 100644 index b16f3e13..00000000 --- a/files/concatfragments.rb +++ /dev/null @@ -1,153 +0,0 @@ -#!/usr/bin/env ruby -# Script to concat files to a config file. -# -# Given a directory like this: -# /path/to/conf.d -# |-- fragments -# | |-- 00_named.conf -# | |-- 10_domain.net -# | `-- zz_footer -# -# The script supports a test option that will build the concat file to a temp location and -# use /usr/bin/cmp to verify if it should be run or not. This would result in the concat happening -# twice on each run but gives you the option to have an unless option in your execs to inhibit rebuilds. -# -# Without the test option and the unless combo your services that depend on the final file would end up -# restarting on each run, or in other manifest models some changes might get missed. -# -# OPTIONS: -# -o The file to create from the sources -# -d The directory where the fragments are kept -# -t Test to find out if a build is needed, basically concats the files to a temp -# location and compare with what's in the final location, return codes are designed -# for use with unless on an exec resource -# -w Add a shell style comment at the top of the created file to warn users that it -# is generated by puppet -# -f Enables the creation of empty output files when no fragments are found -# -n Sort the output numerically rather than the default alpha sort -# -# the command: -# -# concatfragments.rb -o /path/to/conffile.cfg -d /path/to/conf.d -# -# creates /path/to/conf.d/fragments.concat and copies the resulting -# file to /path/to/conffile.cfg. The files will be sorted alphabetically -# pass the -n switch to sort numerically. -# -# The script does error checking on the various dirs and files to make -# sure things don't fail. -require 'optparse' -require 'fileutils' - -settings = { - :outfile => "", - :workdir => "", - :test => false, - :force => false, - :warn => "", - :sortarg => "", - :newline => false -} - -OptionParser.new do |opts| - opts.banner = "Usage: #{$0} [options]" - opts.separator "Specific options:" - - opts.on("-o", "--outfile OUTFILE", String, "The file to create from the sources") do |o| - settings[:outfile] = o - end - - opts.on("-d", "--workdir WORKDIR", String, "The directory where the fragments are kept") do |d| - settings[:workdir] = d - end - - opts.on("-t", "--test", "Test to find out if a build is needed") do - settings[:test] = true - end - - opts.separator "Other options:" - opts.on("-w", "--warn WARNMSG", String, - "Add a shell style comment at the top of the created file to warn users that it is generated by puppet") do |w| - settings[:warn] = w - end - - opts.on("-f", "--force", "Enables the creation of empty output files when no fragments are found") do - settings[:force] = true - end - - opts.on("-n", "--sort", "Sort the output numerically rather than the default alpha sort") do - settings[:sortarg] = "-n" - end - - opts.on("-l", "--line", "Append a newline") do - settings[:newline] = true - end -end.parse! - -# do we have -o? -raise 'Please specify an output file with -o' unless !settings[:outfile].empty? - -# do we have -d? -raise 'Please specify fragments directory with -d' unless !settings[:workdir].empty? - -# can we write to -o? -if File.file?(settings[:outfile]) - if !File.writable?(settings[:outfile]) - raise "Cannot write to #{settings[:outfile]}" - end -else - if !File.writable?(File.dirname(settings[:outfile])) - raise "Cannot write to dirname #{File.dirname(settings[:outfile])} to create #{settings[:outfile]}" - end -end - -# do we have a fragments subdir inside the work dir? -if !File.directory?(File.join(settings[:workdir], "fragments")) && !File.executable?(File.join(settings[:workdir], "fragments")) - raise "Cannot access the fragments directory" -end - -# are there actually any fragments? -if (Dir.entries(File.join(settings[:workdir], "fragments")) - %w{ . .. }).empty? - if !settings[:force] - raise "The fragments directory is empty, cowardly refusing to make empty config files" - end -end - -Dir.chdir(settings[:workdir]) - -if settings[:warn].empty? - File.open("fragments.concat", 'w') { |f| f.write("") } -else - File.open("fragments.concat", 'w') { |f| f.write("#{settings[:warn]}\n") } -end - -# find all the files in the fragments directory, sort them numerically and concat to fragments.concat in the working dir -open('fragments.concat', 'a') do |f| - fragments = Dir.entries("fragments").sort - if settings[:sortarg] == '-n' - fragments = fragments.sort_by { |v| v.split('_').map(&:to_i) } - end - fragments.each { |entry| - if File.file?(File.join("fragments", entry)) - f << File.read(File.join("fragments", entry)) - - # append a newline if we were asked to (invoked with -l) - if settings[:newline] - f << "\n" - end - - end - } -end - -if !settings[:test] - # This is a real run, copy the file to outfile - FileUtils.cp 'fragments.concat', settings[:outfile] -else - # Just compare the result to outfile to help the exec decide - if FileUtils.cmp 'fragments.concat', settings[:outfile] - exit 0 - else - exit 1 - end -end diff --git a/lib/facter/concat_basedir.rb b/lib/facter/concat_basedir.rb deleted file mode 100644 index bfac0710..00000000 --- a/lib/facter/concat_basedir.rb +++ /dev/null @@ -1,11 +0,0 @@ -# == Fact: concat_basedir -# -# A custom fact that sets the default location for fragments -# -# "${::vardir}/concat/" -# -Facter.add("concat_basedir") do - setcode do - File.join(Puppet[:vardir],"concat") - end -end diff --git a/lib/puppet/parser/functions/concat_getparam.rb b/lib/puppet/parser/functions/concat_getparam.rb deleted file mode 100644 index 1757bdc5..00000000 --- a/lib/puppet/parser/functions/concat_getparam.rb +++ /dev/null @@ -1,35 +0,0 @@ -# Test whether a given class or definition is defined -require 'puppet/parser/functions' - -Puppet::Parser::Functions.newfunction(:concat_getparam, - :type => :rvalue, - :doc => <<-'ENDOFDOC' -Takes a resource reference and name of the parameter and -returns value of resource's parameter. - -*Examples:* - - define example_resource($param) { - } - - example_resource { "example_resource_instance": - param => "param_value" - } - - concat_getparam(Example_resource["example_resource_instance"], "param") - -Would return: param_value -ENDOFDOC -) do |vals| - reference, param = vals - raise(ArgumentError, 'Must specify a reference') unless reference - raise(ArgumentError, 'Must specify name of a parameter') unless param and param.instance_of? String - - return '' if param.empty? - - if resource = findresource(reference.to_s) - return resource[param] if resource[param] - end - - return '' -end diff --git a/lib/puppet/parser/functions/concat_is_bool.rb b/lib/puppet/parser/functions/concat_is_bool.rb deleted file mode 100644 index c2c2a9f2..00000000 --- a/lib/puppet/parser/functions/concat_is_bool.rb +++ /dev/null @@ -1,22 +0,0 @@ -# -# concat_is_bool.rb -# - -module Puppet::Parser::Functions - newfunction(:concat_is_bool, :type => :rvalue, :doc => <<-EOS -Returns true if the variable passed to this function is a boolean. - EOS - ) do |arguments| - - raise(Puppet::ParseError, "concat_is_bool(): Wrong number of arguments " + - "given (#{arguments.size} for 1)") if arguments.size != 1 - - type = arguments[0] - - result = type.is_a?(TrueClass) || type.is_a?(FalseClass) - - return result - end -end - -# vim: set ts=2 sw=2 et : diff --git a/manifests/fragment.pp b/manifests/fragment.pp index f9ee2a7c..5a3e3104 100644 --- a/manifests/fragment.pp +++ b/manifests/fragment.pp @@ -1,6 +1,6 @@ # == Define: concat::fragment # -# Puts a file fragment into a directory previous setup using concat +# Creates a file_fragment in the catalogue # # === Options: # @@ -13,115 +13,37 @@ # [*order*] # By default all files gets a 10_ prefix in the directory you can set it to # anything else using this to influence the order of the content in the file -# [*ensure*] -# Present/Absent or destination to a file to include another file -# [*mode*] -# Deprecated -# [*owner*] -# Deprecated -# [*group*] -# Deprecated -# [*backup*] -# Deprecated # define concat::fragment( $target, $content = undef, $source = undef, $order = '10', - $ensure = undef, - $mode = undef, - $owner = undef, - $group = undef, - $backup = undef ) { validate_string($target) validate_string($content) if !(is_string($source) or is_array($source)) { fail('$source is not a string or an Array.') } + if !(is_string($order) or is_integer($order)) { fail('$order is not a string or integer.') } elsif (is_string($order) and $order =~ /[:\n\/]/) { fail("Order cannot contain '/', ':', or '\n'.") } - if $mode { - warning('The $mode parameter to concat::fragment is deprecated and has no effect') - } - if $owner { - warning('The $owner parameter to concat::fragment is deprecated and has no effect') - } - if $group { - warning('The $group parameter to concat::fragment is deprecated and has no effect') - } - if $backup { - warning('The $backup parameter to concat::fragment is deprecated and has no effect') - } - $my_backup = concat_getparam(Concat[$target], 'backup') - if $ensure == undef { - $my_ensure = concat_getparam(Concat[$target], 'ensure') - } else { - if ! ($ensure in [ 'present', 'absent' ]) { - warning('Passing a value other than \'present\' or \'absent\' as the $ensure parameter to concat::fragment is deprecated. If you want to use the content of a file as a fragment please use the $source parameter.') - } - $my_ensure = $ensure - } - - include concat::setup - - $safe_name = regsubst($name, '[/:\n]', '_', 'GM') - $safe_target_name = regsubst($target, '[/:\n]', '_', 'GM') - $concatdir = $concat::setup::concatdir - $fragdir = "${concatdir}/${safe_target_name}" - $fragowner = $concat::setup::fragment_owner - $fraggroup = $concat::setup::fragment_group - $fragmode = $concat::setup::fragment_mode - - # The file type's semantics are problematic in that ensure => present will - # not over write a pre-existing symlink. We are attempting to provide - # backwards compatiblity with previous concat::fragment versions that - # supported the file type's ensure => /target syntax - - # be paranoid and only allow the fragment's file resource's ensure param to - # be file, absent, or a file target - $safe_ensure = $my_ensure ? { - '' => 'file', - undef => 'file', - 'file' => 'file', - 'present' => 'file', - 'absent' => 'absent', - default => $my_ensure, - } - # if it looks line ensure => /target syntax was used, fish that out - if ! ($my_ensure in ['', 'present', 'absent', 'file' ]) { - $ensure_target = $my_ensure - } else { - $ensure_target = undef - } - - # the file type's semantics only allows one of: ensure => /target, content, - # or source - if ($ensure_target and $source) or - ($ensure_target and $content) or - ($source and $content) { - fail('You cannot specify more than one of $content, $source, $ensure => /target') - } - - if ! ($content or $source or $ensure_target) { + if ! ($content or $source) { crit('No content, source or symlink specified') + } elsif ($content and $source) { + fail("Can't use 'source' and 'content' at the same time") } - file { "${fragdir}/fragments/${order}_${safe_name}": - ensure => $safe_ensure, - owner => $fragowner, - group => $fraggroup, - mode => $fragmode, - source => $source, + $safe_target_name = regsubst($target, '[/:\n\s]', '_', 'GM') + + file_fragment { $name: + tag => $safe_target_name, + order => $order, content => $content, - backup => $my_backup, - replace => true, - alias => "concat_fragment_${name}", - notify => Exec["concat_${target}"] + source => $source, } } diff --git a/manifests/init.pp b/manifests/init.pp index c44c0328..305b3d97 100644 --- a/manifests/init.pp +++ b/manifests/init.pp @@ -16,34 +16,21 @@ # Who will own the file # [*mode*] # The mode of the final file -# [*force*] -# Enables creating empty files if no fragments are present -# [*warn*] +# [*warn_header*] # Adds a normal shell style comment top of the file indicating that it is # built by puppet -# [*force*] # [*backup*] # Controls the filebucketing behavior of the final file and see File type # reference for its use. Defaults to 'puppet' # [*replace*] # Whether to replace a file that already exists on the local system # [*order*] -# [*ensure_newline*] -# [*gnu*] -# Deprecated +# Select whether to order associated fragments by 'alpha' or 'numeric'. +# Defaults to 'alpha'. # # === Actions: -# * Creates fragment directories if it didn't exist already -# * Executes the concatfragments.rb script to build the final file, this -# script will create directory/fragments.concat. Execution happens only -# when: -# * The directory changes -# * fragments.concat != final destination, this means rebuilds will happen -# whenever someone changes or deletes the final file. Checking is done -# using /usr/bin/cmp. -# * The Exec gets notified by something else - like the concat::fragment -# define -# * Copies the file over to the final destination using a file resource +# * Creates a file_concat resource from the electrical/puppet-lib-file_concat library. +# * Creates file_fragment resources from electrical/puppet-lib-file_concat # # === Aliases: # @@ -52,213 +39,80 @@ # * The final file can be referenced as File["/path/to/file"] or # File["concat_/path/to/file"] # + define concat( $ensure = 'present', $path = $name, $owner = undef, $group = undef, $mode = '0644', - $warn = false, - $force = false, + $warn_header = false, $backup = 'puppet', $replace = true, $order = 'alpha', - $ensure_newline = false, $validate_cmd = undef, - $gnu = undef ) { validate_re($ensure, '^present$|^absent$') validate_absolute_path($path) validate_string($owner) validate_string($group) validate_string($mode) - if ! (is_string($warn) or $warn == true or $warn == false) { - fail('$warn is not a string or boolean') + if ! (is_string($warn_header) or $warn_header == true or $warn_header == false) { + fail('$warn_header is not a string or boolean') } - validate_bool($force) - if ! concat_is_bool($backup) and ! is_string($backup) { + if ! is_bool($backup) and ! is_string($backup) { fail('$backup must be string or bool!') } validate_bool($replace) validate_re($order, '^alpha$|^numeric$') - validate_bool($ensure_newline) if $validate_cmd and ! is_string($validate_cmd) { fail('$validate_cmd must be a string') } - if $gnu { - warning('The $gnu parameter to concat is deprecated and has no effect') - } - - include concat::setup - $safe_name = regsubst($name, '[/:]', '_', 'G') - $concatdir = $concat::setup::concatdir - $fragdir = "${concatdir}/${safe_name}" - $concat_name = 'fragments.concat.out' - $script_command = $concat::setup::script_command - $default_warn_message = '# This file is managed by Puppet. DO NOT EDIT.' - $bool_warn_message = 'Using stringified boolean values (\'true\', \'yes\', \'on\', \'false\', \'no\', \'off\') to represent boolean true/false as the $warn parameter to concat is deprecated and will be treated as the warning message in a future release' + $safe_name = regsubst($name, '[/:\n\s]', '_', 'G') + $default_warn_message = "# This file is managed by Puppet. DO NOT EDIT.\n" - case $warn { + case $warn_header { true: { $warn_message = $default_warn_message - } - 'true', 'yes', 'on': { - warning($bool_warn_message) - $warn_message = $default_warn_message + $append_header = true } false: { $warn_message = '' } - 'false', 'no', 'off': { - warning($bool_warn_message) - $warn_message = '' - } default: { - $warn_message = $warn + $warn_message = $warn_header + $append_header = true } } - $warnmsg_escaped = regsubst($warn_message, '\'', '\'\\\'\'', 'G') - $warnflag = $warnmsg_escaped ? { - '' => '', - default => "-w '${warnmsg_escaped}'" - } - - $forceflag = $force ? { - true => '-f', - false => '', - } - - $orderflag = $order ? { - 'numeric' => '-n', - 'alpha' => '', - } - - $newlineflag = $ensure_newline ? { - true => '-l', - false => '', - } - - File { - backup => $backup, - } - - # reset poisoned Exec defaults - Exec { - user => undef, - group => undef, - } - if $ensure == 'present' { - file { $fragdir: - ensure => directory, - mode => '0750', - } - - file { "${fragdir}/fragments": - ensure => directory, - mode => '0750', - force => true, - ignore => ['.svn', '.git', '.gitignore'], - notify => Exec["concat_${name}"], - purge => true, - recurse => true, - } - - file { "${fragdir}/fragments.concat": - ensure => present, - mode => '0640', - } - - file { "${fragdir}/${concat_name}": - ensure => present, - mode => '0640', - } - - file { $name: - ensure => present, - owner => $owner, + file_concat { $name: + tag => $safe_name, + path => $path, + owner => $user, group => $group, mode => $mode, replace => $replace, - path => $path, - alias => "concat_${name}", - source => "${fragdir}/${concat_name}", backup => $backup, + order => $order, + validate_cmd => $validate_cmd, } - # Only newer versions of puppet 3.x support the validate_cmd parameter - if $validate_cmd { - File[$name] { - validate_cmd => $validate_cmd, - } - } - - # remove extra whitespace from string interpolation to make testing easier - $command = strip(regsubst("${script_command} -o \"${fragdir}/${concat_name}\" -d \"${fragdir}\" ${warnflag} ${forceflag} ${orderflag} ${newlineflag}", '\s+', ' ', 'G')) - - # make sure ruby is in the path for PE - if defined('$is_pe') and $::is_pe { - if $::kernel == 'windows' { - $command_path = "${::env_windows_installdir}/bin:${::path}" - } else { - $command_path = "/opt/puppet/bin:${::path}" + if $append_header { + file_fragment { "#{$name}_header": + tag => $safe_name, + content => $warn_message, + order => '0', } - } else { - $command_path = $::path - } - - # if puppet is running as root, this exec should also run as root to allow - # the concatfragments.rb script to potentially be installed in path that - # may not be accessible by a target non-root owner. - exec { "concat_${name}": - alias => "concat_${fragdir}", - command => $command, - notify => File[$name], - subscribe => File[$fragdir], - unless => "${command} -t", - path => $command_path, - require => [ - File[$fragdir], - File["${fragdir}/fragments"], - File["${fragdir}/fragments.concat"], - ], } } else { - file { [ - $fragdir, - "${fragdir}/fragments", - "${fragdir}/fragments.concat", - "${fragdir}/${concat_name}" - ]: - ensure => absent, - force => true, - } - - file { $path: - ensure => absent, + file_concat { $name: + ensure => $ensure, + tag => $safe_name, + path => $path, backup => $backup, } - - $absent_exec_command = $::kernel ? { - 'windows' => 'cmd.exe /c exit 0', - default => 'true', - } - - $absent_exec_path = $::kernel ? { - 'windows' => $::path, - default => '/bin:/usr/bin', - } - - # Need to have an unless here for idempotency. - exec { "concat_${name}": - alias => "concat_${fragdir}", - command => $absent_exec_command, - unless => $absent_exec_command, - path => $absent_exec_path, - } } } -# vim:sw=2:ts=2:expandtab:textwidth=79 diff --git a/manifests/setup.pp b/manifests/setup.pp deleted file mode 100644 index c5aedd82..00000000 --- a/manifests/setup.pp +++ /dev/null @@ -1,64 +0,0 @@ -# === Class: concat::setup -# -# Sets up the concat system. This is a private class. -# -# [$concatdir] -# is where the fragments live and is set on the fact concat_basedir. -# Since puppet should always manage files in $concatdir and they should -# not be deleted ever, /tmp is not an option. -# -# It also copies out the concatfragments.{sh,rb} file to ${concatdir}/bin -# -class concat::setup { - if $caller_module_name != $module_name { - warning("${name} is deprecated as a public API of the ${module_name} module and should no longer be directly included in the manifest.") - } - - if $::concat_basedir { - $concatdir = $::concat_basedir - } else { - fail ('$concat_basedir not defined. Try running again with pluginsync=true on the [master] and/or [main] section of your node\'s \'/etc/puppet/puppet.conf\'.') - } - - # owner,group and mode of fragment files (on windows owner and access rights should - # be inherited from concatdir and not explicitly set to avoid problems) - $fragment_owner = $::osfamily ? { 'windows' => undef, default => $::id } - $fragment_mode = $::osfamily ? { 'windows' => undef, default => '0640' } - # test on gid fact availability to support older facter versions - if defined('$gid') and $::gid and $::osfamily != 'Windows' { - $fragment_group = $::gid - } else { - $fragment_group = undef - } - - $script_name = 'concatfragments.rb' - - $script_path = "${concatdir}/bin/${script_name}" - - $default_owner = $::osfamily ? { 'windows' => undef, default => $::id } - - $default_group = $default_owner ? { 'root' => '0', default => undef } - - $script_mode = $::osfamily ? { 'windows' => undef, default => '0755' } - - $script_command = $::osfamily? { - 'windows' => "ruby.exe '${script_path}'", - 'openbsd' => "/usr/local/bin/ruby21 '${script_path}'", - default => $script_path - } - - file { $script_path: - ensure => file, - owner => $default_owner, - group => $default_group, - mode => $script_mode, - source => "puppet:///modules/concat/${script_name}", - } - - file { [ $concatdir, "${concatdir}/bin" ]: - ensure => directory, - owner => $default_owner, - group => $default_group, - mode => '0755', - } -} diff --git a/metadata.json b/metadata.json index 9d7a46dd..49c24c60 100644 --- a/metadata.json +++ b/metadata.json @@ -1,6 +1,6 @@ { "name": "puppetlabs-concat", - "version": "1.2.0", + "version": "2.0.0", "author": "Puppet Labs", "summary": "Construct files from multiple fragments.", "license": "Apache-2.0", @@ -105,6 +105,13 @@ } ], "dependencies": [ - {"name":"puppetlabs/stdlib","version_requirement":">= 3.2.0 < 5.0.0"} - ] + { + "name": "puppetlabs/stdlib", + "version_requirement": ">= 3.2.0 < 5.0.0" + }, + { + "name": "electrical/file_concat", + "version_requirement": ">= 1.0.0" + }, + ], } diff --git a/spec/acceptance/backup_spec.rb b/spec/acceptance/backup_spec.rb index 1d3a5dfe..0a737a55 100644 --- a/spec/acceptance/backup_spec.rb +++ b/spec/acceptance/backup_spec.rb @@ -26,7 +26,7 @@ it 'applies the manifest twice with "Filebucketed" stdout and no stderr' do apply_manifest(pp, :catch_failures => true) do |r| - expect(r.stdout).to match(/Filebucketed #{basedir}\/file to puppet with sum 0140c31db86293a1a1e080ce9b91305f/) # sum is for file contents of 'old contents' + expect(r.stdout).to match(/Filebucketed #{basedir}\/file to puppet with sum 0140c31db86293a1a1e080ce9b91305f/) end apply_manifest(pp, :catch_changes => true) end diff --git a/spec/acceptance/concat_spec.rb b/spec/acceptance/concat_spec.rb index c5c97277..c281954b 100644 --- a/spec/acceptance/concat_spec.rb +++ b/spec/acceptance/concat_spec.rb @@ -38,56 +38,6 @@ apply_manifest(pp, :catch_failures => true) apply_manifest(pp, :catch_changes => true) end - - describe file("#{vardir}/concat") do - it { should be_directory } - it { should be_owned_by username } - it("should be mode", :unless => (fact('osfamily') == 'AIX' or fact('osfamily') == 'windows')) { - should be_mode 755 - } - end - describe file("#{vardir}/concat/bin") do - it { should be_directory } - it { should be_owned_by username } - it("should be mode", :unless => (fact('osfamily') == 'AIX' or fact('osfamily') == 'windows')) { - should be_mode 755 - } - end - describe file("#{vardir}/concat/bin/#{scriptname}") do - it { should be_file } - it { should be_owned_by username } - it("should be mode", :unless => (fact('osfamily') == 'AIX' or fact('osfamily') == 'windows')) { - should be_mode 755 - } - end - describe file("#{vardir}/concat/#{safe_basedir}_file") do - it { should be_directory } - it { should be_owned_by username } - it("should be mode", :unless => (fact('osfamily') == 'AIX' or fact('osfamily') == 'windows')) { - should be_mode 750 - } - end - describe file("#{vardir}/concat/#{safe_basedir}_file/fragments") do - it { should be_directory } - it { should be_owned_by username } - it("should be mode", :unless => (fact('osfamily') == 'AIX' or fact('osfamily') == 'windows')) { - should be_mode 750 - } - end - describe file("#{vardir}/concat/#{safe_basedir}_file/fragments.concat") do - it { should be_file } - it { should be_owned_by username } - it("should be mode", :unless => (fact('osfamily') == 'AIX' or fact('osfamily') == 'windows')) { - should be_mode 640 - } - end - describe file("#{vardir}/concat/#{safe_basedir}_file/fragments.concat.out") do - it { should be_file } - it { should be_owned_by username } - it("should be mode", :unless => (fact('osfamily') == 'AIX' or fact('osfamily') == 'windows')) { - should be_mode 640 - } - end end context 'owner/group root' do @@ -133,20 +83,6 @@ should match '2' } end - describe file("#{vardir}/concat/#{safe_basedir}_file/fragments/01_1") do - it { should be_file } - it { should be_owned_by username } - it("should be mode", :unless => (fact('osfamily') == 'AIX' or fact('osfamily') == 'windows')) { - should be_mode 640 - } - end - describe file("#{vardir}/concat/#{safe_basedir}_file/fragments/02_2") do - it { should be_file } - it { should be_owned_by username } - it("should be mode", :unless => (fact('osfamily') == 'AIX' or fact('osfamily') == 'windows')) { - should be_mode 640 - } - end end context 'ensure' do diff --git a/spec/acceptance/deprecation_warnings_spec.rb b/spec/acceptance/deprecation_warnings_spec.rb deleted file mode 100644 index 11133ea9..00000000 --- a/spec/acceptance/deprecation_warnings_spec.rb +++ /dev/null @@ -1,238 +0,0 @@ -require 'spec_helper_acceptance' - -describe 'deprecation warnings' do - basedir = default.tmpdir('concat') - - shared_examples 'has_warning' do |pp, w| - it 'applies the manifest twice with a stderr regex' do - expect(apply_manifest(pp, :catch_failures => true).stderr).to match(/#{Regexp.escape(w)}/m) - expect(apply_manifest(pp, :catch_changes => true).stderr).to match(/#{Regexp.escape(w)}/m) - end - end - - context 'concat gnu parameter' do - pp = <<-EOS - concat { '#{basedir}/file': - gnu => 'foo', - } - concat::fragment { 'foo': - target => '#{basedir}/file', - content => 'bar', - } - EOS - w = 'The $gnu parameter to concat is deprecated and has no effect' - - it_behaves_like 'has_warning', pp, w - end - - context 'concat warn parameter =>' do - ['true', 'yes', 'on'].each do |warn| - context warn do - pp = <<-EOS - concat { '#{basedir}/file': - warn => '#{warn}', - } - concat::fragment { 'foo': - target => '#{basedir}/file', - content => 'bar', - } - EOS - w = 'Using stringified boolean values (\'true\', \'yes\', \'on\', \'false\', \'no\', \'off\') to represent boolean true/false as the $warn parameter to concat is deprecated and will be treated as the warning message in a future release' - - it_behaves_like 'has_warning', pp, w - - describe file("#{basedir}/file") do - it { should be_file } - its(:content) { - should match '# This file is managed by Puppet. DO NOT EDIT.' - should match 'bar' - } - end - end - end - - ['false', 'no', 'off'].each do |warn| - context warn do - pp = <<-EOS - concat { '#{basedir}/file': - warn => '#{warn}', - } - concat::fragment { 'foo': - target => '#{basedir}/file', - content => 'bar', - } - EOS - w = 'Using stringified boolean values (\'true\', \'yes\', \'on\', \'false\', \'no\', \'off\') to represent boolean true/false as the $warn parameter to concat is deprecated and will be treated as the warning message in a future release' - - it_behaves_like 'has_warning', pp, w - - describe file("#{basedir}/file") do - it { should be_file } - its(:content) { - should_not match '# This file is managed by Puppet. DO NOT EDIT.' - should match 'bar' - } - end - end - end - end - - context 'concat::fragment ensure parameter', :unless => fact('osfamily') == 'windows' do - context 'target file exists' do - before(:all) do - pp = <<-EOS - file { '#{basedir}': - ensure => directory, - } - file { '#{basedir}/file1': - content => "file1 contents\n", - } - EOS - apply_manifest(pp) - end - - pp = <<-EOS - concat { '#{basedir}/file': } - concat::fragment { 'foo': - target => '#{basedir}/file', - ensure => '#{basedir}/file1', - } - EOS - w = 'Passing a value other than \'present\' or \'absent\' as the $ensure parameter to concat::fragment is deprecated. If you want to use the content of a file as a fragment please use the $source parameter.' - - it_behaves_like 'has_warning', pp, w - - describe file("#{basedir}/file") do - it { should be_file } - its(:content) { should match 'file1 contents' } - end - - describe 'the fragment can be changed from a symlink to a plain file', :unless => (fact("osfamily") == "windows") do - pp = <<-EOS - concat { '#{basedir}/file': } - concat::fragment { 'foo': - target => '#{basedir}/file', - content => 'new content', - } - EOS - - it 'applies the manifest twice with no stderr' do - apply_manifest(pp, :catch_failures => true) - apply_manifest(pp, :catch_changes => true) - end - - describe file("#{basedir}/file") do - it { should be_file } - its(:content) { - should match 'new content' - should_not match 'file1 contents' - } - end - end - end # target file exists - - context 'target does not exist', :unless => fact('osfamily') == 'windows' do - pp = <<-EOS - concat { '#{basedir}/file': } - concat::fragment { 'foo': - target => '#{basedir}/file', - ensure => '#{basedir}/file1', - } - EOS - w = 'Passing a value other than \'present\' or \'absent\' as the $ensure parameter to concat::fragment is deprecated. If you want to use the content of a file as a fragment please use the $source parameter.' - - it_behaves_like 'has_warning', pp, w - - describe file("#{basedir}/file") do - it { should be_file } - end - - describe 'the fragment can be changed from a symlink to a plain file', :unless => (fact('osfamily') == 'windows') do - pp = <<-EOS - concat { '#{basedir}/file': } - concat::fragment { 'foo': - target => '#{basedir}/file', - content => 'new content', - } - EOS - - it 'applies the manifest twice with no stderr' do - apply_manifest(pp, :catch_failures => true) - apply_manifest(pp, :catch_changes => true) - end - - describe file("#{basedir}/file") do - it { should be_file } - its(:content) { should match 'new content' } - end - end - end # target file exists - - end # concat::fragment ensure parameter - - context 'concat::fragment mode parameter' do - pp = <<-EOS - concat { '#{basedir}/file': } - concat::fragment { 'foo': - target => '#{basedir}/file', - content => 'bar', - mode => 'bar', - } - EOS - w = 'The $mode parameter to concat::fragment is deprecated and has no effect' - - it_behaves_like 'has_warning', pp, w - end - - context 'concat::fragment owner parameter' do - pp = <<-EOS - concat { '#{basedir}/file': } - concat::fragment { 'foo': - target => '#{basedir}/file', - content => 'bar', - owner => 'bar', - } - EOS - w = 'The $owner parameter to concat::fragment is deprecated and has no effect' - - it_behaves_like 'has_warning', pp, w - end - - context 'concat::fragment group parameter' do - pp = <<-EOS - concat { '#{basedir}/file': } - concat::fragment { 'foo': - target => '#{basedir}/file', - content => 'bar', - group => 'bar', - } - EOS - w = 'The $group parameter to concat::fragment is deprecated and has no effect' - - it_behaves_like 'has_warning', pp, w - end - - context 'concat::fragment backup parameter' do - pp = <<-EOS - concat { '#{basedir}/file': } - concat::fragment { 'foo': - target => '#{basedir}/file', - content => 'bar', - backup => 'bar', - } - EOS - w = 'The $backup parameter to concat::fragment is deprecated and has no effect' - - it_behaves_like 'has_warning', pp, w - end - - context 'include concat::setup' do - pp = <<-EOS - include concat::setup - EOS - w = 'concat::setup is deprecated as a public API of the concat module and should no longer be directly included in the manifest.' - - it_behaves_like 'has_warning', pp, w - end - -end diff --git a/spec/acceptance/empty_spec.rb b/spec/acceptance/empty_spec.rb deleted file mode 100644 index 4ab6e1e6..00000000 --- a/spec/acceptance/empty_spec.rb +++ /dev/null @@ -1,23 +0,0 @@ -require 'spec_helper_acceptance' - -describe 'concat force empty parameter' do - basedir = default.tmpdir('concat') - context 'should run successfully' do - pp = <<-EOS - concat { '#{basedir}/file': - mode => '0644', - force => true, - } - EOS - - it 'applies the manifest twice with no stderr' do - apply_manifest(pp, :catch_failures => true) - apply_manifest(pp, :catch_changes => true) - end - - describe file("#{basedir}/file") do - it { should be_file } - its(:content) { should_not match /1\n2/ } - end - end -end diff --git a/spec/acceptance/fragment_source_spec.rb b/spec/acceptance/fragment_source_spec.rb index a174e02b..5d66e4d9 100644 --- a/spec/acceptance/fragment_source_spec.rb +++ b/spec/acceptance/fragment_source_spec.rb @@ -145,7 +145,7 @@ EOS it 'applies the manifest with resource failures' do - apply_manifest(pp, :expect_failures => true) + expect(apply_manifest(pp, :catch_failures => true).stderr).to match(/Failed to generate additional resources using 'eval_generate'/) end describe file("#{basedir}/fail_no_source") do #FIXME: Serverspec::Type::File doesn't support exists? for some reason. so... hack. diff --git a/spec/acceptance/newline_spec.rb b/spec/acceptance/newline_spec.rb deleted file mode 100644 index c1fa16a1..00000000 --- a/spec/acceptance/newline_spec.rb +++ /dev/null @@ -1,67 +0,0 @@ -require 'spec_helper_acceptance' - -describe 'concat ensure_newline parameter' do - basedir = default.tmpdir('concat') - context '=> false' do - before(:all) do - pp = <<-EOS - file { '#{basedir}': - ensure => directory - } - EOS - - apply_manifest(pp) - end - pp = <<-EOS - concat { '#{basedir}/file': - ensure_newline => false, - } - concat::fragment { '1': - target => '#{basedir}/file', - content => '1', - } - concat::fragment { '2': - target => '#{basedir}/file', - content => '2', - } - EOS - - it 'applies the manifest twice with no stderr' do - apply_manifest(pp, :catch_failures => true) - apply_manifest(pp, :catch_changes => true) - end - - describe file("#{basedir}/file") do - it { should be_file } - its(:content) { should match '12' } - end - end - - context '=> true' do - pp = <<-EOS - concat { '#{basedir}/file': - ensure_newline => true, - } - concat::fragment { '1': - target => '#{basedir}/file', - content => '1', - } - concat::fragment { '2': - target => '#{basedir}/file', - content => '2', - } - EOS - - it 'applies the manifest twice with no stderr' do - apply_manifest(pp, :catch_failures => true) - apply_manifest(pp, :catch_changes => true) - end - - describe file("#{basedir}/file") do - it { should be_file } - its(:content) { - should match /1\n2\n/ - } - end - end -end diff --git a/spec/acceptance/nodesets/default.yml b/spec/acceptance/nodesets/default.yml index b8268515..4e2cb809 100644 --- a/spec/acceptance/nodesets/default.yml +++ b/spec/acceptance/nodesets/default.yml @@ -1,10 +1,10 @@ HOSTS: - centos-66-x64: + centos-65-x64: roles: - master platform: el-6-x86_64 - box : puppetlabs/centos-6.6-64-nocm - box_url : http://puppet-vagrant-boxes.puppetlabs.com/centos-6.6-64-nocm + box : centos-65-x64-vbox436-nocm + box_url : http://puppet-vagrant-boxes.puppetlabs.com/centos-65-x64-virtualbox-nocm.box hypervisor : vagrant CONFIG: - type: git + type: foss diff --git a/spec/acceptance/order_spec.rb b/spec/acceptance/order_spec.rb index c158dd88..b746b1c9 100644 --- a/spec/acceptance/order_spec.rb +++ b/spec/acceptance/order_spec.rb @@ -37,7 +37,7 @@ end describe 'alpha' do - it_behaves_like 'sortby', 'alpha', /string10string1string2/ + it_behaves_like 'sortby', 'alpha', /string1string10string2/ end describe 'numeric' do @@ -83,7 +83,7 @@ end end describe 'alpha' do - it_should_behave_like 'order_by', 'alpha', /string2string1string3/ + it_should_behave_like 'order_by', 'alpha', /string3string2string1/ end describe 'numeric' do it_should_behave_like 'order_by', 'numeric', /string3string2string1/ diff --git a/spec/acceptance/replace_spec.rb b/spec/acceptance/replace_spec.rb index bd597ed1..68d7383f 100644 --- a/spec/acceptance/replace_spec.rb +++ b/spec/acceptance/replace_spec.rb @@ -226,7 +226,7 @@ end end - # XXX concat's force param currently enables the creation of empty files + # XXX # when there are no fragments, and the replace param will only replace # files and symlinks, not directories. The semantics either need to be # changed, extended, or a new param introduced to control directory @@ -234,7 +234,6 @@ context 'should succeed', :pending => 'not yet implemented' do pp = <<-EOS concat { '#{basedir}/file': - force => true, } concat::fragment { '1': diff --git a/spec/acceptance/warn_spec.rb b/spec/acceptance/warn_header_spec.rb similarity index 94% rename from spec/acceptance/warn_spec.rb rename to spec/acceptance/warn_header_spec.rb index 2788607c..52615e1d 100644 --- a/spec/acceptance/warn_spec.rb +++ b/spec/acceptance/warn_header_spec.rb @@ -1,11 +1,11 @@ require 'spec_helper_acceptance' -describe 'concat warn =>' do +describe 'concat warn_header =>' do basedir = default.tmpdir('concat') context 'true should enable default warning message' do pp = <<-EOS concat { '#{basedir}/file': - warn => true, + warn_header => true, } concat::fragment { '1': @@ -38,7 +38,7 @@ context 'false should not enable default warning message' do pp = <<-EOS concat { '#{basedir}/file': - warn => false, + warn_header => false, } concat::fragment { '1': @@ -71,7 +71,7 @@ context '# foo should overide default warning message' do pp = <<-EOS concat { '#{basedir}/file': - warn => '# foo', + warn_header => "# foo\n", } concat::fragment { '1': diff --git a/spec/spec_helper_acceptance.rb b/spec/spec_helper_acceptance.rb index da994f8d..1a3ffc0a 100644 --- a/spec/spec_helper_acceptance.rb +++ b/spec/spec_helper_acceptance.rb @@ -16,21 +16,29 @@ hosts.each do |host| on hosts, "mkdir -p #{host['distmoduledir']}" if host['platform'] =~ /sles-1/i || host['platform'] =~ /solaris-1/i - get_stdlib = <<-EOS + get_deps = <<-EOS package{'wget':} - exec{'download': + exec{'download-stdlib': command => "wget -P /root/ https://forgeapi.puppetlabs.com/v3/files/puppetlabs-stdlib-4.5.1.tar.gz --no-check-certificate", path => ['/opt/csw/bin/','/usr/bin/'] } + exec{'download-file_concat': + command => "wget -P /root/ https://forgeapi.puppetlabs.com/v3/files/electrical-file_concat-1.0.0.tar.gz --no-check-certificate", + path => ['/opt/csw/bin/','/usr/bin/'] + } EOS - apply_manifest_on(host, get_stdlib) + apply_manifest_on(host, get_deps) # have to use force otherwise it checks ssl cert even though it is a local file on host, puppet('module install /root/puppetlabs-stdlib-4.5.1.tar.gz --force --ignore-dependencies'), {:acceptable_exit_codes => [0, 1]} + on host, puppet('module install /root/electrical-file_concat-1.0.0.tar.gz --force --ignore-dependencies'), {:acceptable_exit_codes => [0, 1]} elsif host['platform'] =~ /windows/i on host, shell('curl -k -o c:/puppetlabs-stdlib-4.5.1.tar.gz https://forgeapi.puppetlabs.com/v3/files/puppetlabs-stdlib-4.5.1.tar.gz') on host, puppet('module install c:/puppetlabs-stdlib-4.5.1.tar.gz --force --ignore-dependencies'), {:acceptable_exit_codes => [0, 1]} + on host, shell('curl -k -o c:/electrical-file_concat-1.0.0.tar.gz https://forgeapi.puppetlabs.com/v3/files/electrical-file_concat-1.0.0.tar.gz') + on host, puppet('module install c:/electrical-file_concat-1.0.0.tar.gz --force --ignore-dependencies'), {:acceptable_exit_codes => [0, 1]} else on host, puppet('module install puppetlabs-stdlib'), {:acceptable_exit_codes => [0, 1]} + on host, puppet('module install electrical-file_concat'), {:acceptable_exit_codes => [0, 1]} end end end diff --git a/spec/unit/classes/concat_setup_spec.rb b/spec/unit/classes/concat_setup_spec.rb deleted file mode 100644 index c6ff93e0..00000000 --- a/spec/unit/classes/concat_setup_spec.rb +++ /dev/null @@ -1,98 +0,0 @@ -require 'spec_helper' - -describe 'concat::setup', :type => :class do - - shared_examples 'setup' do |concatdir| - concatdir = '/foo' if concatdir.nil? - - let(:facts) do - { - :concat_basedir => concatdir, - :caller_module_name => 'Test', - :osfamily => 'Debian', - :id => 'root', - :is_pe => false, - } - end - - it do - should contain_file("#{concatdir}/bin/concatfragments.rb").with({ - :mode => '0755', - :owner => 'root', - :group => 0, - :source => 'puppet:///modules/concat/concatfragments.rb', - }) - end - - [concatdir, "#{concatdir}/bin"].each do |file| - it do - should contain_file(file).with({ - :ensure => 'directory', - :mode => '0755', - :owner => 'root', - :group => 0, - }) - end - end - end - - context 'facts' do - context 'concat_basedir =>' do - context '/foo' do - it_behaves_like 'setup', '/foo' - end - end - end # facts - - context 'deprecated as a public class' do - it 'should create a warning' do - skip('rspec-puppet support for testing warning()') - end - end - - context "on osfamily Solaris" do - concatdir = '/foo' - let(:facts) do - { - :concat_basedir => concatdir, - :caller_module_name => 'Test', - :osfamily => 'Solaris', - :id => 'root', - :is_pe => false, - } - end - - it do - should contain_file("#{concatdir}/bin/concatfragments.rb").with({ - :ensure => 'file', - :owner => 'root', - :group => 0, - :mode => '0755', - :source => 'puppet:///modules/concat/concatfragments.rb', - }) - end - end # on osfamily Solaris - - context "on osfamily windows" do - concatdir = '/foo' - let(:facts) do - { - :concat_basedir => concatdir, - :caller_module_name => 'Test', - :osfamily => 'windows', - :id => 'batman', - :is_pe => false, - } - end - - it do - should contain_file("#{concatdir}/bin/concatfragments.rb").with({ - :ensure => 'file', - :owner => nil, - :group => nil, - :mode => nil, - :source => 'puppet:///modules/concat/concatfragments.rb', - }) - end - end # on osfamily windows -end diff --git a/spec/unit/defines/concat_fragment_spec.rb b/spec/unit/defines/concat_fragment_spec.rb index 6cf3e429..98be58a1 100644 --- a/spec/unit/defines/concat_fragment_spec.rb +++ b/spec/unit/defines/concat_fragment_spec.rb @@ -9,50 +9,21 @@ :content => nil, :source => nil, :order => 10, - :ensure => 'present', }.merge(params) - safe_name = title.gsub(/[\/\n]/, '_') - safe_target_name = p[:target].gsub(/[\/\n]/, '_') - concatdir = '/var/lib/puppet/concat' - fragdir = "#{concatdir}/#{safe_target_name}" id = 'root' gid = 'root' - if p[:ensure] == 'absent' - safe_ensure = p[:ensure] - else - safe_ensure = 'file' - end let(:title) { title } - let(:facts) do - { - :concat_basedir => concatdir, - :id => id, - :gid => gid, - :osfamily => 'Debian', - :path => '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', - :is_pe => false, - } - end let(:params) { params } let(:pre_condition) do "concat{ '#{p[:target]}': }" end it do - should contain_class('concat::setup') should contain_concat(p[:target]) - should contain_file("#{fragdir}/fragments/#{p[:order]}_#{safe_name}").with({ - :ensure => safe_ensure, - :owner => id, - :group => gid, - :mode => '0640', - :source => p[:source], - :content => p[:content], - :alias => "concat_fragment_#{title}", - :backup => 'puppet', - }) + should contain_file_concat(p[:target]) + should contain_file_fragment(title) end end @@ -60,6 +31,7 @@ ['0', '1', 'a', 'z'].each do |title| it_behaves_like 'fragment', title, { :target => '/etc/motd', + :content => "content for #{title}" } end end # title @@ -69,42 +41,21 @@ context target do it_behaves_like 'fragment', target, { :target => '/etc/motd', + :content => "content for #{target}" } end end context 'false' do let(:title) { 'motd_header' } - let(:facts) {{ :concat_basedir => '/tmp', :is_pe => false }} let(:params) {{ :target => false }} it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /is not a string/) + expect { catalogue }.to raise_error(Puppet::Error, /is not a string/) end end end # target => - context 'ensure =>' do - ['present', 'absent'].each do |ens| - context ens do - it_behaves_like 'fragment', 'motd_header', { - :ensure => ens, - :target => '/etc/motd', - } - end - end - - context 'any value other than \'present\' or \'absent\'' do - let(:title) { 'motd_header' } - let(:facts) {{ :concat_basedir => '/tmp', :is_pe => false }} - let(:params) {{ :ensure => 'invalid', :target => '/etc/motd' }} - - it 'should create a warning' do - skip('rspec-puppet support for testing warning()') - end - end - end # ensure => - context 'content =>' do ['', 'ashp is our hero'].each do |content| context content do @@ -117,11 +68,10 @@ context 'false' do let(:title) { 'motd_header' } - let(:facts) {{ :concat_basedir => '/tmp', :is_pe => false }} let(:params) {{ :content => false, :target => '/etc/motd' }} it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /is not a string/) + expect { catalogue }.to raise_error(Puppet::Error, /is not a string/) end end end # content => @@ -138,193 +88,28 @@ context 'false' do let(:title) { 'motd_header' } - let(:facts) {{ :concat_basedir => '/tmp', :is_pe => false }} let(:params) {{ :source => false, :target => '/etc/motd' }} it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /is not a string or an Array/) + expect { catalogue }.to raise_error(Puppet::Error, /is not a string or an Array/) end end end # source => - context 'order =>' do - ['', '42', 'a', 'z'].each do |order| - context '\'\'' do - it_behaves_like 'fragment', 'motd_header', { - :order => order, - :target => '/etc/motd', - } - end - end - - context 'false' do - let(:title) { 'motd_header' } - let(:facts) {{ :concat_basedir => '/tmp', :is_pe => false }} - let(:params) {{ :order => false, :target => '/etc/motd' }} - - it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /is not a string or integer/) - end - end - - context '123:456' do - let(:title) { 'motd_header' } - let(:facts) {{ :concat_basedir => '/tmp', :is_pe => false }} - let(:params) {{ :order => '123:456', :target => '/etc/motd' }} - - it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /cannot contain/) - end - end - context '123/456' do - let(:title) { 'motd_header' } - let(:facts) {{ :concat_basedir => '/tmp', :is_pe => false }} - let(:params) {{ :order => '123/456', :target => '/etc/motd' }} - - it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /cannot contain/) - end - end - context '123\n456' do - let(:title) { 'motd_header' } - let(:facts) {{ :concat_basedir => '/tmp', :is_pe => false }} - let(:params) {{ :order => "123\n456", :target => '/etc/motd' }} - - it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /cannot contain/) - end - end - end # order => - context 'more than one content source' do - error_msg = 'You cannot specify more than one of $content, $source, $ensure => /target' - - context 'ensure => target and source' do - let(:title) { 'motd_header' } - let(:facts) do - { - :concat_basedir => '/tmp', - :osfamily => 'Debian', - :id => 'root', - :is_pe => false, - :gid => 'root', - } - end - let(:params) do - { - :target => '/etc/motd', - :ensure => '/foo', - :source => '/bar', - } - end - - it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /#{Regexp.escape(error_msg)}/m) - end - end - - context 'ensure => target and content' do - let(:title) { 'motd_header' } - let(:facts) do - { - :concat_basedir => '/tmp', - :osfamily => 'Debian', - :id => 'root', - :is_pe => false, - :gid => 'root', - } - end - let(:params) do - { - :target => '/etc/motd', - :ensure => '/foo', - :content => 'bar', - } - end - - it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /#{Regexp.escape(error_msg)}/m) - end - end - context 'source and content' do let(:title) { 'motd_header' } - let(:facts) do - { - :concat_basedir => '/tmp', - :osfamily => 'Debian', - :id => 'root', - :is_pe => false, - :gid => 'root', - } - end let(:params) do { - :target => '/etc/motd', + :target => '/etc/motd', :source => '/foo', :content => 'bar', } end it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /#{Regexp.escape(error_msg)}/m) + expect { catalogue }.to raise_error(Puppet::Error, /Can't use 'source' and 'content' at the same time/) end end - end # more than one content source - - describe 'deprecated parameter' do - context 'mode =>' do - context '1755' do - it_behaves_like 'fragment', 'motd_header', { - :mode => '1755', - :target => '/etc/motd', - } - - it 'should create a warning' do - skip('rspec-puppet support for testing warning()') - end - end - end # mode => - - context 'owner =>' do - context 'apenny' do - it_behaves_like 'fragment', 'motd_header', { - :owner => 'apenny', - :target => '/etc/motd', - } - - it 'should create a warning' do - skip('rspec-puppet support for testing warning()') - end - end - end # owner => - - context 'group =>' do - context 'apenny' do - it_behaves_like 'fragment', 'motd_header', { - :group => 'apenny', - :target => '/etc/motd', - } - - it 'should create a warning' do - skip('rspec-puppet support for testing warning()') - end - end - end # group => - - context 'backup =>' do - context 'foo' do - it_behaves_like 'fragment', 'motd_header', { - :backup => 'foo', - :target => '/etc/motd', - } - - it 'should create a warning' do - skip('rspec-puppet support for testing warning()') - end - end - end # backup => - end # deprecated params - end diff --git a/spec/unit/defines/concat_spec.rb b/spec/unit/defines/concat_spec.rb index 115a0f55..176f24a3 100644 --- a/spec/unit/defines/concat_spec.rb +++ b/spec/unit/defines/concat_spec.rb @@ -13,20 +13,15 @@ :owner => nil, :group => nil, :mode => '0644', - :warn => false, - :force => false, + :warn_header => false, + #:force => false, :backup => 'puppet', :replace => true, - :order => 'alpha', - :ensure_newline => false, - :validate_cmd => nil, }.merge(params) safe_name = title.gsub('/', '_') - concatdir = '/var/lib/puppet/concat' - fragdir = "#{concatdir}/#{safe_name}" concat_name = 'fragments.concat.out' - default_warn_message = '# This file is managed by Puppet. DO NOT EDIT.' + default_warn_message = "# This file is managed by Puppet. DO NOT EDIT.\n" file_defaults = { :backup => p[:backup], @@ -36,7 +31,6 @@ let(:params) { params } let(:facts) do { - :concat_basedir => concatdir, :id => id, :osfamily => 'Debian', :path => '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', @@ -47,117 +41,23 @@ if p[:ensure] == 'present' it do - should contain_file(fragdir).with(file_defaults.merge({ - :ensure => 'directory', - :mode => '0750', - })) - end - - it do - should contain_file("#{fragdir}/fragments").with(file_defaults.merge({ - :ensure => 'directory', - :mode => '0750', - :force => true, - :ignore => ['.svn', '.git', '.gitignore'], - :purge => true, - :recurse => true, - })) - end - - [ - "#{fragdir}/fragments.concat", - "#{fragdir}/#{concat_name}", - ].each do |file| - it do - should contain_file(file).with(file_defaults.merge({ - :ensure => 'present', - :mode => '0640', - })) - end - end - - it do - should contain_file(title).with(file_defaults.merge({ + should contain_concat(title).with(file_defaults.merge({ :ensure => 'present', :owner => p[:owner], :group => p[:group], :mode => p[:mode], - :replace => p[:replace], :path => p[:path], - :alias => "concat_#{title}", - :source => "#{fragdir}/#{concat_name}", - :validate_cmd => p[:validate_cmd], :backup => p[:backup], + :replace => p[:replace], })) end - - cmd = "#{concatdir}/bin/concatfragments.rb " + - "-o \"#{concatdir}/#{safe_name}/fragments.concat.out\" " + - "-d \"#{concatdir}/#{safe_name}\"" - - # flag order: fragdir, warnflag, forceflag, orderflag, newlineflag - if p.has_key?(:warn) - case p[:warn] - when TrueClass - message = default_warn_message - when 'true', 'yes', 'on' - # should generate a stringified boolean warning - message = default_warn_message - when FalseClass - message = nil - when 'false', 'no', 'off' - # should generate a stringified boolean warning - message = nil - else - message = p[:warn] - end - - unless message.nil? - cmd += " -w \'#{message}\'" - end - end - - cmd += " -f" if p[:force] - cmd += " -n" if p[:order] == 'numeric' - cmd += " -l" if p[:ensure_newline] == true - - it do - should contain_exec("concat_#{title}").with({ - :alias => "concat_#{fragdir}", - :command => cmd, - :unless => "#{cmd} -t", - }) - end else - [ - fragdir, - "#{fragdir}/fragments", - "#{fragdir}/fragments.concat", - "#{fragdir}/#{concat_name}", - ].each do |file| - it do - should contain_file(file).with(file_defaults.merge({ - :ensure => 'absent', - :force => true, - })) - end - end - it do - should contain_file(title).with(file_defaults.merge({ + should contain_concat(title).with(file_defaults.merge({ :ensure => 'absent', :backup => p[:backup], })) end - - it do - should contain_exec("concat_#{title}").with({ - :alias => "concat_#{fragdir}", - :command => 'true', - :unless => 'true', - :path => '/bin:/usr/bin', - }) - end end end @@ -175,14 +75,14 @@ context title do let(:title) { title } it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /is not an absolute path/) + expect { catalogue }.to raise_error(Puppet::Error, /is not an absolute path/) end end end end context 'with path param' do - ['./foo', 'foo', 'foo/bar'].each do |title| + ['/foo', 'foo', 'foo/bar'].each do |title| context title do it_behaves_like 'concat', title, { :path => '/etc/foo.bar' } end @@ -205,7 +105,7 @@ let(:title) { '/etc/foo.bar' } let(:params) {{ :ensure => 'invalid' }} it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /#{Regexp.escape('does not match "^present$|^absent$"')}/) + expect { catalogue }.to raise_error(Puppet::Error, /#{Regexp.escape('does not match "^present$|^absent$"')}/) end end end # ensure => @@ -220,7 +120,7 @@ let(:title) { '/etc/foo.bar' } let(:params) {{ :path => path }} it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /is not an absolute path/) + expect { catalogue }.to raise_error(Puppet::Error, /is not an absolute path/) end end end @@ -235,7 +135,7 @@ let(:title) { '/etc/foo.bar' } let(:params) {{ :owner => false }} it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /is not a string/) + expect { catalogue }.to raise_error(Puppet::Error, /is not a string/) end end end # owner => @@ -249,7 +149,7 @@ let(:title) { '/etc/foo.bar' } let(:params) {{ :group => false }} it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /is not a string/) + expect { catalogue }.to raise_error(Puppet::Error, /is not a string/) end end end # group => @@ -263,22 +163,22 @@ let(:title) { '/etc/foo.bar' } let(:params) {{ :mode => false }} it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /is not a string/) + expect { catalogue }.to raise_error(Puppet::Error, /is not a string/) end end end # mode => - context 'warn =>' do + context 'warn_header =>' do [true, false, '# foo'].each do |warn| context warn do - it_behaves_like 'concat', '/etc/foo.bar', { :warn => warn } + it_behaves_like 'concat', '/etc/foo.bar', { :warn_header => warn } end end context '(stringified boolean)' do ['true', 'yes', 'on', 'false', 'no', 'off'].each do |warn| context warn do - it_behaves_like 'concat', '/etc/foo.bar', { :warn => warn } + it_behaves_like 'concat', '/etc/foo.bar', { :warn_header => warn } it 'should create a warning' do skip('rspec-puppet support for testing warning()') @@ -289,28 +189,28 @@ context '123' do let(:title) { '/etc/foo.bar' } - let(:params) {{ :warn => 123 }} + let(:params) {{ :warn_header => 123 }} it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /is not a string or boolean/) + expect { catalogue }.to raise_error(Puppet::Error, /is not a string or boolean/) end end end # warn => - context 'force =>' do - [true, false].each do |force| - context force do - it_behaves_like 'concat', '/etc/foo.bar', { :force => force } - end - end - - context '123' do - let(:title) { '/etc/foo.bar' } - let(:params) {{ :force => 123 }} - it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /is not a boolean/) - end - end - end # force => + #context 'force =>' do + # [true, false].each do |force| + # context force do + # it_behaves_like 'concat', '/etc/foo.bar', { :force => force } + # end + # end + + # context '123' do + # let(:title) { '/etc/foo.bar' } + # let(:params) {{ :force => 123 }} + # it 'should fail' do + # expect { catalogue }.to raise_error(Puppet::Error, /is not a boolean/) + # end + # end + #end # force => context 'backup =>' do context 'reverse' do @@ -329,7 +229,7 @@ let(:title) { '/etc/foo.bar' } let(:params) {{ :backup => [] }} it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /backup must be string or bool/) + expect { catalogue }.to raise_error(Puppet::Error, /backup must be string or bool/) end end end # backup => @@ -345,71 +245,24 @@ let(:title) { '/etc/foo.bar' } let(:params) {{ :replace => 123 }} it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /is not a boolean/) + expect { catalogue }.to raise_error(Puppet::Error, /is not a boolean/) end end end # replace => - context 'order =>' do - ['alpha', 'numeric'].each do |order| - context order do - it_behaves_like 'concat', '/etc/foo.bar', { :order => order } - end - end - - context 'invalid' do - let(:title) { '/etc/foo.bar' } - let(:params) {{ :order => 'invalid' }} - it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /#{Regexp.escape('does not match "^alpha$|^numeric$"')}/) - end - end - end # order => - - context 'ensure_newline =>' do - [true, false].each do |ensure_newline| - context 'true' do - it_behaves_like 'concat', '/etc/foo.bar', { :ensure_newline => ensure_newline} - end - end - - context '123' do - let(:title) { '/etc/foo.bar' } - let(:params) {{ :ensure_newline => 123 }} - it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /is not a boolean/) - end - end - end # ensure_newline => - - context 'validate_cmd =>' do - context '/usr/bin/test -e %' do - it_behaves_like 'concat', '/etc/foo.bar', { :validate_cmd => '/usr/bin/test -e %' } - end - - [ 1234, true ].each do |cmd| - context cmd do - let(:title) { '/etc/foo.bar' } - let(:params) {{ :validate_cmd => cmd }} - it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /\$validate_cmd must be a string/) - end - end - end - end # validate_cmd => - - describe 'deprecated parameter' do - context 'gnu =>' do - context 'foo' do - it_behaves_like 'concat', '/etc/foo.bar', { :gnu => 'foo'} - - it 'should create a warning' do - skip('rspec-puppet support for testing warning()') - end - end - end - end - + #context 'order =>' do + # ['alpha', 'numeric'].each do |order| + # context order do + # it_behaves_like 'concat', '/etc/foo.bar', { :order => order } + # end + # end + + # context 'invalid' do + # let(:title) { '/etc/foo.bar' } + # let(:params) {{ :order => 'invalid' }} + # it 'should fail' do + # expect { catalogue }.to raise_error(Puppet::Error, /#{Regexp.escape('does not match "^alpha$|^numeric$"')}/) + # end + # end + #end # order => end - -# vim:sw=2:ts=2:expandtab:textwidth=79 diff --git a/spec/unit/facts/concat_basedir_spec.rb b/spec/unit/facts/concat_basedir_spec.rb deleted file mode 100644 index 41bc90f1..00000000 --- a/spec/unit/facts/concat_basedir_spec.rb +++ /dev/null @@ -1,18 +0,0 @@ -require 'spec_helper' - -describe 'concat_basedir', :type => :fact do - before(:each) { Facter.clear } - - context 'Puppet[:vardir] ==' do - it '/var/lib/puppet' do - Puppet.stubs(:[]).with(:vardir).returns('/var/lib/puppet') - Facter.fact(:concat_basedir).value.should == '/var/lib/puppet/concat' - end - - it '/home/apenny/.puppet/var' do - Puppet.stubs(:[]).with(:vardir).returns('/home/apenny/.puppet/var') - Facter.fact(:concat_basedir).value.should == '/home/apenny/.puppet/var/concat' - end - end - -end From 8198296c7b15cfd540709ef7992bf8ea711a2ebb Mon Sep 17 00:00:00 2001 From: Bryan Jen Date: Mon, 13 Apr 2015 15:06:01 -0700 Subject: [PATCH 03/76] Updates CHANGELOG and metadata for 1.2.1 release --- CHANGELOG.md | 14 ++++++++++++++ metadata.json | 2 +- 2 files changed, 15 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 2043b604..7ba7ebeb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,17 @@ +##2015-04-14 - Supported Release 1.2.1 +###Summary + +This release includes bugfixes, test improvements, and a rewritten README. + +####Bugfixes + +- Verifies existence of $is_pe before using it. +- Adds validation for $order param to not allow restricted characters. +- Sets the group id on Fragments for versions of Facter that contain the $gid fact. +- Sets the script group to 0 if the script is owned by root. +- Explicitly sets script and concat directories to the same owner and group. +- Re-adds support for fragment backup, so that static compiler can work with filebucket (MODULES-1700). + ##2015-02-17 - Supported Release 1.2.0 ###Summary diff --git a/metadata.json b/metadata.json index 9d7a46dd..ce932111 100644 --- a/metadata.json +++ b/metadata.json @@ -1,6 +1,6 @@ { "name": "puppetlabs-concat", - "version": "1.2.0", + "version": "1.2.1", "author": "Puppet Labs", "summary": "Construct files from multiple fragments.", "license": "Apache-2.0", From 71c815bb3a2a18214a0fc40159160d6580561dc1 Mon Sep 17 00:00:00 2001 From: Bryan Jen Date: Wed, 15 Apr 2015 09:35:27 -0700 Subject: [PATCH 04/76] fix linting errors --- metadata.json | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/metadata.json b/metadata.json index 49c24c60..bdb1ea78 100644 --- a/metadata.json +++ b/metadata.json @@ -112,6 +112,6 @@ { "name": "electrical/file_concat", "version_requirement": ">= 1.0.0" - }, - ], + } + ] } From 4e61c338f4c189f8dfeccaaa6a9fa8d6f685ed8a Mon Sep 17 00:00:00 2001 From: Bryan Jen Date: Wed, 15 Apr 2015 09:58:05 -0700 Subject: [PATCH 05/76] more geppetto lint fixes --- manifests/init.pp | 2 +- tests/init.pp | 1 - 2 files changed, 1 insertion(+), 2 deletions(-) diff --git a/manifests/init.pp b/manifests/init.pp index 305b3d97..9c3c21d8 100644 --- a/manifests/init.pp +++ b/manifests/init.pp @@ -90,7 +90,7 @@ file_concat { $name: tag => $safe_name, path => $path, - owner => $user, + owner => $owner, group => $group, mode => $mode, replace => $replace, diff --git a/tests/init.pp b/tests/init.pp index fd214271..c7f19346 100644 --- a/tests/init.pp +++ b/tests/init.pp @@ -1,6 +1,5 @@ concat { '/tmp/concat': ensure => present, - force => true, owner => 'root', group => 'root', mode => '0644', From c96ae6a143963b2abdb7a2f6e59660212ecb2796 Mon Sep 17 00:00:00 2001 From: Hunter Haugen Date: Wed, 15 Apr 2015 09:59:53 -0700 Subject: [PATCH 06/76] The jenkins geppetto plugin is still complaining --- metadata.json | 10 ++-------- 1 file changed, 2 insertions(+), 8 deletions(-) diff --git a/metadata.json b/metadata.json index bdb1ea78..e4688ec7 100644 --- a/metadata.json +++ b/metadata.json @@ -105,13 +105,7 @@ } ], "dependencies": [ - { - "name": "puppetlabs/stdlib", - "version_requirement": ">= 3.2.0 < 5.0.0" - }, - { - "name": "electrical/file_concat", - "version_requirement": ">= 1.0.0" - } + {"name":"puppetlabs/stdlib","version_requirement":">= 3.2.0 < 5.0.0"}, + {"name":"electrical/file_concat","version_requirement":">= 1.0.0"} ] } From cd03959116f7b27ab3ec7ccace98bd10be435397 Mon Sep 17 00:00:00 2001 From: Bryan Jen Date: Thu, 16 Apr 2015 15:49:03 -0700 Subject: [PATCH 07/76] re-add removed params for backwards compatibility --- manifests/fragment.pp | 6 +++ manifests/init.pp | 15 ++++-- spec/acceptance/deprecation_warnings_spec.rb | 44 +++++++++++++++ spec/acceptance/warn_header_spec.rb | 8 +-- spec/unit/defines/concat_spec.rb | 57 +++++++------------- 5 files changed, 84 insertions(+), 46 deletions(-) create mode 100644 spec/acceptance/deprecation_warnings_spec.rb diff --git a/manifests/fragment.pp b/manifests/fragment.pp index 5a3e3104..fabc3596 100644 --- a/manifests/fragment.pp +++ b/manifests/fragment.pp @@ -16,11 +16,17 @@ # define concat::fragment( $target, + $ensure = undef, $content = undef, $source = undef, $order = '10', ) { validate_string($target) + + if $ensure != undef { + warning('The $ensure parameter to concat::fragment is deprecated and has no effect.') + } + validate_string($content) if !(is_string($source) or is_array($source)) { fail('$source is not a string or an Array.') diff --git a/manifests/init.pp b/manifests/init.pp index 9c3c21d8..981a5c27 100644 --- a/manifests/init.pp +++ b/manifests/init.pp @@ -46,7 +46,8 @@ $owner = undef, $group = undef, $mode = '0644', - $warn_header = false, + $warn = false, + $force = undef, $backup = 'puppet', $replace = true, $order = 'alpha', @@ -57,8 +58,8 @@ validate_string($owner) validate_string($group) validate_string($mode) - if ! (is_string($warn_header) or $warn_header == true or $warn_header == false) { - fail('$warn_header is not a string or boolean') + if ! (is_string($warn) or $warn == true or $warn == false) { + fail('$warn is not a string or boolean') } if ! is_bool($backup) and ! is_string($backup) { fail('$backup must be string or bool!') @@ -69,10 +70,14 @@ fail('$validate_cmd must be a string') } + if $force != undef { + warning('The $force parameter to concat is deprecated and has no effect.') + } + $safe_name = regsubst($name, '[/:\n\s]', '_', 'G') $default_warn_message = "# This file is managed by Puppet. DO NOT EDIT.\n" - case $warn_header { + case $warn { true: { $warn_message = $default_warn_message $append_header = true @@ -81,7 +86,7 @@ $warn_message = '' } default: { - $warn_message = $warn_header + $warn_message = $warn $append_header = true } } diff --git a/spec/acceptance/deprecation_warnings_spec.rb b/spec/acceptance/deprecation_warnings_spec.rb new file mode 100644 index 00000000..8a689a76 --- /dev/null +++ b/spec/acceptance/deprecation_warnings_spec.rb @@ -0,0 +1,44 @@ +require 'spec_helper_acceptance' + +describe 'deprecation warnings' do + basedir = default.tmpdir('concat') + + shared_examples 'has_warning' do |pp, w| + it 'applies the manifest twice with a stderr regex' do + expect(apply_manifest(pp, :catch_failures => true).stderr).to match(/#{Regexp.escape(w)}/m) + expect(apply_manifest(pp, :catch_changes => true).stderr).to match(/#{Regexp.escape(w)}/m) + end + end + + context 'concat force parameter' do + pp = <<-EOS + concat { '#{basedir}/file': + force => false, + } + concat::fragment { 'foo': + target => '#{basedir}/file', + content => 'bar', + } + EOS + w = 'The $force parameter to concat is deprecated and has no effect.' + + it_behaves_like 'has_warning', pp, w + end + + context 'concat::fragment ensure parameter' do + context 'target file exists' do + pp = <<-EOS + concat { '#{basedir}/file': + } + concat::fragment { 'foo': + target => '#{basedir}/file', + ensure => false, + content => 'bar', + } + EOS + w = 'The $ensure parameter to concat::fragment is deprecated and has no effect.' + + it_behaves_like 'has_warning', pp, w + end + end +end diff --git a/spec/acceptance/warn_header_spec.rb b/spec/acceptance/warn_header_spec.rb index 52615e1d..b73414e3 100644 --- a/spec/acceptance/warn_header_spec.rb +++ b/spec/acceptance/warn_header_spec.rb @@ -1,11 +1,11 @@ require 'spec_helper_acceptance' -describe 'concat warn_header =>' do +describe 'concat warn =>' do basedir = default.tmpdir('concat') context 'true should enable default warning message' do pp = <<-EOS concat { '#{basedir}/file': - warn_header => true, + warn => true, } concat::fragment { '1': @@ -38,7 +38,7 @@ context 'false should not enable default warning message' do pp = <<-EOS concat { '#{basedir}/file': - warn_header => false, + warn => false, } concat::fragment { '1': @@ -71,7 +71,7 @@ context '# foo should overide default warning message' do pp = <<-EOS concat { '#{basedir}/file': - warn_header => "# foo\n", + warn => "# foo\n", } concat::fragment { '1': diff --git a/spec/unit/defines/concat_spec.rb b/spec/unit/defines/concat_spec.rb index 176f24a3..35767aea 100644 --- a/spec/unit/defines/concat_spec.rb +++ b/spec/unit/defines/concat_spec.rb @@ -13,8 +13,7 @@ :owner => nil, :group => nil, :mode => '0644', - :warn_header => false, - #:force => false, + :warn => false, :backup => 'puppet', :replace => true, }.merge(params) @@ -168,17 +167,17 @@ end end # mode => - context 'warn_header =>' do + context 'warn =>' do [true, false, '# foo'].each do |warn| context warn do - it_behaves_like 'concat', '/etc/foo.bar', { :warn_header => warn } + it_behaves_like 'concat', '/etc/foo.bar', { :warn => warn } end end context '(stringified boolean)' do ['true', 'yes', 'on', 'false', 'no', 'off'].each do |warn| context warn do - it_behaves_like 'concat', '/etc/foo.bar', { :warn_header => warn } + it_behaves_like 'concat', '/etc/foo.bar', { :warn => warn } it 'should create a warning' do skip('rspec-puppet support for testing warning()') @@ -189,29 +188,13 @@ context '123' do let(:title) { '/etc/foo.bar' } - let(:params) {{ :warn_header => 123 }} + let(:params) {{ :warn => 123 }} it 'should fail' do expect { catalogue }.to raise_error(Puppet::Error, /is not a string or boolean/) end end end # warn => - #context 'force =>' do - # [true, false].each do |force| - # context force do - # it_behaves_like 'concat', '/etc/foo.bar', { :force => force } - # end - # end - - # context '123' do - # let(:title) { '/etc/foo.bar' } - # let(:params) {{ :force => 123 }} - # it 'should fail' do - # expect { catalogue }.to raise_error(Puppet::Error, /is not a boolean/) - # end - # end - #end # force => - context 'backup =>' do context 'reverse' do it_behaves_like 'concat', '/etc/foo.bar', { :backup => 'reverse' } @@ -250,19 +233,19 @@ end end # replace => - #context 'order =>' do - # ['alpha', 'numeric'].each do |order| - # context order do - # it_behaves_like 'concat', '/etc/foo.bar', { :order => order } - # end - # end - - # context 'invalid' do - # let(:title) { '/etc/foo.bar' } - # let(:params) {{ :order => 'invalid' }} - # it 'should fail' do - # expect { catalogue }.to raise_error(Puppet::Error, /#{Regexp.escape('does not match "^alpha$|^numeric$"')}/) - # end - # end - #end # order => + context 'order =>' do + ['alpha', 'numeric'].each do |order| + context order do + it_behaves_like 'concat', '/etc/foo.bar', { :order => order } + end + end + + context 'invalid' do + let(:title) { '/etc/foo.bar' } + let(:params) {{ :order => 'invalid' }} + it 'should fail' do + expect { catalogue }.to raise_error(Puppet::Error, /#{Regexp.escape('does not match "^alpha$|^numeric$"')}/) + end + end + end # order => end From 6cfa62bce433f44c4ea6279adcb886e038e44df1 Mon Sep 17 00:00:00 2001 From: Bryan Jen Date: Thu, 16 Apr 2015 15:06:10 -0700 Subject: [PATCH 08/76] fixes windows. --- .fixtures.yml | 2 +- manifests/fragment.pp | 2 +- manifests/init.pp | 2 +- metadata.json | 2 +- spec/spec_helper_acceptance.rb | 6 +++--- 5 files changed, 7 insertions(+), 7 deletions(-) diff --git a/.fixtures.yml b/.fixtures.yml index fb0f76a6..3d0f23a9 100644 --- a/.fixtures.yml +++ b/.fixtures.yml @@ -5,6 +5,6 @@ fixtures: ref: '4.5.1' 'file_concat': repo: 'git://github.com/electrical/puppet-lib-file_concat.git' - branch: '1.0.0' + ref: '1.0.1' symlinks: 'concat': '#{source_dir}' diff --git a/manifests/fragment.pp b/manifests/fragment.pp index fabc3596..a3d42ef0 100644 --- a/manifests/fragment.pp +++ b/manifests/fragment.pp @@ -44,7 +44,7 @@ fail("Can't use 'source' and 'content' at the same time") } - $safe_target_name = regsubst($target, '[/:\n\s]', '_', 'GM') + $safe_target_name = regsubst($target, '[/:\n\s\\]', '_', 'GM') file_fragment { $name: tag => $safe_target_name, diff --git a/manifests/init.pp b/manifests/init.pp index 981a5c27..69dc46ef 100644 --- a/manifests/init.pp +++ b/manifests/init.pp @@ -74,7 +74,7 @@ warning('The $force parameter to concat is deprecated and has no effect.') } - $safe_name = regsubst($name, '[/:\n\s]', '_', 'G') + $safe_name = regsubst($name, '[/:\n\s\\]', '_', 'G') $default_warn_message = "# This file is managed by Puppet. DO NOT EDIT.\n" case $warn { diff --git a/metadata.json b/metadata.json index e4688ec7..137499e9 100644 --- a/metadata.json +++ b/metadata.json @@ -106,6 +106,6 @@ ], "dependencies": [ {"name":"puppetlabs/stdlib","version_requirement":">= 3.2.0 < 5.0.0"}, - {"name":"electrical/file_concat","version_requirement":">= 1.0.0"} + {"name":"electrical/file_concat","version_requirement":">= 1.0.1"} ] } diff --git a/spec/spec_helper_acceptance.rb b/spec/spec_helper_acceptance.rb index 1a3ffc0a..80aa2f75 100644 --- a/spec/spec_helper_acceptance.rb +++ b/spec/spec_helper_acceptance.rb @@ -23,18 +23,18 @@ path => ['/opt/csw/bin/','/usr/bin/'] } exec{'download-file_concat': - command => "wget -P /root/ https://forgeapi.puppetlabs.com/v3/files/electrical-file_concat-1.0.0.tar.gz --no-check-certificate", + command => "wget -P /root/ https://forgeapi.puppetlabs.com/v3/files/electrical-file_concat-1.0.1.tar.gz --no-check-certificate", path => ['/opt/csw/bin/','/usr/bin/'] } EOS apply_manifest_on(host, get_deps) # have to use force otherwise it checks ssl cert even though it is a local file on host, puppet('module install /root/puppetlabs-stdlib-4.5.1.tar.gz --force --ignore-dependencies'), {:acceptable_exit_codes => [0, 1]} - on host, puppet('module install /root/electrical-file_concat-1.0.0.tar.gz --force --ignore-dependencies'), {:acceptable_exit_codes => [0, 1]} + on host, puppet('module install /root/electrical-file_concat-1.0.1.tar.gz --force --ignore-dependencies'), {:acceptable_exit_codes => [0, 1]} elsif host['platform'] =~ /windows/i on host, shell('curl -k -o c:/puppetlabs-stdlib-4.5.1.tar.gz https://forgeapi.puppetlabs.com/v3/files/puppetlabs-stdlib-4.5.1.tar.gz') on host, puppet('module install c:/puppetlabs-stdlib-4.5.1.tar.gz --force --ignore-dependencies'), {:acceptable_exit_codes => [0, 1]} - on host, shell('curl -k -o c:/electrical-file_concat-1.0.0.tar.gz https://forgeapi.puppetlabs.com/v3/files/electrical-file_concat-1.0.0.tar.gz') + on host, shell('curl -k -o c:/electrical-file_concat-1.0.1.tar.gz https://forgeapi.puppetlabs.com/v3/files/electrical-file_concat-1.0.1.tar.gz') on host, puppet('module install c:/electrical-file_concat-1.0.0.tar.gz --force --ignore-dependencies'), {:acceptable_exit_codes => [0, 1]} else on host, puppet('module install puppetlabs-stdlib'), {:acceptable_exit_codes => [0, 1]} From 38094096362302e41bc46ddfb0bbb7b69eeba3a2 Mon Sep 17 00:00:00 2001 From: Bryan Jen Date: Fri, 17 Apr 2015 11:08:04 -0700 Subject: [PATCH 09/76] fixes acceptance tests on windows --- spec/spec_helper_acceptance.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/spec/spec_helper_acceptance.rb b/spec/spec_helper_acceptance.rb index 80aa2f75..56c6ea76 100644 --- a/spec/spec_helper_acceptance.rb +++ b/spec/spec_helper_acceptance.rb @@ -35,7 +35,7 @@ on host, shell('curl -k -o c:/puppetlabs-stdlib-4.5.1.tar.gz https://forgeapi.puppetlabs.com/v3/files/puppetlabs-stdlib-4.5.1.tar.gz') on host, puppet('module install c:/puppetlabs-stdlib-4.5.1.tar.gz --force --ignore-dependencies'), {:acceptable_exit_codes => [0, 1]} on host, shell('curl -k -o c:/electrical-file_concat-1.0.1.tar.gz https://forgeapi.puppetlabs.com/v3/files/electrical-file_concat-1.0.1.tar.gz') - on host, puppet('module install c:/electrical-file_concat-1.0.0.tar.gz --force --ignore-dependencies'), {:acceptable_exit_codes => [0, 1]} + on host, puppet('module install c:/electrical-file_concat-1.0.1.tar.gz --force --ignore-dependencies'), {:acceptable_exit_codes => [0, 1]} else on host, puppet('module install puppetlabs-stdlib'), {:acceptable_exit_codes => [0, 1]} on host, puppet('module install electrical-file_concat'), {:acceptable_exit_codes => [0, 1]} From e562bcdaf03309d3c841ba35fffe5f4d7d36947b Mon Sep 17 00:00:00 2001 From: Felix Kellner Date: Mon, 20 Apr 2015 10:54:06 +0200 Subject: [PATCH 10/76] Revert and fix regex --- manifests/fragment.pp | 2 +- manifests/init.pp | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/manifests/fragment.pp b/manifests/fragment.pp index a3d42ef0..fabc3596 100644 --- a/manifests/fragment.pp +++ b/manifests/fragment.pp @@ -44,7 +44,7 @@ fail("Can't use 'source' and 'content' at the same time") } - $safe_target_name = regsubst($target, '[/:\n\s\\]', '_', 'GM') + $safe_target_name = regsubst($target, '[/:\n\s]', '_', 'GM') file_fragment { $name: tag => $safe_target_name, diff --git a/manifests/init.pp b/manifests/init.pp index 69dc46ef..981a5c27 100644 --- a/manifests/init.pp +++ b/manifests/init.pp @@ -74,7 +74,7 @@ warning('The $force parameter to concat is deprecated and has no effect.') } - $safe_name = regsubst($name, '[/:\n\s\\]', '_', 'G') + $safe_name = regsubst($name, '[/:\n\s]', '_', 'G') $default_warn_message = "# This file is managed by Puppet. DO NOT EDIT.\n" case $warn { From 2d64759c5755c562747c7936b58552b215f893a5 Mon Sep 17 00:00:00 2001 From: Pete Soloway Date: Fri, 17 Apr 2015 13:02:12 -0700 Subject: [PATCH 11/76] Update README per cd03959116f7b27ab3ec7ccace98bd10be435397 The concat `force` and fragment `ensure` parameters have been added back and deprecated. The `warn_header` parameter has changed back to `warn`. Updating README to reflect this. --- README.md | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index aba73b42..29137548 100644 --- a/README.md +++ b/README.md @@ -27,7 +27,7 @@ The concat module lets you gather `concat::fragment` resources from your other m ###What concat affects -The concat module requires the [file_concat module](https://forge.puppetlabs.com/ispavailability/file_concat). If you don't have file_concat installed, concat installs it for you. +The concat module requires the [file_concat module](https://forge.puppetlabs.com/electrical/file_concat). If you don't have file_concat installed, concat installs it for you. ###Beginning with concat @@ -140,6 +140,10 @@ Specifies whether (and how) to back up the destination file before overwriting i Specifies whether the destination file should exist. Setting to 'absent' tells Puppet to delete the destination file if it exists, and negates the effect of any other parameters. Valid options: 'present' and 'absent'. Default value: 'present'. +#####`force` + +Deprecated as of concat v2.0.0. Has no effect. + #####`group` Specifies a permissions group for the destination file. Valid options: a string containing a group name. Default value: undefined. @@ -170,11 +174,11 @@ Specifies whether to overwrite the destination file if it already exists. Valid Specifies a validation command to apply to the destination file. Requires Puppet version 3.5 or newer. Valid options: a string to be passed to a file resource. Default value: undefined. -#####`warn_header` +#####`warn` -Specifies whether to add a header message at the top of the destination file so users know it was autogenerated by Puppet. In earlier versions of the concat module, this parameter was called `warn`. Valid options: the booleans 'true' and 'false', or a string to serve as the header. Default value: 'false'. +Specifies whether to add a header message at the top of the destination file. Valid options: the booleans 'true' and 'false', or a string to serve as the header. Default value: 'false'. -If you set 'warn_header' to 'true', `concat` adds the following message: +If you set 'warn' to 'true', `concat` adds the following message: ~~~ # This file is managed by Puppet. DO NOT EDIT. @@ -188,6 +192,10 @@ Except where noted, all the below parameters are optional. Supplies the content of the fragment. **Note**: You must supply either a `content` parameter or a `source` parameter. Valid options: a string. Default value: undef. +#####`ensure` + +Deprecated as of concat v2.0.0. Has no effect. + #####`order` Reorders your fragments within the destination file. Fragments that share the same order number are ordered by name. Valid options: a string (recommended) or an integer. Default value: '10'. @@ -206,11 +214,8 @@ The following functionality existed in previous versions of the concat module, b Parameters removed from `concat`: * `ensure_newline` -* `force` -* `warn` (replaced by `warn_header`) Parameters removed from `concat::fragment`: -* `ensure` * `gnu` * `backup` * `group` From ef4aac80ec2d8616243652245d26f66c6ac090fe Mon Sep 17 00:00:00 2001 From: Bryan Jen Date: Mon, 20 Apr 2015 13:38:11 -0700 Subject: [PATCH 12/76] fix for strict variables checking --- manifests/init.pp | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/manifests/init.pp b/manifests/init.pp index 981a5c27..83eb191c 100644 --- a/manifests/init.pp +++ b/manifests/init.pp @@ -80,14 +80,15 @@ case $warn { true: { $warn_message = $default_warn_message - $append_header = true + $_append_header = true } false: { $warn_message = '' + $_append_header = false } default: { $warn_message = $warn - $append_header = true + $_append_header = true } } @@ -104,7 +105,7 @@ validate_cmd => $validate_cmd, } - if $append_header { + if $_append_header { file_fragment { "#{$name}_header": tag => $safe_name, content => $warn_message, From 0459ebace52184319f80241ec0b569588ca09cb8 Mon Sep 17 00:00:00 2001 From: Bryan Jen Date: Thu, 23 Apr 2015 15:55:30 -0700 Subject: [PATCH 13/76] MODULES-1933: fixes backup passing in fragments without concat resource --- manifests/fragment.pp | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/manifests/fragment.pp b/manifests/fragment.pp index f9ee2a7c..7b13b004 100644 --- a/manifests/fragment.pp +++ b/manifests/fragment.pp @@ -57,7 +57,13 @@ if $backup { warning('The $backup parameter to concat::fragment is deprecated and has no effect') } + $my_backup = concat_getparam(Concat[$target], 'backup') + $_backup = $my_backup ? { + '' => undef, + default => $my_backup + } + if $ensure == undef { $my_ensure = concat_getparam(Concat[$target], 'ensure') } else { @@ -119,7 +125,7 @@ mode => $fragmode, source => $source, content => $content, - backup => $my_backup, + backup => $_backup, replace => true, alias => "concat_fragment_${name}", notify => Exec["concat_${target}"] From cd21d8b37fa8e43715bdfe06d4903813eb43704a Mon Sep 17 00:00:00 2001 From: Bryan Jen Date: Thu, 23 Apr 2015 17:20:22 -0700 Subject: [PATCH 14/76] update metadata.json with correct dependencies and setting exact version for file_concat dependency --- metadata.json | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/metadata.json b/metadata.json index 137499e9..9cd3909a 100644 --- a/metadata.json +++ b/metadata.json @@ -97,7 +97,7 @@ "requirements": [ { "name": "pe", - "version_requirement": "3.x" + "version_requirement": ">= 3.7.0 < 4.0" }, { "name": "puppet", @@ -105,7 +105,7 @@ } ], "dependencies": [ - {"name":"puppetlabs/stdlib","version_requirement":">= 3.2.0 < 5.0.0"}, - {"name":"electrical/file_concat","version_requirement":">= 1.0.1"} + {"name":"puppetlabs/stdlib","version_requirement":">= 4.5.0 < 5.0.0"}, + {"name":"electrical/file_concat","version_requirement":"1.0.1"} ] } From 1534050de2158daad5f149694a9f24878bf01ce6 Mon Sep 17 00:00:00 2001 From: Bryan Jen Date: Fri, 24 Apr 2015 13:24:17 -0700 Subject: [PATCH 15/76] fixes the metadata.json to be geppeto compliant --- metadata.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/metadata.json b/metadata.json index 9cd3909a..2aea043a 100644 --- a/metadata.json +++ b/metadata.json @@ -97,7 +97,7 @@ "requirements": [ { "name": "pe", - "version_requirement": ">= 3.7.0 < 4.0" + "version_requirement": ">= 3.7.0 < 4.0.0" }, { "name": "puppet", From 6876399da9818a5e6dfb6f6c2b076994bd26db7d Mon Sep 17 00:00:00 2001 From: Bryan Jen Date: Fri, 1 May 2015 11:09:01 -0700 Subject: [PATCH 16/76] re-refactor of concat to not depend on file_concat --- .fixtures.yml | 3 - README.md | 12 +- lib/puppet/type/concat_file.rb | 154 ++++++++++++++++++++++ lib/puppet/type/concat_fragment.rb | 48 +++++++ manifests/fragment.pp | 4 +- manifests/init.pp | 17 +-- metadata.json | 1 - spec/spec_helper_acceptance.rb | 8 -- spec/unit/defines/concat_fragment_spec.rb | 4 +- 9 files changed, 214 insertions(+), 37 deletions(-) create mode 100644 lib/puppet/type/concat_file.rb create mode 100644 lib/puppet/type/concat_fragment.rb diff --git a/.fixtures.yml b/.fixtures.yml index 3d0f23a9..ca236c57 100644 --- a/.fixtures.yml +++ b/.fixtures.yml @@ -3,8 +3,5 @@ fixtures: 'stdlib': repo: 'git://github.com/puppetlabs/puppetlabs-stdlib.git' ref: '4.5.1' - 'file_concat': - repo: 'git://github.com/electrical/puppet-lib-file_concat.git' - ref: '1.0.1' symlinks: 'concat': '#{source_dir}' diff --git a/README.md b/README.md index 29137548..2bd24938 100644 --- a/README.md +++ b/README.md @@ -23,12 +23,6 @@ The concat module lets you construct files from multiple ordered fragments of te The concat module lets you gather `concat::fragment` resources from your other modules and order them into a coherent file through a single `concat` resource. -##Setup - -###What concat affects - -The concat module requires the [file_concat module](https://forge.puppetlabs.com/electrical/file_concat). If you don't have file_concat installed, concat installs it for you. - ###Beginning with concat To start using concat you need to create: @@ -240,4 +234,8 @@ For more information, see our [module contribution guide.](https://docs.puppetla ###Contributors -To see who's already involved, see the [list of contributors.](https://github.com/puppetlabs/puppetlabs-concat/graphs/contributors) \ No newline at end of file +Richard Pijnenburg ([@Richardp82](http://twitter.com/richardp82)) + +Joshua Hoblitt ([@jhoblitt](http://twitter.com/jhoblitt)) + +[More contributors.](https://github.com/puppetlabs/puppetlabs-concat/graphs/contributors) diff --git a/lib/puppet/type/concat_file.rb b/lib/puppet/type/concat_file.rb new file mode 100644 index 00000000..bbac0db0 --- /dev/null +++ b/lib/puppet/type/concat_file.rb @@ -0,0 +1,154 @@ +require 'puppet/type/file/owner' +require 'puppet/type/file/group' +require 'puppet/type/file/mode' +require 'puppet/util/checksums' + +Puppet::Type.newtype(:concat_file) do + @doc = "Gets all the file fragments and puts these into the target file. + This will mostly be used with exported resources. + + example: + Concat_fragment <<| tag == 'unique_tag' |>> + + concat_file { '/tmp/file: + tag => 'unique_tag', # Mandatory + path => '/tmp/file', # Optional. If given it overrides the resource name + owner => 'root', # Optional. Default to undef + group => 'root', # Optional. Default to undef + mode => '0644' # Optional. Default to undef + order => 'numeric' # Optional, Default to 'numeric' + } + " + ensurable do + defaultvalues + + defaultto { :present } + end + + def exists? + self[:ensure] == :present + end + + newparam(:name, :namevar => true) do + desc "Resource name" + end + + newparam(:tag) do + desc "Tag reference to collect all concat_fragment's with the same tag" + end + + newparam(:path) do + desc "The output file" + defaultto do + resource.value(:name) + end + end + + newparam(:owner, :parent => Puppet::Type::File::Owner) do + desc "Desired file owner." + end + + newparam(:group, :parent => Puppet::Type::File::Group) do + desc "Desired file group." + end + + newparam(:mode, :parent => Puppet::Type::File::Mode) do + desc "Desired file mode." + end + + newparam(:order) do + desc "Controls the ordering of fragments. Can be set to alphabetical or numeric." + defaultto 'numeric' + end + + newparam(:backup) do + desc "Controls the filebucketing behavior of the final file and see File type reference for its use." + defaultto 'puppet' + end + + newparam(:replace) do + desc "Whether to replace a file that already exists on the local system." + defaultto true + end + + newparam(:validate_cmd) do + desc "Validates file." + end + + autorequire(:concat_fragment) do + catalog.resources.collect do |r| + if r.is_a?(Puppet::Type.type(:concat_fragment)) && r[:tag] == self[:tag] + r.name + end + end.compact + end + + def should_content + return @generated_content if @generated_content + @generated_content = "" + content_fragments = [] + + resources = catalog.resources.select do |r| + r.is_a?(Puppet::Type.type(:concat_fragment)) && r[:tag] == self[:tag] + end + + resources.each do |r| + content_fragments << ["#{r[:order]}___#{r[:name]}", fragment_content(r)] + end + + if self[:order] == 'numeric' + sorted = content_fragments.sort do |a, b| + def decompound(d) + d.split('___').map { |v| v =~ /^\d+$/ ? v.to_i : v } + end + + decompound(a[0]) <=> decompound(b[0]) + end + else + sorted = content_fragments.sort do |a, b| + def decompound(d) + d.split('___').first + end + + decompound(a[0]) <=> decompound(b[0]) + end + end + + @generated_content = sorted.map { |cf| cf[1] }.join + + @generated_content + end + + def fragment_content(r) + if r[:content].nil? == false + fragment_content = r[:content] + elsif r[:source].nil? == false + @source = nil + Array(r[:source]).each do |source| + if Puppet::FileServing::Metadata.indirection.find(source) + @source = source + break + end + end + self.fail "Could not retrieve source(s) #{r[:source].join(", ")}" unless @source + tmp = Puppet::FileServing::Content.indirection.find(@source, :environment => catalog.environment) + fragment_content = tmp.content unless tmp.nil? + end + fragment_content + end + + def eval_generate + file_opts = { + :ensure => self[:ensure] == :absent ? :absent : :file, + :content => self.should_content, + } + + [:path, :owner, :group, :mode, :replace, :backup].each do |param| + unless self[param].nil? + file_opts[param] = self[param] + end + end + + [Puppet::Type.type(:file).new(file_opts)] + end +end diff --git a/lib/puppet/type/concat_fragment.rb b/lib/puppet/type/concat_fragment.rb new file mode 100644 index 00000000..3a1e722b --- /dev/null +++ b/lib/puppet/type/concat_fragment.rb @@ -0,0 +1,48 @@ +Puppet::Type.newtype(:concat_fragment) do + @doc = "Create a concat fragment to be used by concat. + the `concat_fragment` type creates a file fragment to be collected by concat based on the tag. + The example is based on exported resources. + + Example: + @@concat_fragment { \"uniqe_name_${::fqdn}\": + tag => 'unique_name', + order => 10, # Optional. Default to 10 + content => 'some content' # OR + content => template('template.erb') # OR + source => 'puppet:///path/to/file' + } + " + + newparam(:name, :namevar => true) do + desc "Unique name" + end + + newparam(:content) do + desc "Content" + end + + newparam(:source) do + desc "Source" + end + + newparam(:order) do + desc "Order" + defaultto '10' + validate do |val| + fail Puppet::ParseError, '$order is not a string or integer.' if !(val.is_a? String or val.is_a? Integer) + fail Puppet::ParseError, "Order cannot contain '/', ':', or '\n'." if val.to_s =~ /[:\n\/]/ + end + end + + newparam(:tag) do + desc "Tag name to be used by concat to collect all concat_fragments by tag name" + end + + validate do + # Check if either source or content is set. raise error if none is set + fail Puppet::ParseError, "Set either 'source' or 'content'" if self[:source].nil? && self[:content].nil? + + # Check if both are set, if so rais error + fail Puppet::ParseError, "Can't use 'source' and 'content' at the same time" if !self[:source].nil? && !self[:content].nil? + end +end diff --git a/manifests/fragment.pp b/manifests/fragment.pp index fabc3596..9cfae661 100644 --- a/manifests/fragment.pp +++ b/manifests/fragment.pp @@ -1,6 +1,6 @@ # == Define: concat::fragment # -# Creates a file_fragment in the catalogue +# Creates a concat_fragment in the catalogue # # === Options: # @@ -46,7 +46,7 @@ $safe_target_name = regsubst($target, '[/:\n\s]', '_', 'GM') - file_fragment { $name: + concat_fragment { $name: tag => $safe_target_name, order => $order, content => $content, diff --git a/manifests/init.pp b/manifests/init.pp index 83eb191c..986e6a7e 100644 --- a/manifests/init.pp +++ b/manifests/init.pp @@ -28,17 +28,6 @@ # Select whether to order associated fragments by 'alpha' or 'numeric'. # Defaults to 'alpha'. # -# === Actions: -# * Creates a file_concat resource from the electrical/puppet-lib-file_concat library. -# * Creates file_fragment resources from electrical/puppet-lib-file_concat -# -# === Aliases: -# -# * The exec can notified using Exec["concat_/path/to/file"] or -# Exec["concat_/path/to/directory"] -# * The final file can be referenced as File["/path/to/file"] or -# File["concat_/path/to/file"] -# define concat( $ensure = 'present', @@ -93,7 +82,7 @@ } if $ensure == 'present' { - file_concat { $name: + concat_file { $name: tag => $safe_name, path => $path, owner => $owner, @@ -106,14 +95,14 @@ } if $_append_header { - file_fragment { "#{$name}_header": + concat_fragment { "#{$name}_header": tag => $safe_name, content => $warn_message, order => '0', } } } else { - file_concat { $name: + concat_file { $name: ensure => $ensure, tag => $safe_name, path => $path, diff --git a/metadata.json b/metadata.json index 9cd3909a..fd7986ff 100644 --- a/metadata.json +++ b/metadata.json @@ -106,6 +106,5 @@ ], "dependencies": [ {"name":"puppetlabs/stdlib","version_requirement":">= 4.5.0 < 5.0.0"}, - {"name":"electrical/file_concat","version_requirement":"1.0.1"} ] } diff --git a/spec/spec_helper_acceptance.rb b/spec/spec_helper_acceptance.rb index 56c6ea76..d27df7da 100644 --- a/spec/spec_helper_acceptance.rb +++ b/spec/spec_helper_acceptance.rb @@ -22,23 +22,15 @@ command => "wget -P /root/ https://forgeapi.puppetlabs.com/v3/files/puppetlabs-stdlib-4.5.1.tar.gz --no-check-certificate", path => ['/opt/csw/bin/','/usr/bin/'] } - exec{'download-file_concat': - command => "wget -P /root/ https://forgeapi.puppetlabs.com/v3/files/electrical-file_concat-1.0.1.tar.gz --no-check-certificate", - path => ['/opt/csw/bin/','/usr/bin/'] - } EOS apply_manifest_on(host, get_deps) # have to use force otherwise it checks ssl cert even though it is a local file on host, puppet('module install /root/puppetlabs-stdlib-4.5.1.tar.gz --force --ignore-dependencies'), {:acceptable_exit_codes => [0, 1]} - on host, puppet('module install /root/electrical-file_concat-1.0.1.tar.gz --force --ignore-dependencies'), {:acceptable_exit_codes => [0, 1]} elsif host['platform'] =~ /windows/i on host, shell('curl -k -o c:/puppetlabs-stdlib-4.5.1.tar.gz https://forgeapi.puppetlabs.com/v3/files/puppetlabs-stdlib-4.5.1.tar.gz') on host, puppet('module install c:/puppetlabs-stdlib-4.5.1.tar.gz --force --ignore-dependencies'), {:acceptable_exit_codes => [0, 1]} - on host, shell('curl -k -o c:/electrical-file_concat-1.0.1.tar.gz https://forgeapi.puppetlabs.com/v3/files/electrical-file_concat-1.0.1.tar.gz') - on host, puppet('module install c:/electrical-file_concat-1.0.1.tar.gz --force --ignore-dependencies'), {:acceptable_exit_codes => [0, 1]} else on host, puppet('module install puppetlabs-stdlib'), {:acceptable_exit_codes => [0, 1]} - on host, puppet('module install electrical-file_concat'), {:acceptable_exit_codes => [0, 1]} end end end diff --git a/spec/unit/defines/concat_fragment_spec.rb b/spec/unit/defines/concat_fragment_spec.rb index 98be58a1..317b02cd 100644 --- a/spec/unit/defines/concat_fragment_spec.rb +++ b/spec/unit/defines/concat_fragment_spec.rb @@ -22,8 +22,8 @@ it do should contain_concat(p[:target]) - should contain_file_concat(p[:target]) - should contain_file_fragment(title) + should contain_concat_file(p[:target]) + should contain_concat_fragment(title) end end From 118e802cc0e01e07e42c0e77e971219f8d5ad544 Mon Sep 17 00:00:00 2001 From: Bryan Jen Date: Fri, 1 May 2015 14:44:43 -0700 Subject: [PATCH 17/76] readd ensure_newline param and tests for backwards compatibility --- lib/puppet/type/concat_file.rb | 25 ++++++++---- manifests/init.pp | 31 ++++++++++----- spec/acceptance/newline_spec.rb | 67 ++++++++++++++++++++++++++++++++ spec/unit/defines/concat_spec.rb | 32 +++++++++++++++ 4 files changed, 138 insertions(+), 17 deletions(-) create mode 100644 spec/acceptance/newline_spec.rb diff --git a/lib/puppet/type/concat_file.rb b/lib/puppet/type/concat_file.rb index bbac0db0..59e5c6f8 100644 --- a/lib/puppet/type/concat_file.rb +++ b/lib/puppet/type/concat_file.rb @@ -10,13 +10,14 @@ example: Concat_fragment <<| tag == 'unique_tag' |>> - concat_file { '/tmp/file: - tag => 'unique_tag', # Mandatory - path => '/tmp/file', # Optional. If given it overrides the resource name - owner => 'root', # Optional. Default to undef - group => 'root', # Optional. Default to undef - mode => '0644' # Optional. Default to undef - order => 'numeric' # Optional, Default to 'numeric' + concat_file { '/tmp/file': + tag => 'unique_tag', # Mandatory + path => '/tmp/file', # Optional. If given it overrides the resource name + owner => 'root', # Optional. Default to undef + group => 'root', # Optional. Default to undef + mode => '0644' # Optional. Default to undef + order => 'numeric' # Optional, Default to 'numeric' + ensure_newline => false # Optional, Defaults to false } " ensurable do @@ -75,6 +76,11 @@ def exists? desc "Validates file." end + newparam(:ensure_newline) do + desc "Whether to ensure there is a newline after each fragment." + defaultto false + end + autorequire(:concat_fragment) do catalog.resources.collect do |r| if r.is_a?(Puppet::Type.type(:concat_fragment)) && r[:tag] == self[:tag] @@ -134,6 +140,11 @@ def fragment_content(r) tmp = Puppet::FileServing::Content.indirection.find(@source, :environment => catalog.environment) fragment_content = tmp.content unless tmp.nil? end + + if self[:ensure_newline] + fragment_content<<"\n" + end + fragment_content end diff --git a/manifests/init.pp b/manifests/init.pp index 986e6a7e..3948fd50 100644 --- a/manifests/init.pp +++ b/manifests/init.pp @@ -16,7 +16,7 @@ # Who will own the file # [*mode*] # The mode of the final file -# [*warn_header*] +# [*warn*] # Adds a normal shell style comment top of the file indicating that it is # built by puppet # [*backup*] @@ -27,6 +27,13 @@ # [*order*] # Select whether to order associated fragments by 'alpha' or 'numeric'. # Defaults to 'alpha'. +# [*ensure_newline*] +# Specifies whether to ensure there's a new line at the end of each fragment. +# Valid options: 'true' and 'false'. Default value: 'false'. +# [*validate_cmd*] +# Specifies a validation command to apply to the destination file. +# Requires Puppet version 3.5 or newer. Valid options: a string to be passed +# to a file resource. Default value: undefined. # define concat( @@ -40,6 +47,7 @@ $backup = 'puppet', $replace = true, $order = 'alpha', + $ensure_newline = false, $validate_cmd = undef, ) { validate_re($ensure, '^present$|^absent$') @@ -55,6 +63,8 @@ } validate_bool($replace) validate_re($order, '^alpha$|^numeric$') + validate_bool($ensure_newline) + if $validate_cmd and ! is_string($validate_cmd) { fail('$validate_cmd must be a string') } @@ -83,15 +93,16 @@ if $ensure == 'present' { concat_file { $name: - tag => $safe_name, - path => $path, - owner => $owner, - group => $group, - mode => $mode, - replace => $replace, - backup => $backup, - order => $order, - validate_cmd => $validate_cmd, + tag => $safe_name, + path => $path, + owner => $owner, + group => $group, + mode => $mode, + replace => $replace, + backup => $backup, + order => $order, + ensure_newline => $ensure_newline, + validate_cmd => $validate_cmd, } if $_append_header { diff --git a/spec/acceptance/newline_spec.rb b/spec/acceptance/newline_spec.rb new file mode 100644 index 00000000..c1fa16a1 --- /dev/null +++ b/spec/acceptance/newline_spec.rb @@ -0,0 +1,67 @@ +require 'spec_helper_acceptance' + +describe 'concat ensure_newline parameter' do + basedir = default.tmpdir('concat') + context '=> false' do + before(:all) do + pp = <<-EOS + file { '#{basedir}': + ensure => directory + } + EOS + + apply_manifest(pp) + end + pp = <<-EOS + concat { '#{basedir}/file': + ensure_newline => false, + } + concat::fragment { '1': + target => '#{basedir}/file', + content => '1', + } + concat::fragment { '2': + target => '#{basedir}/file', + content => '2', + } + EOS + + it 'applies the manifest twice with no stderr' do + apply_manifest(pp, :catch_failures => true) + apply_manifest(pp, :catch_changes => true) + end + + describe file("#{basedir}/file") do + it { should be_file } + its(:content) { should match '12' } + end + end + + context '=> true' do + pp = <<-EOS + concat { '#{basedir}/file': + ensure_newline => true, + } + concat::fragment { '1': + target => '#{basedir}/file', + content => '1', + } + concat::fragment { '2': + target => '#{basedir}/file', + content => '2', + } + EOS + + it 'applies the manifest twice with no stderr' do + apply_manifest(pp, :catch_failures => true) + apply_manifest(pp, :catch_changes => true) + end + + describe file("#{basedir}/file") do + it { should be_file } + its(:content) { + should match /1\n2\n/ + } + end + end +end diff --git a/spec/unit/defines/concat_spec.rb b/spec/unit/defines/concat_spec.rb index 35767aea..b2254f4e 100644 --- a/spec/unit/defines/concat_spec.rb +++ b/spec/unit/defines/concat_spec.rb @@ -248,4 +248,36 @@ end end end # order => + + context 'ensure_newline =>' do + [true, false].each do |ensure_newline| + context 'true' do + it_behaves_like 'concat', '/etc/foo.bar', { :ensure_newline => ensure_newline} + end + end + + context '123' do + let(:title) { '/etc/foo.bar' } + let(:params) {{ :ensure_newline => 123 }} + it 'should fail' do + expect { catalogue }.to raise_error(Puppet::Error, /is not a boolean/) + end + end + end # ensure_newline => + + context 'validate_cmd =>' do + context '/usr/bin/test -e %' do + it_behaves_like 'concat', '/etc/foo.bar', { :validate_cmd => '/usr/bin/test -e %' } + end + + [ 1234, true ].each do |cmd| + context cmd do + let(:title) { '/etc/foo.bar' } + let(:params) {{ :validate_cmd => cmd }} + it 'should fail' do + expect { catalogue }.to raise_error(Puppet::Error, /\$validate_cmd must be a string/) + end + end + end + end # validate_cmd => end From 1a2975a694fda9e4bce4026bc8e55b216b3a6fb7 Mon Sep 17 00:00:00 2001 From: Travis Fields Date: Fri, 1 May 2015 15:28:20 -0700 Subject: [PATCH 18/76] (maint) Only add newline if string does not end with newline --- lib/puppet/type/concat_file.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/puppet/type/concat_file.rb b/lib/puppet/type/concat_file.rb index 59e5c6f8..6a6225e3 100644 --- a/lib/puppet/type/concat_file.rb +++ b/lib/puppet/type/concat_file.rb @@ -142,7 +142,7 @@ def fragment_content(r) end if self[:ensure_newline] - fragment_content<<"\n" + fragment_content<<"\n" unless fragment_content =~ /\n$/ end fragment_content From 32e417afed48dc214498fca4cd72690aba4d5660 Mon Sep 17 00:00:00 2001 From: Justin Stoller Date: Sun, 3 May 2015 13:39:43 -0700 Subject: [PATCH 19/76] (maint) fixup invalid JSON Previously there was a trailing comma after the last item in a list. That is invalid in JSON. This commit removes that one comma. --- metadata.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/metadata.json b/metadata.json index 1f664c84..be39148a 100644 --- a/metadata.json +++ b/metadata.json @@ -105,6 +105,6 @@ } ], "dependencies": [ - {"name":"puppetlabs/stdlib","version_requirement":">= 4.5.0 < 5.0.0"}, + {"name":"puppetlabs/stdlib","version_requirement":">= 4.5.0 < 5.0.0"} ] } From 40cbb29779bffb702794dba0f7a70697d536fd6d Mon Sep 17 00:00:00 2001 From: Morgan Haskel Date: Wed, 6 May 2015 14:09:57 -0700 Subject: [PATCH 20/76] sync via modulesync --- .gitignore | 1 - .travis.yml | 1 + Gemfile | 31 ++++--------------------------- Rakefile | 2 +- 4 files changed, 6 insertions(+), 29 deletions(-) diff --git a/.gitignore b/.gitignore index ef01482d..b5db85e0 100644 --- a/.gitignore +++ b/.gitignore @@ -7,4 +7,3 @@ spec/fixtures/ coverage/ .idea/ *.iml -*.swp diff --git a/.travis.yml b/.travis.yml index ec6f08dc..827da319 100644 --- a/.travis.yml +++ b/.travis.yml @@ -1,4 +1,5 @@ --- +sudo: false language: ruby bundler_args: --without system_tests script: "bundle exec rake validate && bundle exec rake lint && bundle exec rake spec SPEC_OPTS='--format documentation'" diff --git a/Gemfile b/Gemfile index f7319c22..cc77f38c 100644 --- a/Gemfile +++ b/Gemfile @@ -1,49 +1,26 @@ source ENV['GEM_SOURCE'] || "https://rubygems.org" -def location_for(place, fake_version = nil) - if place =~ /^(git:[^#]*)#(.*)/ - [fake_version, { :git => $1, :branch => $2, :require => false }].compact - elsif place =~ /^file:\/\/(.*)/ - ['>= 0', { :path => File.expand_path($1), :require => false }] - else - [place, { :require => false }] - end -end - group :development, :unit_tests do - gem 'rake', :require => false - gem 'rspec-core', '~> 3.1.7', :require => false - gem 'rspec-puppet', '~> 2.0', :require => false + gem 'rspec-core', '3.1.7', :require => false gem 'puppetlabs_spec_helper', :require => false - gem 'puppet-lint', :require => false gem 'simplecov', :require => false gem 'puppet_facts', :require => false gem 'json', :require => false - gem 'pry' end -beaker_version = ENV['BEAKER_VERSION'] -beaker_rspec_version = ENV['BEAKER_RSPEC_VERSION'] group :system_tests do - if beaker_version - gem 'beaker', *location_for(beaker_version) - end - if beaker_rspec_version - gem 'beaker-rspec', *location_for(beaker_rspec_version) - else - gem 'beaker-rspec', :require => false - end + gem 'beaker-rspec', :require => false gem 'serverspec', :require => false end if facterversion = ENV['FACTER_GEM_VERSION'] - gem 'facter', *location_for(facterversion) + gem 'facter', facterversion, :require => false else gem 'facter', :require => false end if puppetversion = ENV['PUPPET_GEM_VERSION'] - gem 'puppet', *location_for(puppetversion) + gem 'puppet', puppetversion, :require => false else gem 'puppet', :require => false end diff --git a/Rakefile b/Rakefile index e3be95b0..181157e6 100644 --- a/Rakefile +++ b/Rakefile @@ -1,7 +1,7 @@ require 'puppetlabs_spec_helper/rake_tasks' require 'puppet-lint/tasks/puppet-lint' -PuppetLint.configuration.fail_on_warnings +PuppetLint.configuration.fail_on_warnings = true PuppetLint.configuration.send('relative') PuppetLint.configuration.send('disable_80chars') PuppetLint.configuration.send('disable_class_inherits_from_params_class') From 16988dfcd71f013f0e042716e636e553598dce03 Mon Sep 17 00:00:00 2001 From: Morgan Haskel Date: Wed, 6 May 2015 14:30:35 -0700 Subject: [PATCH 21/76] sync via modulesync --- Gemfile | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) diff --git a/Gemfile b/Gemfile index cc77f38c..fb7202c8 100644 --- a/Gemfile +++ b/Gemfile @@ -1,5 +1,15 @@ source ENV['GEM_SOURCE'] || "https://rubygems.org" +def location_for(place, fake_version = nil) + if place =~ /^(git:[^#]*)#(.*)/ + [fake_version, { :git => $1, :branch => $2, :require => false }].compact + elsif place =~ /^file:\/\/(.*)/ + ['>= 0', { :path => File.expand_path($1), :require => false }] + else + [place, { :require => false }] + end +end + group :development, :unit_tests do gem 'rspec-core', '3.1.7', :require => false gem 'puppetlabs_spec_helper', :require => false @@ -9,10 +19,16 @@ group :development, :unit_tests do end group :system_tests do - gem 'beaker-rspec', :require => false + if beaker_rspec_version = ENV['BEAKER_RSPEC_VERSION'] + gem 'beaker-rspec', *location_for(beaker_rspec_version) + else + gem 'beaker-rspec', :require => false + end gem 'serverspec', :require => false end + + if facterversion = ENV['FACTER_GEM_VERSION'] gem 'facter', facterversion, :require => false else From 754c927c276add02f337826cb308325cd2ef6f13 Mon Sep 17 00:00:00 2001 From: Morgan Haskel Date: Wed, 6 May 2015 14:32:24 -0700 Subject: [PATCH 22/76] beaker gemfile fixes --- Gemfile | 3 +++ 1 file changed, 3 insertions(+) diff --git a/Gemfile b/Gemfile index fb7202c8..2b1b7cd8 100644 --- a/Gemfile +++ b/Gemfile @@ -19,6 +19,9 @@ group :development, :unit_tests do end group :system_tests do + if beaker_version = ENV['BEAKER_VERSION'] + gem 'beaker', *location_for(beaker_version) + end if beaker_rspec_version = ENV['BEAKER_RSPEC_VERSION'] gem 'beaker-rspec', *location_for(beaker_rspec_version) else From 84ee577947cba2ce70bbd94431218c3dd3a9ef16 Mon Sep 17 00:00:00 2001 From: Bryan Jen Date: Thu, 7 May 2015 14:26:58 -0700 Subject: [PATCH 23/76] fix Gemfile --- Gemfile | 2 ++ 1 file changed, 2 insertions(+) diff --git a/Gemfile b/Gemfile index 2b1b7cd8..5eb00e1d 100644 --- a/Gemfile +++ b/Gemfile @@ -12,10 +12,12 @@ end group :development, :unit_tests do gem 'rspec-core', '3.1.7', :require => false + gem 'rspec-puppet', '~> 2.0', :require => false gem 'puppetlabs_spec_helper', :require => false gem 'simplecov', :require => false gem 'puppet_facts', :require => false gem 'json', :require => false + gem 'pry' end group :system_tests do From 4be434dc06bdd35376dd5c7f324eacaa012a662c Mon Sep 17 00:00:00 2001 From: Morgan Haskel Date: Fri, 8 May 2015 10:50:43 -0700 Subject: [PATCH 24/76] Lint fixes --- manifests/init.pp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/manifests/init.pp b/manifests/init.pp index 3948fd50..5bf53fc8 100644 --- a/manifests/init.pp +++ b/manifests/init.pp @@ -106,7 +106,7 @@ } if $_append_header { - concat_fragment { "#{$name}_header": + concat_fragment { "${name}_header": tag => $safe_name, content => $warn_message, order => '0', From 8d0442672fd619c487edb643731182f453440a47 Mon Sep 17 00:00:00 2001 From: Bryan Jen Date: Mon, 11 May 2015 16:21:43 -0700 Subject: [PATCH 25/76] 1.2.2 release prep --- CHANGELOG.md | 8 ++++++++ metadata.json | 2 +- 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7ba7ebeb..8c279a7d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,11 @@ +##2015-05-12 - Supported Release 1.2.2 +###Summary + +This release includes a bugfix. + +####Bugfixes +- Fixes a bug introduced by MODULES-1700, in handling default retrieval of fragment backup parameter. + ##2015-04-14 - Supported Release 1.2.1 ###Summary diff --git a/metadata.json b/metadata.json index ce932111..1c6bb98f 100644 --- a/metadata.json +++ b/metadata.json @@ -1,6 +1,6 @@ { "name": "puppetlabs-concat", - "version": "1.2.1", + "version": "1.2.2", "author": "Puppet Labs", "summary": "Construct files from multiple fragments.", "license": "Apache-2.0", From b5f17644b5a07bd7de514aac512750e1b0ce5567 Mon Sep 17 00:00:00 2001 From: Bryan Jen Date: Mon, 11 May 2015 17:04:48 -0700 Subject: [PATCH 26/76] update metadata.json --- metadata.json | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/metadata.json b/metadata.json index 1c6bb98f..663434c2 100644 --- a/metadata.json +++ b/metadata.json @@ -7,6 +7,9 @@ "source": "https://github.com/puppetlabs/puppetlabs-concat", "project_page": "https://github.com/puppetlabs/puppetlabs-concat", "issues_url": "https://tickets.puppetlabs.com/browse/MODULES", + "dependencies": [ + {"name":"puppetlabs/stdlib","version_requirement":">= 3.2.0 < 5.0.0"} + ], "operatingsystem_support": [ { "operatingsystem": "RedHat", @@ -103,8 +106,5 @@ "name": "puppet", "version_requirement": "3.x" } - ], - "dependencies": [ - {"name":"puppetlabs/stdlib","version_requirement":">= 3.2.0 < 5.0.0"} ] } From 9c5857fdc6310be588c7df88095161fa9e07b9de Mon Sep 17 00:00:00 2001 From: Bryan Jen Date: Mon, 11 May 2015 16:21:43 -0700 Subject: [PATCH 27/76] 1.2.2 release prep --- CHANGELOG.md | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 2043b604..ddedcdd7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,26 @@ +======= +##2015-05-12 - Supported Release 1.2.2 +###Summary + +This release includes a bugfix. + +####Bugfixes +- Fixes a bug introduced by MODULES-1700, in handling default retrieval of fragment backup parameter. + +##2015-04-14 - Supported Release 1.2.1 +###Summary + +This release includes bugfixes, test improvements, and a rewritten README. + +####Bugfixes + +- Verifies existence of $is_pe before using it. +- Adds validation for $order param to not allow restricted characters. +- Sets the group id on Fragments for versions of Facter that contain the $gid fact. +- Sets the script group to 0 if the script is owned by root. +- Explicitly sets script and concat directories to the same owner and group. +- Re-adds support for fragment backup, so that static compiler can work with filebucket (MODULES-1700). + ##2015-02-17 - Supported Release 1.2.0 ###Summary From 26033e6d5804ce992390f03eaad320eaad659ab4 Mon Sep 17 00:00:00 2001 From: Bryan Jen Date: Tue, 12 May 2015 13:11:44 -0700 Subject: [PATCH 28/76] 2.0.0 release prep --- CHANGELOG.md | 12 +++++++ README.md | 88 +++++++++++++++++++++++++++++++++++++++++++++++---- metadata.json | 6 ++-- 3 files changed, 97 insertions(+), 9 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ddedcdd7..59204bb8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,4 +1,16 @@ ======= +##2015-05-12 - Supported Release 2.0.0 +###Summary + +This is a major release. Includes re-implementation of concat to use native Type and Providers, resulting in significantly improved performance and testability. Also includes a bugfix to alpha ordering of fragments. + +####Features +- Re-implementation of concat to use native Type and Providers. + +####Bugfixes +- Fixes a bug in alpha ordering of fragments. + + ##2015-05-12 - Supported Release 1.2.2 ###Summary diff --git a/README.md b/README.md index 2bd24938..db62c129 100644 --- a/README.md +++ b/README.md @@ -120,9 +120,13 @@ When you're finished, the motd file will look something like this: * `concat`: Manages a file, compiled from one or more text fragments. * `concat::fragment`: Manages a fragment of text to be compiled into a file. +###Types +* `concat_file`: Generates a file with content from fragments sharing a common unique tag. +* `concat_fragment`: Manages the fragment. + ###Parameters -####`concat` +####Define: `concat` All the parameters listed below are optional. @@ -134,6 +138,10 @@ Specifies whether (and how) to back up the destination file before overwriting i Specifies whether the destination file should exist. Setting to 'absent' tells Puppet to delete the destination file if it exists, and negates the effect of any other parameters. Valid options: 'present' and 'absent'. Default value: 'present'. +#####`ensure_newline` + +Specifies whether to add a line break at the end of each fragment that doesn't already end in one. Valid options: 'true' and 'false'. Default value: 'false'. + #####`force` Deprecated as of concat v2.0.0. Has no effect. @@ -178,7 +186,7 @@ If you set 'warn' to 'true', `concat` adds the following message: # This file is managed by Puppet. DO NOT EDIT. ~~~ -####`concat::fragment` +####Define: `concat::fragment` Except where noted, all the below parameters are optional. @@ -202,12 +210,80 @@ Specifies a file to read into the content of the fragment. **Note**: You must su *Required.* Specifies the destination file of the fragment. Valid options: a string containing an absolute path. -###Removed functionality -The following functionality existed in previous versions of the concat module, but were removed in version 2.0.0: +####Type: `concat_file` + +#####`backup` + +Specifies whether (and how) to back up the destination file before overwriting it. Your value gets passed on to Puppet's [native `file` resource](https://docs.puppetlabs.com/references/latest/type.html#file-attribute-backup) for execution. Valid options: 'true', 'false', or a string representing either a target filebucket or a filename extension beginning with ".". Default value: 'puppet'. + +#####`ensure` + +Specifies whether the destination file should exist. Setting to 'absent' tells Puppet to delete the destination file if it exists, and negates the effect of any other parameters. Valid options: 'present' and 'absent'. Default value: 'present'. + +#####`ensure_newline` + +Specifies whether to add a line break at the end of each fragment that doesn't already end in one. Valid options: 'true' and 'false'. Default value: 'false'. + +#####`group` + +Specifies a permissions group for the destination file. Valid options: a string containing a group name. Default value: undefined. + +#####`mode` + +Specifies the permissions mode of the destination file. Valid options: a string containing a permission mode value in octal notation. Default value: '0644'. + +#####`order` + +Specifies a method for sorting your fragments by name within the destination file. Valid options: 'alpha' (e.g., '1, 10, 2') or 'numeric' (e.g., '1, 2, 10'). Default value: 'numeric'. + +You can override this setting for individual fragments by adjusting the `order` parameter in their `concat::fragment` declarations. + +#####`owner` + +Specifies the owner of the destination file. Valid options: a string containing a username. Default value: undefined. + +#####`path` + +Specifies a destination file for the combined fragments. Valid options: a string containing an absolute path. Default value: the title of your declared resource. + +#####`replace` + +Specifies whether to overwrite the destination file if it already exists. Valid options: 'true' and 'false'. Default value: 'true'. + +####`tag` + +*Required.* Specifies a unique tag reference to collect all concat_fragments with the same tag. + +#####`validate_cmd` + +Specifies a validation command to apply to the destination file. Requires Puppet version 3.5 or newer. Valid options: a string to be passed to a file resource. Default value: undefined. + +####Type: `concat_fragment` + +#####`content` + +Supplies the content of the fragment. **Note**: You must supply either a `content` parameter or a `source` parameter. Valid options: a string. Default value: undef. + +#####`order` + +Reorders your fragments within the destination file. Fragments that share the same order number are ordered by name. Valid options: a string (recommended) or an integer. Default value: '10'. + +#####`source` + +Specifies a file to read into the content of the fragment. **Note**: You must supply either a `content` parameter or a `source` parameter. Valid options: a string or an array, containing one or more Puppet URLs. Default value: undefined. + +#####`tag` + +*Required.* Specifies a unique tag to be used by concat_file to reference and collect content. + +#####`target` + +*Required.* Specifies the destination file of the fragment. Valid options: a string containing an absolute path. + +###Removed functionality -Parameters removed from `concat`: -* `ensure_newline` +The following functionality existed in previous versions of the concat module, but was removed in version 2.0.0: Parameters removed from `concat::fragment`: * `gnu` diff --git a/metadata.json b/metadata.json index be39148a..c255ea49 100644 --- a/metadata.json +++ b/metadata.json @@ -7,6 +7,9 @@ "source": "https://github.com/puppetlabs/puppetlabs-concat", "project_page": "https://github.com/puppetlabs/puppetlabs-concat", "issues_url": "https://tickets.puppetlabs.com/browse/MODULES", + "dependencies": [ + {"name":"puppetlabs/stdlib","version_requirement":">= 4.5.0 < 5.0.0"} + ], "operatingsystem_support": [ { "operatingsystem": "RedHat", @@ -104,7 +107,4 @@ "version_requirement": "3.x" } ], - "dependencies": [ - {"name":"puppetlabs/stdlib","version_requirement":">= 4.5.0 < 5.0.0"} - ] } From ca73b725a21c8abbde24f39424460837b86ef50f Mon Sep 17 00:00:00 2001 From: Bryan Jen Date: Tue, 12 May 2015 13:57:06 -0700 Subject: [PATCH 29/76] fix linting --- metadata.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/metadata.json b/metadata.json index c255ea49..92fb8a18 100644 --- a/metadata.json +++ b/metadata.json @@ -106,5 +106,5 @@ "name": "puppet", "version_requirement": "3.x" } - ], + ] } From f4d4ca09b7fcfe94e02a2ca93165017542632da6 Mon Sep 17 00:00:00 2001 From: Hunter Haugen Date: Wed, 27 May 2015 08:11:28 -0700 Subject: [PATCH 30/76] Add ability to unittest puppet 4 --- .travis.yml | 3 +++ Gemfile | 2 -- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/.travis.yml b/.travis.yml index 827da319..57ad8550 100644 --- a/.travis.yml +++ b/.travis.yml @@ -12,5 +12,8 @@ matrix: env: PUPPET_GEM_VERSION="~> 3.0" - rvm: 2.0.0 env: PUPPET_GEM_VERSION="~> 3.0" + allow_failures: + - rvm: 2.1.6 + env: PUPPET_GEM_VERSION="~> 4.0" STRICT_VARIABLES="yes" notifications: email: false diff --git a/Gemfile b/Gemfile index 5eb00e1d..2b1b7cd8 100644 --- a/Gemfile +++ b/Gemfile @@ -12,12 +12,10 @@ end group :development, :unit_tests do gem 'rspec-core', '3.1.7', :require => false - gem 'rspec-puppet', '~> 2.0', :require => false gem 'puppetlabs_spec_helper', :require => false gem 'simplecov', :require => false gem 'puppet_facts', :require => false gem 'json', :require => false - gem 'pry' end group :system_tests do From 22927590249d9b21d334df9d041103f7ee3c6a35 Mon Sep 17 00:00:00 2001 From: Bryan Jen Date: Tue, 26 May 2015 17:20:56 -0700 Subject: [PATCH 31/76] fixes dependency bug in creating the target file --- lib/puppet/type/concat_file.rb | 8 ++++-- spec/acceptance/concurrency_spec.rb | 37 +++++++++++++++++++++++++ spec/acceptance/fragment_source_spec.rb | 1 - 3 files changed, 43 insertions(+), 3 deletions(-) create mode 100644 spec/acceptance/concurrency_spec.rb diff --git a/lib/puppet/type/concat_file.rb b/lib/puppet/type/concat_file.rb index 6a6225e3..9b3fde4d 100644 --- a/lib/puppet/type/concat_file.rb +++ b/lib/puppet/type/concat_file.rb @@ -148,10 +148,9 @@ def fragment_content(r) fragment_content end - def eval_generate + def generate file_opts = { :ensure => self[:ensure] == :absent ? :absent : :file, - :content => self.should_content, } [:path, :owner, :group, :mode, :replace, :backup].each do |param| @@ -162,4 +161,9 @@ def eval_generate [Puppet::Type.type(:file).new(file_opts)] end + + def eval_generate + catalog.resource("File[#{self[:path]}]")[:content] = should_content + [] + end end diff --git a/spec/acceptance/concurrency_spec.rb b/spec/acceptance/concurrency_spec.rb new file mode 100644 index 00000000..fcffdbd1 --- /dev/null +++ b/spec/acceptance/concurrency_spec.rb @@ -0,0 +1,37 @@ +require 'spec_helper_acceptance' + +describe 'with file recursive purge' do + basedir = default.tmpdir('concat') + context 'should still create concat file' do + pp = <<-EOS + file { '#{basedir}/bar': + ensure => directory, + purge => true, + recurse => true, + } + + concat { "foobar": + ensure => 'present', + path => '#{basedir}/bar/foobar', + } + + concat::fragment { 'foo': + target => 'foobar', + content => 'foo', + } + EOS + + it 'applies the manifest twice with no stderr' do + apply_manifest(pp, :catch_failures => true) + apply_manifest(pp, :catch_changes => true) + end + + describe file("#{basedir}/bar/foobar") do + it { should be_file } + its(:content) { + should match 'foo' + } + end + end +end + diff --git a/spec/acceptance/fragment_source_spec.rb b/spec/acceptance/fragment_source_spec.rb index 5d66e4d9..b208979a 100644 --- a/spec/acceptance/fragment_source_spec.rb +++ b/spec/acceptance/fragment_source_spec.rb @@ -149,7 +149,6 @@ end describe file("#{basedir}/fail_no_source") do #FIXME: Serverspec::Type::File doesn't support exists? for some reason. so... hack. - it { should_not be_file } it { should_not be_directory } end end From db3382992b29721a97bbb014fbdbafff3acf9d02 Mon Sep 17 00:00:00 2001 From: Bryan Jen Date: Wed, 27 May 2015 17:01:51 -0700 Subject: [PATCH 32/76] fix fragment target handling Needed to bind fragment targets to it's parent concat resource's title so that we can find them in the catalog. Throws a warning if the target is not found. --- README.md | 4 ++-- lib/puppet/type/concat_fragment.rb | 16 +++++++++++++ manifests/fragment.pp | 1 + manifests/init.pp | 1 + ...tion_warnings_spec.rb => warnings_spec.rb} | 23 ++++++++++++++++--- 5 files changed, 40 insertions(+), 5 deletions(-) rename spec/acceptance/{deprecation_warnings_spec.rb => warnings_spec.rb} (68%) diff --git a/README.md b/README.md index db62c129..dccc7984 100644 --- a/README.md +++ b/README.md @@ -208,7 +208,7 @@ Specifies a file to read into the content of the fragment. **Note**: You must su #####`target` -*Required.* Specifies the destination file of the fragment. Valid options: a string containing an absolute path. +*Required.* Specifies the destination file of the fragment. Valid options: a string containing the title of the parent `concat` resource. ####Type: `concat_file` @@ -279,7 +279,7 @@ Specifies a file to read into the content of the fragment. **Note**: You must su #####`target` -*Required.* Specifies the destination file of the fragment. Valid options: a string containing an absolute path. +*Required.* Specifies the destination file of the fragment. Valid options: a string containing the title of the parent `concat_file` resource. ###Removed functionality diff --git a/lib/puppet/type/concat_fragment.rb b/lib/puppet/type/concat_fragment.rb index 3a1e722b..4567f2fb 100644 --- a/lib/puppet/type/concat_fragment.rb +++ b/lib/puppet/type/concat_fragment.rb @@ -17,6 +17,10 @@ desc "Unique name" end + newparam(:target) do + desc "Target" + end + newparam(:content) do desc "Content" end @@ -38,7 +42,19 @@ desc "Tag name to be used by concat to collect all concat_fragments by tag name" end + autorequire(:file) do + unless catalog.resource("Concat_file[#{self[:target]}]") + warning "Target Concat_file[#{self[:target]}] not found in the catalog" + end + end + validate do + # Check if target is set + fail Puppet::ParseError, "Target not set" if self[:target].nil? + + # Check if tag is set + fail Puppet::ParseError, "Tag not set" if self[:tag].nil? + # Check if either source or content is set. raise error if none is set fail Puppet::ParseError, "Set either 'source' or 'content'" if self[:source].nil? && self[:content].nil? diff --git a/manifests/fragment.pp b/manifests/fragment.pp index 9cfae661..cc336435 100644 --- a/manifests/fragment.pp +++ b/manifests/fragment.pp @@ -47,6 +47,7 @@ $safe_target_name = regsubst($target, '[/:\n\s]', '_', 'GM') concat_fragment { $name: + target => $target, tag => $safe_target_name, order => $order, content => $content, diff --git a/manifests/init.pp b/manifests/init.pp index 5bf53fc8..6e97a952 100644 --- a/manifests/init.pp +++ b/manifests/init.pp @@ -107,6 +107,7 @@ if $_append_header { concat_fragment { "${name}_header": + target => $name, tag => $safe_name, content => $warn_message, order => '0', diff --git a/spec/acceptance/deprecation_warnings_spec.rb b/spec/acceptance/warnings_spec.rb similarity index 68% rename from spec/acceptance/deprecation_warnings_spec.rb rename to spec/acceptance/warnings_spec.rb index 8a689a76..89b3c3c5 100644 --- a/spec/acceptance/deprecation_warnings_spec.rb +++ b/spec/acceptance/warnings_spec.rb @@ -1,6 +1,6 @@ require 'spec_helper_acceptance' -describe 'deprecation warnings' do +describe 'warnings' do basedir = default.tmpdir('concat') shared_examples 'has_warning' do |pp, w| @@ -10,7 +10,7 @@ end end - context 'concat force parameter' do + context 'concat force parameter deprecation' do pp = <<-EOS concat { '#{basedir}/file': force => false, @@ -25,7 +25,7 @@ it_behaves_like 'has_warning', pp, w end - context 'concat::fragment ensure parameter' do + context 'concat::fragment ensure parameter deprecation' do context 'target file exists' do pp = <<-EOS concat { '#{basedir}/file': @@ -41,4 +41,21 @@ it_behaves_like 'has_warning', pp, w end end + + context 'concat::fragment target not found' do + context 'target not found' do + pp = <<-EOS + concat { 'file': + path => '#{basedir}/file', + } + concat::fragment { 'foo': + target => '#{basedir}/file', + content => 'bar', + } + EOS + w = 'not found in the catalog' + + it_behaves_like 'has_warning', pp, w + end + end end From 78fdb4e794e0858c058694a5db8790fb08fe853f Mon Sep 17 00:00:00 2001 From: Bryan Jen Date: Thu, 28 May 2015 16:51:35 -0700 Subject: [PATCH 33/76] adds file autorequire autorequire will add parent directories of the concat_file to the dependency chain. So they can be created in the right order if they do not exist. --- lib/puppet/type/concat_file.rb | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/lib/puppet/type/concat_file.rb b/lib/puppet/type/concat_file.rb index 9b3fde4d..79776c5b 100644 --- a/lib/puppet/type/concat_file.rb +++ b/lib/puppet/type/concat_file.rb @@ -89,6 +89,22 @@ def exists? end.compact end + # Copied from puppet's file type + # Autorequire the nearest ancestor directory found in the catalog. + autorequire(:file) do + req = [] + path = Pathname.new(self[:path]) + if !path.root? + # Start at our parent, to avoid autorequiring ourself + parents = path.parent.enum_for(:ascend) + if found = parents.find { |p| catalog.resource(:file, p.to_s) } + req << found.to_s + end + end + + req + end + def should_content return @generated_content if @generated_content @generated_content = "" From b271bff5059fc193f8a07275dc8fa7bda005599c Mon Sep 17 00:00:00 2001 From: David Schmitt Date: Fri, 29 May 2015 10:42:30 +0100 Subject: [PATCH 34/76] (MODULES-2080) Call out changed behaviour of 'warn' parameter Before 2.0.0, this parameter would add a newline at the end of the warn message. To improve flexibilty, this was removed. Please add it explicitely if you need it. --- README.md | 6 +++++- manifests/init.pp | 5 ++++- 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index db62c129..3a7f9ccf 100644 --- a/README.md +++ b/README.md @@ -183,9 +183,13 @@ Specifies whether to add a header message at the top of the destination file. Va If you set 'warn' to 'true', `concat` adds the following message: ~~~ -# This file is managed by Puppet. DO NOT EDIT. +# This file is managed by Puppet. DO NOT EDIT.\n ~~~ +Before 2.0.0, this parameter would add a newline at the end of the warn +message. To improve flexibilty, this was removed. Please add it explicitely if +you need it. + ####Define: `concat::fragment` Except where noted, all the below parameters are optional. diff --git a/manifests/init.pp b/manifests/init.pp index 5bf53fc8..46d4896d 100644 --- a/manifests/init.pp +++ b/manifests/init.pp @@ -18,7 +18,10 @@ # The mode of the final file # [*warn*] # Adds a normal shell style comment top of the file indicating that it is -# built by puppet +# built by puppet. +# Before 2.0.0, this parameter would add a newline at the end of the warn +# message. To improve flexibilty, this was removed. Please add it explicitely +# if you need it. # [*backup*] # Controls the filebucketing behavior of the final file and see File type # reference for its use. Defaults to 'puppet' From 64b9288ac77ccfb51701845e0375690373c4419c Mon Sep 17 00:00:00 2001 From: Bryan Jen Date: Fri, 29 May 2015 09:09:07 -0700 Subject: [PATCH 35/76] fix defaulted force behavior With Concat 2.0.x we've deprecated the -force parameter and the resulting bug caused concat files that have no content/fragments to overwrite the target file with an empty file. Concat should only create an empty file if no file exists, and if the file exists, then it should not overwrite it with an empty file. --- lib/puppet/type/concat_file.rb | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/lib/puppet/type/concat_file.rb b/lib/puppet/type/concat_file.rb index 9b3fde4d..0744fadd 100644 --- a/lib/puppet/type/concat_file.rb +++ b/lib/puppet/type/concat_file.rb @@ -163,7 +163,11 @@ def generate end def eval_generate - catalog.resource("File[#{self[:path]}]")[:content] = should_content + content = should_content + + if !content.nil? and !content.empty? + catalog.resource("File[#{self[:path]}]")[:content] = content + end [] end end From f1b7a50f68300107a900b93dd29facf4ac1bcf35 Mon Sep 17 00:00:00 2001 From: David Schmitt Date: Mon, 1 Jun 2015 10:36:22 +0100 Subject: [PATCH 36/76] Fixup the wording in README See discussion in https://github.com/puppetlabs/puppetlabs-concat/pull/320/files#r31339621 --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 3a7f9ccf..2b988346 100644 --- a/README.md +++ b/README.md @@ -180,10 +180,10 @@ Specifies a validation command to apply to the destination file. Requires Puppet Specifies whether to add a header message at the top of the destination file. Valid options: the booleans 'true' and 'false', or a string to serve as the header. Default value: 'false'. -If you set 'warn' to 'true', `concat` adds the following message: +If you set 'warn' to 'true', `concat` adds the following line with an `order` of `0`: ~~~ -# This file is managed by Puppet. DO NOT EDIT.\n +# This file is managed by Puppet. DO NOT EDIT. ~~~ Before 2.0.0, this parameter would add a newline at the end of the warn From abf40f0485e2e40cc55e58ce9ffac440d85a850d Mon Sep 17 00:00:00 2001 From: Bryan Jen Date: Mon, 1 Jun 2015 10:45:00 -0700 Subject: [PATCH 37/76] update README with correct behavior for fragment parameter --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 873f6478..69b9a898 100644 --- a/README.md +++ b/README.md @@ -218,7 +218,7 @@ Specifies a file to read into the content of the fragment. **Note**: You must su #####`target` -*Required.* Specifies the destination file of the fragment. Valid options: a string containing an absolute path. +*Required.* Specifies the destination file of the fragment. Valid options: a string containing the title of the parent `concat` resource. ###Deprecations @@ -280,4 +280,4 @@ For more information, see our [module contribution guide.](https://docs.puppetla ###Contributors -To see who's already involved, see the [list of contributors.](https://github.com/puppetlabs/puppetlabs-concat/graphs/contributors) \ No newline at end of file +To see who's already involved, see the [list of contributors.](https://github.com/puppetlabs/puppetlabs-concat/graphs/contributors) From d4376cb3f5d98d5236eaa3bcb07951ab3cb9e631 Mon Sep 17 00:00:00 2001 From: Bryan Jen Date: Mon, 1 Jun 2015 12:05:31 -0700 Subject: [PATCH 38/76] updates init.pp to pass linting --- manifests/init.pp | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/manifests/init.pp b/manifests/init.pp index c44c0328..57f6425b 100644 --- a/manifests/init.pp +++ b/manifests/init.pp @@ -99,6 +99,7 @@ $default_warn_message = '# This file is managed by Puppet. DO NOT EDIT.' $bool_warn_message = 'Using stringified boolean values (\'true\', \'yes\', \'on\', \'false\', \'no\', \'off\') to represent boolean true/false as the $warn parameter to concat is deprecated and will be treated as the warning message in a future release' + # lint:ignore:quoted_booleans case $warn { true: { $warn_message = $default_warn_message @@ -118,6 +119,7 @@ $warn_message = $warn } } + # lint:endignore $warnmsg_escaped = regsubst($warn_message, '\'', '\'\\\'\'', 'G') $warnflag = $warnmsg_escaped ? { @@ -177,15 +179,15 @@ } file { $name: - ensure => present, - owner => $owner, - group => $group, - mode => $mode, - replace => $replace, - path => $path, - alias => "concat_${name}", - source => "${fragdir}/${concat_name}", - backup => $backup, + ensure => present, + owner => $owner, + group => $group, + mode => $mode, + replace => $replace, + path => $path, + alias => "concat_${name}", + source => "${fragdir}/${concat_name}", + backup => $backup, } # Only newer versions of puppet 3.x support the validate_cmd parameter @@ -243,7 +245,7 @@ $absent_exec_command = $::kernel ? { 'windows' => 'cmd.exe /c exit 0', - default => 'true', + default => true, } $absent_exec_path = $::kernel ? { From 6751adb2826ae063f3eb5052895dbc13babc6251 Mon Sep 17 00:00:00 2001 From: Bryan Jen Date: Mon, 1 Jun 2015 15:30:04 -0700 Subject: [PATCH 39/76] 1.2.3 Release prep --- CHANGELOG.md | 9 +++++++++ metadata.json | 8 ++++---- 2 files changed, 13 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8c279a7d..2e606413 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,12 @@ +##2015-06-02 - Supported Release 1.2.3 +###Summary + +This release includes a README fix to document correct behavior of fragment target parameter. + +####Bugfixes + +- README Fix to correctly document how a fragment $target param should work. + ##2015-05-12 - Supported Release 1.2.2 ###Summary diff --git a/metadata.json b/metadata.json index 663434c2..1d459926 100644 --- a/metadata.json +++ b/metadata.json @@ -1,15 +1,12 @@ { "name": "puppetlabs-concat", - "version": "1.2.2", + "version": "1.2.3", "author": "Puppet Labs", "summary": "Construct files from multiple fragments.", "license": "Apache-2.0", "source": "https://github.com/puppetlabs/puppetlabs-concat", "project_page": "https://github.com/puppetlabs/puppetlabs-concat", "issues_url": "https://tickets.puppetlabs.com/browse/MODULES", - "dependencies": [ - {"name":"puppetlabs/stdlib","version_requirement":">= 3.2.0 < 5.0.0"} - ], "operatingsystem_support": [ { "operatingsystem": "RedHat", @@ -106,5 +103,8 @@ "name": "puppet", "version_requirement": "3.x" } + ], + "dependencies": [ + {"name":"puppetlabs/stdlib","version_requirement":">= 3.2.0 < 5.0.0"} ] } From 5267d3587c50e574cb96feda857f7f5b74e4e1ae Mon Sep 17 00:00:00 2001 From: Bryan Jen Date: Mon, 1 Jun 2015 15:30:04 -0700 Subject: [PATCH 40/76] 1.2.3 Release prep --- CHANGELOG.md | 9 +++++++++ metadata.json | 6 +++--- 2 files changed, 12 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 59204bb8..8ef82026 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,15 @@ This is a major release. Includes re-implementation of concat to use native Type ####Bugfixes - Fixes a bug in alpha ordering of fragments. +======= +##2015-06-02 - Supported Release 1.2.3 +###Summary + +This release includes a README fix to document correct behavior of fragment target parameter. + +####Bugfixes + +- README Fix to correctly document how a fragment $target param should work. ##2015-05-12 - Supported Release 1.2.2 ###Summary diff --git a/metadata.json b/metadata.json index 92fb8a18..a4a9e301 100644 --- a/metadata.json +++ b/metadata.json @@ -7,9 +7,6 @@ "source": "https://github.com/puppetlabs/puppetlabs-concat", "project_page": "https://github.com/puppetlabs/puppetlabs-concat", "issues_url": "https://tickets.puppetlabs.com/browse/MODULES", - "dependencies": [ - {"name":"puppetlabs/stdlib","version_requirement":">= 4.5.0 < 5.0.0"} - ], "operatingsystem_support": [ { "operatingsystem": "RedHat", @@ -106,5 +103,8 @@ "name": "puppet", "version_requirement": "3.x" } + ], + "dependencies": [ + {"name":"puppetlabs/stdlib","version_requirement":">= 3.2.0 < 5.0.0"} ] } From d6ee5a8606d9a5161c6f4d1e9bd9dd833701d9c1 Mon Sep 17 00:00:00 2001 From: Bryan Jen Date: Mon, 1 Jun 2015 15:41:42 -0700 Subject: [PATCH 41/76] Release 2.0.1 prep --- CHANGELOG.md | 11 +++++++++++ metadata.json | 2 +- 2 files changed, 12 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8ef82026..87a0443c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,14 @@ +======= +##2015-06-02 - Supported Release 2.0.1 +###Summary + +This is a bugfix release. + +####Bugfixes +- Fixes dependency graphing with concurrent modification of the same file. +- Fixes handling fragment target. +- Fixes the defaulted force behavior to handle empty concats correctly. + ======= ##2015-05-12 - Supported Release 2.0.0 ###Summary diff --git a/metadata.json b/metadata.json index a4a9e301..31d9144d 100644 --- a/metadata.json +++ b/metadata.json @@ -1,6 +1,6 @@ { "name": "puppetlabs-concat", - "version": "2.0.0", + "version": "2.0.1", "author": "Puppet Labs", "summary": "Construct files from multiple fragments.", "license": "Apache-2.0", From a5cce42a5ba266dfc052585a252492003696638f Mon Sep 17 00:00:00 2001 From: Bryan Jen Date: Mon, 1 Jun 2015 16:52:27 -0700 Subject: [PATCH 42/76] fixes bug introduced by a linting fix --- manifests/init.pp | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/manifests/init.pp b/manifests/init.pp index 57f6425b..077a29bd 100644 --- a/manifests/init.pp +++ b/manifests/init.pp @@ -243,10 +243,12 @@ backup => $backup, } + # lint:ignore:quoted_booleans $absent_exec_command = $::kernel ? { 'windows' => 'cmd.exe /c exit 0', - default => true, + default => 'true', } + # lint:endignore $absent_exec_path = $::kernel ? { 'windows' => $::path, From 09a380c92f1251634f1f7ce3d1fc2ae389ae7820 Mon Sep 17 00:00:00 2001 From: mh Date: Thu, 4 Jun 2015 15:46:22 +0200 Subject: [PATCH 43/76] (MODULES-2023) - autorequire the file we are generating concat_file generates a file resource which will manage the content, while the interface for requirements is Concat[] or at least Concat_file[]. The file is currently flowing within the graph without any dependency. However, so that require and notifies are working properly, the file must be present before the concat_file resource is finished being managed and potentially other resources depending on that one are being managed. This also makes the copy & paste from looking for the parent directory as originaly described within MODULES-2023 obsolete, as this will be handled by the file itself. --- lib/puppet/type/concat_file.rb | 15 ++------------- 1 file changed, 2 insertions(+), 13 deletions(-) diff --git a/lib/puppet/type/concat_file.rb b/lib/puppet/type/concat_file.rb index 3d503fb6..84061036 100644 --- a/lib/puppet/type/concat_file.rb +++ b/lib/puppet/type/concat_file.rb @@ -89,20 +89,9 @@ def exists? end.compact end - # Copied from puppet's file type - # Autorequire the nearest ancestor directory found in the catalog. + # Autorequire the file we are generating below autorequire(:file) do - req = [] - path = Pathname.new(self[:path]) - if !path.root? - # Start at our parent, to avoid autorequiring ourself - parents = path.parent.enum_for(:ascend) - if found = parents.find { |p| catalog.resource(:file, p.to_s) } - req << found.to_s - end - end - - req + [self[:path]] end def should_content From 4a78188541027d74b44a912e9483755dcc9fd35c Mon Sep 17 00:00:00 2001 From: Luis Fernandez Alvarez Date: Wed, 3 Jun 2015 12:07:31 +0200 Subject: [PATCH 44/76] (MODULES-2094) Extend regexp to remove parenthesis on safe names (MODULES-2094) Adds acceptance tests for special characters --- manifests/fragment.pp | 2 +- manifests/init.pp | 2 +- spec/acceptance/concat_spec.rb | 32 ++++++++++++++++++++++++++++++++ 3 files changed, 34 insertions(+), 2 deletions(-) diff --git a/manifests/fragment.pp b/manifests/fragment.pp index 9cfae661..0c64acef 100644 --- a/manifests/fragment.pp +++ b/manifests/fragment.pp @@ -44,7 +44,7 @@ fail("Can't use 'source' and 'content' at the same time") } - $safe_target_name = regsubst($target, '[/:\n\s]', '_', 'GM') + $safe_target_name = regsubst($target, '[/:\n\s\(\)]', '_', 'GM') concat_fragment { $name: tag => $safe_target_name, diff --git a/manifests/init.pp b/manifests/init.pp index 46d4896d..e481fc09 100644 --- a/manifests/init.pp +++ b/manifests/init.pp @@ -76,7 +76,7 @@ warning('The $force parameter to concat is deprecated and has no effect.') } - $safe_name = regsubst($name, '[/:\n\s]', '_', 'G') + $safe_name = regsubst($name, '[/:\n\s\(\)]', '_', 'G') $default_warn_message = "# This file is managed by Puppet. DO NOT EDIT.\n" case $warn { diff --git a/spec/acceptance/concat_spec.rb b/spec/acceptance/concat_spec.rb index c281954b..fd130fed 100644 --- a/spec/acceptance/concat_spec.rb +++ b/spec/acceptance/concat_spec.rb @@ -149,5 +149,37 @@ it { should_not be_file } end end + context 'works when set to present with path that has special characters' do + before(:all) do + pp = <<-EOS + file { '#{basedir}': + ensure => directory, + } + EOS + apply_manifest(pp) + end + pp=" + concat { 'file(a:b)': + ensure => present, + path => '#{basedir}/file(a:b)', + mode => '0644', + } + concat::fragment { '1': + target => 'file(a:b)', + content => '1', + order => '01', + } + " + + it_behaves_like 'successfully_applied', pp + + describe file("#{basedir}/file(a:b)") do + it { should be_file } + it("should be mode", :unless => (fact('osfamily') == 'AIX' or fact('osfamily') == 'windows')) { + should be_mode 644 + } + its(:content) { should match '1' } + end + end end end From 2059f817ac980b0c8cf3e72fae080fab01b05fb5 Mon Sep 17 00:00:00 2001 From: Jonathan Tripathy Date: Fri, 5 Jun 2015 12:44:31 +0100 Subject: [PATCH 45/76] Updated travisci file to do testing for Puppet 4 --- .travis.yml | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/.travis.yml b/.travis.yml index 57ad8550..001fb858 100644 --- a/.travis.yml +++ b/.travis.yml @@ -12,8 +12,7 @@ matrix: env: PUPPET_GEM_VERSION="~> 3.0" - rvm: 2.0.0 env: PUPPET_GEM_VERSION="~> 3.0" - allow_failures: - - rvm: 2.1.6 - env: PUPPET_GEM_VERSION="~> 4.0" STRICT_VARIABLES="yes" + - rvm: 2.1.6 + env: PUPPET_GEM_VERSION="~> 4.0" STRICT_VARIABLES="yes" notifications: email: false From 0ea02e4c6c42752e93680d2f5954cd70fbeb9267 Mon Sep 17 00:00:00 2001 From: Bryan Jen Date: Fri, 5 Jun 2015 11:50:29 -0700 Subject: [PATCH 46/76] fixes special characters test to support windows file restrictions --- spec/acceptance/concat_spec.rb | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/spec/acceptance/concat_spec.rb b/spec/acceptance/concat_spec.rb index fd130fed..f84a38fd 100644 --- a/spec/acceptance/concat_spec.rb +++ b/spec/acceptance/concat_spec.rb @@ -150,6 +150,8 @@ end end context 'works when set to present with path that has special characters' do + filename = fact('osfamily') == 'windows' ? 'file(1)' : 'file(1:2)' + before(:all) do pp = <<-EOS file { '#{basedir}': @@ -159,13 +161,13 @@ apply_manifest(pp) end pp=" - concat { 'file(a:b)': + concat { '#{filename}': ensure => present, - path => '#{basedir}/file(a:b)', + path => '#{basedir}/#{filename}', mode => '0644', } concat::fragment { '1': - target => 'file(a:b)', + target => '#{filename}', content => '1', order => '01', } @@ -173,7 +175,7 @@ it_behaves_like 'successfully_applied', pp - describe file("#{basedir}/file(a:b)") do + describe file("#{basedir}/#{filename}") do it { should be_file } it("should be mode", :unless => (fact('osfamily') == 'AIX' or fact('osfamily') == 'windows')) { should be_mode 644 From d39884e624aa3b9e51d48161665d40e02e186bdb Mon Sep 17 00:00:00 2001 From: Justin Stoller Date: Mon, 8 Jun 2015 21:52:31 -0700 Subject: [PATCH 47/76] (maint) allow setting PUPPET_VERSION in acceptance --- spec/spec_helper_acceptance.rb | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/spec/spec_helper_acceptance.rb b/spec/spec_helper_acceptance.rb index d27df7da..1293aeb9 100644 --- a/spec/spec_helper_acceptance.rb +++ b/spec/spec_helper_acceptance.rb @@ -5,7 +5,10 @@ unless ENV['RS_PROVISION'] == 'no' or ENV['BEAKER_provision'] == 'no' # This will install the latest available package on el and deb based # systems fail on windows and osx, and install via gem on other *nixes - foss_opts = {:default_action => 'gem_install'} + foss_opts = { + :default_action => 'gem_install', + :version => (ENV['PUPPET_VERSION'] || '3.8.1'), + } if default.is_pe?; then install_pe; From 9dd00aab2de3c4b364de8bd90e630173e3f4ed19 Mon Sep 17 00:00:00 2001 From: Hunter Haugen Date: Thu, 11 Jun 2015 15:28:34 -0700 Subject: [PATCH 48/76] Add helper to install puppet/pe/puppet-agent --- Gemfile | 1 + spec/spec_helper_acceptance.rb | 14 +++----------- 2 files changed, 4 insertions(+), 11 deletions(-) diff --git a/Gemfile b/Gemfile index cc4048ae..a1cfe6c1 100644 --- a/Gemfile +++ b/Gemfile @@ -33,6 +33,7 @@ group :system_tests do gem 'beaker-rspec', :require => false end gem 'serverspec', :require => false + gem 'beaker-puppet_install_helper', :require => false end if facterversion = ENV['FACTER_GEM_VERSION'] diff --git a/spec/spec_helper_acceptance.rb b/spec/spec_helper_acceptance.rb index da994f8d..80510464 100644 --- a/spec/spec_helper_acceptance.rb +++ b/spec/spec_helper_acceptance.rb @@ -1,20 +1,12 @@ require 'beaker-rspec/spec_helper' require 'beaker-rspec/helpers/serverspec' require 'acceptance/specinfra_stubs' +require 'beaker/puppet_install_helper' -unless ENV['RS_PROVISION'] == 'no' or ENV['BEAKER_provision'] == 'no' - # This will install the latest available package on el and deb based - # systems fail on windows and osx, and install via gem on other *nixes - foss_opts = {:default_action => 'gem_install'} - - if default.is_pe?; then - install_pe; - else - install_puppet(foss_opts); - end +run_puppet_install_helper +unless ENV['RS_PROVISION'] == 'no' or ENV['BEAKER_provision'] == 'no' hosts.each do |host| - on hosts, "mkdir -p #{host['distmoduledir']}" if host['platform'] =~ /sles-1/i || host['platform'] =~ /solaris-1/i get_stdlib = <<-EOS package{'wget':} From e71af8118d5eacdafe3e76d0e2eb0d6508710f60 Mon Sep 17 00:00:00 2001 From: Bryan Jen Date: Fri, 12 Jun 2015 10:11:50 -0700 Subject: [PATCH 49/76] =?UTF-8?q?(=E2=95=AF=C2=B0=E2=96=A1=C2=B0)=E2=95=AF?= =?UTF-8?q?=EF=B8=B5=E2=94=BB=E2=94=81=E2=94=BB=20the=202.0.x=20rewrite?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .fixtures.yml | 2 +- CHANGELOG.md | 24 -- README.md | 132 ++++------ files/concatfragments.rb | 153 +++++++++++ lib/facter/concat_basedir.rb | 11 + .../parser/functions/concat_getparam.rb | 35 +++ lib/puppet/parser/functions/concat_is_bool.rb | 22 ++ lib/puppet/type/concat_file.rb | 178 ------------- lib/puppet/type/concat_fragment.rb | 64 ----- manifests/fragment.pp | 113 +++++++-- manifests/init.pp | 234 +++++++++++++---- manifests/setup.pp | 64 +++++ metadata.json | 4 +- spec/acceptance/backup_spec.rb | 2 +- spec/acceptance/concat_spec.rb | 98 +++++--- spec/acceptance/concurrency_spec.rb | 37 --- spec/acceptance/deprecation_warnings_spec.rb | 238 ++++++++++++++++++ spec/acceptance/empty_spec.rb | 23 ++ spec/acceptance/fragment_source_spec.rb | 3 +- spec/acceptance/nodesets/default.yml | 8 +- spec/acceptance/order_spec.rb | 4 +- spec/acceptance/replace_spec.rb | 3 +- .../{warn_header_spec.rb => warn_spec.rb} | 2 +- spec/acceptance/warnings_spec.rb | 61 ----- spec/spec_helper_acceptance.rb | 11 +- spec/unit/classes/concat_setup_spec.rb | 98 ++++++++ spec/unit/defines/concat_fragment_spec.rb | 233 ++++++++++++++++- spec/unit/defines/concat_spec.rb | 166 ++++++++++-- spec/unit/facts/concat_basedir_spec.rb | 18 ++ tests/init.pp | 1 + 30 files changed, 1448 insertions(+), 594 deletions(-) create mode 100644 files/concatfragments.rb create mode 100644 lib/facter/concat_basedir.rb create mode 100644 lib/puppet/parser/functions/concat_getparam.rb create mode 100644 lib/puppet/parser/functions/concat_is_bool.rb delete mode 100644 lib/puppet/type/concat_file.rb delete mode 100644 lib/puppet/type/concat_fragment.rb create mode 100644 manifests/setup.pp delete mode 100644 spec/acceptance/concurrency_spec.rb create mode 100644 spec/acceptance/deprecation_warnings_spec.rb create mode 100644 spec/acceptance/empty_spec.rb rename spec/acceptance/{warn_header_spec.rb => warn_spec.rb} (98%) delete mode 100644 spec/acceptance/warnings_spec.rb create mode 100644 spec/unit/classes/concat_setup_spec.rb create mode 100644 spec/unit/facts/concat_basedir_spec.rb diff --git a/.fixtures.yml b/.fixtures.yml index ca236c57..67added0 100644 --- a/.fixtures.yml +++ b/.fixtures.yml @@ -2,6 +2,6 @@ fixtures: repositories: 'stdlib': repo: 'git://github.com/puppetlabs/puppetlabs-stdlib.git' - ref: '4.5.1' + ref: '4.2.0' symlinks: 'concat': '#{source_dir}' diff --git a/CHANGELOG.md b/CHANGELOG.md index 87a0443c..2e606413 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,27 +1,3 @@ -======= -##2015-06-02 - Supported Release 2.0.1 -###Summary - -This is a bugfix release. - -####Bugfixes -- Fixes dependency graphing with concurrent modification of the same file. -- Fixes handling fragment target. -- Fixes the defaulted force behavior to handle empty concats correctly. - -======= -##2015-05-12 - Supported Release 2.0.0 -###Summary - -This is a major release. Includes re-implementation of concat to use native Type and Providers, resulting in significantly improved performance and testability. Also includes a bugfix to alpha ordering of fragments. - -####Features -- Re-implementation of concat to use native Type and Providers. - -####Bugfixes -- Fixes a bug in alpha ordering of fragments. - -======= ##2015-06-02 - Supported Release 1.2.3 ###Summary diff --git a/README.md b/README.md index 8a041cee..69b9a898 100644 --- a/README.md +++ b/README.md @@ -9,9 +9,9 @@ * [Beginning with concat](#beginning-with-concat) 4. [Usage - Configuration options and additional functionality](#usage) 5. [Reference - An under-the-hood peek at what the module is doing and how](#reference) - * [Defines](#defines) + * [Public Defines](#public-defines) * [Parameters](#parameters) - * [Removed functionality](#removed-functionality) + * [Deprecations](#deprecations) 6. [Limitations - OS compatibility, etc.](#limitations) 7. [Development - Guide for contributing to the module](#development) @@ -21,7 +21,14 @@ The concat module lets you construct files from multiple ordered fragments of te ##Module Description -The concat module lets you gather `concat::fragment` resources from your other modules and order them into a coherent file through a single `concat` resource. +The concat module lets you gather `concat::fragment` resources from your other modules and order them through a single `concat` resource into a coherent file. It does this through a Ruby script and a temporary holding space for the fragments. + +##Setup + +###What concat affects + +* Installs `concatfragments.rb`. +* Adds a `concat/` directory into Puppet's `vardir`. ###Beginning with concat @@ -116,17 +123,15 @@ When you're finished, the motd file will look something like this: ##Reference -###Defines +**Note**: Several of this module's parameters and features have been deprecated. See the [Deprecations](#deprecations) section below. + +###Public defines * `concat`: Manages a file, compiled from one or more text fragments. * `concat::fragment`: Manages a fragment of text to be compiled into a file. -###Types -* `concat_file`: Generates a file with content from fragments sharing a common unique tag. -* `concat_fragment`: Manages the fragment. - ###Parameters -####Define: `concat` +####`concat` All the parameters listed below are optional. @@ -138,13 +143,14 @@ Specifies whether (and how) to back up the destination file before overwriting i Specifies whether the destination file should exist. Setting to 'absent' tells Puppet to delete the destination file if it exists, and negates the effect of any other parameters. Valid options: 'present' and 'absent'. Default value: 'present'. + #####`ensure_newline` -Specifies whether to add a line break at the end of each fragment that doesn't already end in one. Valid options: 'true' and 'false'. Default value: 'false'. +Specifies whether to ensure there's a new line at the end of each fragment. Valid options: 'true' and 'false'. Default value: 'false'. #####`force` -Deprecated as of concat v2.0.0. Has no effect. +In case no fragments have been added, this parameter specifies whether to go ahead and create a potentially empty file. Valid options: 'true' and 'false'. Default value: 'false'. #####`group` @@ -162,10 +168,12 @@ You can override this setting for individual fragments by adjusting the `order` #####`owner` + Specifies the owner of the destination file. Valid options: a string containing a username. Default value: undefined. #####`path` + Specifies a destination file for the combined fragments. Valid options: a string containing an absolute path. Default value: the title of your declared resource. #####`replace` @@ -178,19 +186,17 @@ Specifies a validation command to apply to the destination file. Requires Puppet #####`warn` -Specifies whether to add a header message at the top of the destination file. Valid options: the booleans 'true' and 'false', or a string to serve as the header. Default value: 'false'. +Specifies whether to add a warning message at the top of the destination file so users know it was autogenerated by Puppet. Valid options: 'true', 'false', or a string to be delivered as a warning message. Default value: 'false'. -If you set 'warn' to 'true', `concat` adds the following line with an `order` of `0`: + +If you set this parameter to 'true', Puppet adds the following message: ~~~ # This file is managed by Puppet. DO NOT EDIT. ~~~ -Before 2.0.0, this parameter would add a newline at the end of the warn -message. To improve flexibilty, this was removed. Please add it explicitely if -you need it. +####`concat::fragment` -####Define: `concat::fragment` Except where noted, all the below parameters are optional. @@ -200,7 +206,7 @@ Supplies the content of the fragment. **Note**: You must supply either a `conten #####`ensure` -Deprecated as of concat v2.0.0. Has no effect. +Specifies whether the fragment should be included in the destination file or discarded. Valid options: 'present' and 'absent'. Default value: 'present'. #####`order` @@ -214,91 +220,51 @@ Specifies a file to read into the content of the fragment. **Note**: You must su *Required.* Specifies the destination file of the fragment. Valid options: a string containing the title of the parent `concat` resource. +###Deprecations -####Type: `concat_file` - -#####`backup` - -Specifies whether (and how) to back up the destination file before overwriting it. Your value gets passed on to Puppet's [native `file` resource](https://docs.puppetlabs.com/references/latest/type.html#file-attribute-backup) for execution. Valid options: 'true', 'false', or a string representing either a target filebucket or a filename extension beginning with ".". Default value: 'puppet'. - -#####`ensure` - -Specifies whether the destination file should exist. Setting to 'absent' tells Puppet to delete the destination file if it exists, and negates the effect of any other parameters. Valid options: 'present' and 'absent'. Default value: 'present'. - -#####`ensure_newline` - -Specifies whether to add a line break at the end of each fragment that doesn't already end in one. Valid options: 'true' and 'false'. Default value: 'false'. - -#####`group` - -Specifies a permissions group for the destination file. Valid options: a string containing a group name. Default value: undefined. - -#####`mode` - -Specifies the permissions mode of the destination file. Valid options: a string containing a permission mode value in octal notation. Default value: '0644'. - -#####`order` - -Specifies a method for sorting your fragments by name within the destination file. Valid options: 'alpha' (e.g., '1, 10, 2') or 'numeric' (e.g., '1, 2, 10'). Default value: 'numeric'. +**`concat` has the following deprecations** -You can override this setting for individual fragments by adjusting the `order` parameter in their `concat::fragment` declarations. +#####`gnu` -#####`owner` +Generates a catalog compile time warning and has no effect. This parameter was silently ignored in version `1.0.0` and will be removed in a future release. -Specifies the owner of the destination file. Valid options: a string containing a username. Default value: undefined. +#####stringified 'true'/'false' values deprecated in `warn` -#####`path` +Passing stringified boolean values (strings of 'true' and 'false') to the `warn` parameter of `concat` is deprecated. Generates a catalog compile time warning, and will be silently treated as the concatenated file header/warning message in a future release. -Specifies a destination file for the combined fragments. Valid options: a string containing an absolute path. Default value: the title of your declared resource. +Please migrate to using the Puppet DSL's native [Boolean data +type](http://docs.puppetlabs.com/puppet/3/reference/lang_datatypes.html#booleans). -#####`replace` -Specifies whether to overwrite the destination file if it already exists. Valid options: 'true' and 'false'. Default value: 'true'. - -####`tag` - -*Required.* Specifies a unique tag reference to collect all concat_fragments with the same tag. - -#####`validate_cmd` - -Specifies a validation command to apply to the destination file. Requires Puppet version 3.5 or newer. Valid options: a string to be passed to a file resource. Default value: undefined. +**`concat::fragment` has the following deprecations** -####Type: `concat_fragment` +#####`backup` -#####`content` +Generates a catalog compile time warning and has no effect. In the `1.0.0` release this parameter controlled file bucketing of the file fragment. Bucketing the fragment(s) is redundant with bucketing the final concatenated file and this feature has been removed. -Supplies the content of the fragment. **Note**: You must supply either a `content` parameter or a `source` parameter. Valid options: a string. Default value: undef. -#####`order` - -Reorders your fragments within the destination file. Fragments that share the same order number are ordered by name. Valid options: a string (recommended) or an integer. Default value: '10'. +#####`group` -#####`source` +Generates a catalog compile time warning and has no effect. Had no user-visible effect in version `1.0.0` and will be removed in a future release. -Specifies a file to read into the content of the fragment. **Note**: You must supply either a `content` parameter or a `source` parameter. Valid options: a string or an array, containing one or more Puppet URLs. Default value: undefined. +#####`mode` -#####`tag` +Generates a catalog compile time warning and has no effect. Had no user-visible effect in version `1.0.0` and will be removed in a future release. -*Required.* Specifies a unique tag to be used by concat_file to reference and collect content. -#####`target` +#####`owner` -*Required.* Specifies the destination file of the fragment. Valid options: a string containing the title of the parent `concat_file` resource. +Generates a catalog compile time warning and has no effect. Had no user-visible effect in version `1.0.0` and will be removed in a future release. -###Removed functionality +#####file paths are deprecated in `ensure` -The following functionality existed in previous versions of the concat module, but was removed in version 2.0.0: +Passing a value other than 'present' or 'absent' in the `ensure` parameter of `concat::fragment` is **deprecated**, and generates a catalog compile time warning. The warning will become a catalog compilation failure in a future release. -Parameters removed from `concat::fragment`: -* `gnu` -* `backup` -* `group` -* `mode` -* `owner` +If you want to use the content of a file as a fragment please use the [`source`](#source) parameter. -The `concat::setup` class has also been removed. +####`concat::setup` -Prior to concat version 2.0.0, if you set the `warn` parameter to a string value of 'true', 'false', 'yes', 'no', 'on', or 'off', the module translated the string to the corresponding boolean value. In concat version 2.0.0 and newer, the `warn_header` parameter treats those values the same as other strings and uses them as the content of your header message. To avoid that, pass the 'true' and 'false' values as booleans instead of strings. +The `concat::setup` class should no longer be directly included in the manifest. It will be removed in a future release. ##Limitations @@ -314,8 +280,4 @@ For more information, see our [module contribution guide.](https://docs.puppetla ###Contributors -Richard Pijnenburg ([@Richardp82](http://twitter.com/richardp82)) - -Joshua Hoblitt ([@jhoblitt](http://twitter.com/jhoblitt)) - -[More contributors.](https://github.com/puppetlabs/puppetlabs-concat/graphs/contributors) +To see who's already involved, see the [list of contributors.](https://github.com/puppetlabs/puppetlabs-concat/graphs/contributors) diff --git a/files/concatfragments.rb b/files/concatfragments.rb new file mode 100644 index 00000000..b16f3e13 --- /dev/null +++ b/files/concatfragments.rb @@ -0,0 +1,153 @@ +#!/usr/bin/env ruby +# Script to concat files to a config file. +# +# Given a directory like this: +# /path/to/conf.d +# |-- fragments +# | |-- 00_named.conf +# | |-- 10_domain.net +# | `-- zz_footer +# +# The script supports a test option that will build the concat file to a temp location and +# use /usr/bin/cmp to verify if it should be run or not. This would result in the concat happening +# twice on each run but gives you the option to have an unless option in your execs to inhibit rebuilds. +# +# Without the test option and the unless combo your services that depend on the final file would end up +# restarting on each run, or in other manifest models some changes might get missed. +# +# OPTIONS: +# -o The file to create from the sources +# -d The directory where the fragments are kept +# -t Test to find out if a build is needed, basically concats the files to a temp +# location and compare with what's in the final location, return codes are designed +# for use with unless on an exec resource +# -w Add a shell style comment at the top of the created file to warn users that it +# is generated by puppet +# -f Enables the creation of empty output files when no fragments are found +# -n Sort the output numerically rather than the default alpha sort +# +# the command: +# +# concatfragments.rb -o /path/to/conffile.cfg -d /path/to/conf.d +# +# creates /path/to/conf.d/fragments.concat and copies the resulting +# file to /path/to/conffile.cfg. The files will be sorted alphabetically +# pass the -n switch to sort numerically. +# +# The script does error checking on the various dirs and files to make +# sure things don't fail. +require 'optparse' +require 'fileutils' + +settings = { + :outfile => "", + :workdir => "", + :test => false, + :force => false, + :warn => "", + :sortarg => "", + :newline => false +} + +OptionParser.new do |opts| + opts.banner = "Usage: #{$0} [options]" + opts.separator "Specific options:" + + opts.on("-o", "--outfile OUTFILE", String, "The file to create from the sources") do |o| + settings[:outfile] = o + end + + opts.on("-d", "--workdir WORKDIR", String, "The directory where the fragments are kept") do |d| + settings[:workdir] = d + end + + opts.on("-t", "--test", "Test to find out if a build is needed") do + settings[:test] = true + end + + opts.separator "Other options:" + opts.on("-w", "--warn WARNMSG", String, + "Add a shell style comment at the top of the created file to warn users that it is generated by puppet") do |w| + settings[:warn] = w + end + + opts.on("-f", "--force", "Enables the creation of empty output files when no fragments are found") do + settings[:force] = true + end + + opts.on("-n", "--sort", "Sort the output numerically rather than the default alpha sort") do + settings[:sortarg] = "-n" + end + + opts.on("-l", "--line", "Append a newline") do + settings[:newline] = true + end +end.parse! + +# do we have -o? +raise 'Please specify an output file with -o' unless !settings[:outfile].empty? + +# do we have -d? +raise 'Please specify fragments directory with -d' unless !settings[:workdir].empty? + +# can we write to -o? +if File.file?(settings[:outfile]) + if !File.writable?(settings[:outfile]) + raise "Cannot write to #{settings[:outfile]}" + end +else + if !File.writable?(File.dirname(settings[:outfile])) + raise "Cannot write to dirname #{File.dirname(settings[:outfile])} to create #{settings[:outfile]}" + end +end + +# do we have a fragments subdir inside the work dir? +if !File.directory?(File.join(settings[:workdir], "fragments")) && !File.executable?(File.join(settings[:workdir], "fragments")) + raise "Cannot access the fragments directory" +end + +# are there actually any fragments? +if (Dir.entries(File.join(settings[:workdir], "fragments")) - %w{ . .. }).empty? + if !settings[:force] + raise "The fragments directory is empty, cowardly refusing to make empty config files" + end +end + +Dir.chdir(settings[:workdir]) + +if settings[:warn].empty? + File.open("fragments.concat", 'w') { |f| f.write("") } +else + File.open("fragments.concat", 'w') { |f| f.write("#{settings[:warn]}\n") } +end + +# find all the files in the fragments directory, sort them numerically and concat to fragments.concat in the working dir +open('fragments.concat', 'a') do |f| + fragments = Dir.entries("fragments").sort + if settings[:sortarg] == '-n' + fragments = fragments.sort_by { |v| v.split('_').map(&:to_i) } + end + fragments.each { |entry| + if File.file?(File.join("fragments", entry)) + f << File.read(File.join("fragments", entry)) + + # append a newline if we were asked to (invoked with -l) + if settings[:newline] + f << "\n" + end + + end + } +end + +if !settings[:test] + # This is a real run, copy the file to outfile + FileUtils.cp 'fragments.concat', settings[:outfile] +else + # Just compare the result to outfile to help the exec decide + if FileUtils.cmp 'fragments.concat', settings[:outfile] + exit 0 + else + exit 1 + end +end diff --git a/lib/facter/concat_basedir.rb b/lib/facter/concat_basedir.rb new file mode 100644 index 00000000..bfac0710 --- /dev/null +++ b/lib/facter/concat_basedir.rb @@ -0,0 +1,11 @@ +# == Fact: concat_basedir +# +# A custom fact that sets the default location for fragments +# +# "${::vardir}/concat/" +# +Facter.add("concat_basedir") do + setcode do + File.join(Puppet[:vardir],"concat") + end +end diff --git a/lib/puppet/parser/functions/concat_getparam.rb b/lib/puppet/parser/functions/concat_getparam.rb new file mode 100644 index 00000000..1757bdc5 --- /dev/null +++ b/lib/puppet/parser/functions/concat_getparam.rb @@ -0,0 +1,35 @@ +# Test whether a given class or definition is defined +require 'puppet/parser/functions' + +Puppet::Parser::Functions.newfunction(:concat_getparam, + :type => :rvalue, + :doc => <<-'ENDOFDOC' +Takes a resource reference and name of the parameter and +returns value of resource's parameter. + +*Examples:* + + define example_resource($param) { + } + + example_resource { "example_resource_instance": + param => "param_value" + } + + concat_getparam(Example_resource["example_resource_instance"], "param") + +Would return: param_value +ENDOFDOC +) do |vals| + reference, param = vals + raise(ArgumentError, 'Must specify a reference') unless reference + raise(ArgumentError, 'Must specify name of a parameter') unless param and param.instance_of? String + + return '' if param.empty? + + if resource = findresource(reference.to_s) + return resource[param] if resource[param] + end + + return '' +end diff --git a/lib/puppet/parser/functions/concat_is_bool.rb b/lib/puppet/parser/functions/concat_is_bool.rb new file mode 100644 index 00000000..c2c2a9f2 --- /dev/null +++ b/lib/puppet/parser/functions/concat_is_bool.rb @@ -0,0 +1,22 @@ +# +# concat_is_bool.rb +# + +module Puppet::Parser::Functions + newfunction(:concat_is_bool, :type => :rvalue, :doc => <<-EOS +Returns true if the variable passed to this function is a boolean. + EOS + ) do |arguments| + + raise(Puppet::ParseError, "concat_is_bool(): Wrong number of arguments " + + "given (#{arguments.size} for 1)") if arguments.size != 1 + + type = arguments[0] + + result = type.is_a?(TrueClass) || type.is_a?(FalseClass) + + return result + end +end + +# vim: set ts=2 sw=2 et : diff --git a/lib/puppet/type/concat_file.rb b/lib/puppet/type/concat_file.rb deleted file mode 100644 index 84061036..00000000 --- a/lib/puppet/type/concat_file.rb +++ /dev/null @@ -1,178 +0,0 @@ -require 'puppet/type/file/owner' -require 'puppet/type/file/group' -require 'puppet/type/file/mode' -require 'puppet/util/checksums' - -Puppet::Type.newtype(:concat_file) do - @doc = "Gets all the file fragments and puts these into the target file. - This will mostly be used with exported resources. - - example: - Concat_fragment <<| tag == 'unique_tag' |>> - - concat_file { '/tmp/file': - tag => 'unique_tag', # Mandatory - path => '/tmp/file', # Optional. If given it overrides the resource name - owner => 'root', # Optional. Default to undef - group => 'root', # Optional. Default to undef - mode => '0644' # Optional. Default to undef - order => 'numeric' # Optional, Default to 'numeric' - ensure_newline => false # Optional, Defaults to false - } - " - ensurable do - defaultvalues - - defaultto { :present } - end - - def exists? - self[:ensure] == :present - end - - newparam(:name, :namevar => true) do - desc "Resource name" - end - - newparam(:tag) do - desc "Tag reference to collect all concat_fragment's with the same tag" - end - - newparam(:path) do - desc "The output file" - defaultto do - resource.value(:name) - end - end - - newparam(:owner, :parent => Puppet::Type::File::Owner) do - desc "Desired file owner." - end - - newparam(:group, :parent => Puppet::Type::File::Group) do - desc "Desired file group." - end - - newparam(:mode, :parent => Puppet::Type::File::Mode) do - desc "Desired file mode." - end - - newparam(:order) do - desc "Controls the ordering of fragments. Can be set to alphabetical or numeric." - defaultto 'numeric' - end - - newparam(:backup) do - desc "Controls the filebucketing behavior of the final file and see File type reference for its use." - defaultto 'puppet' - end - - newparam(:replace) do - desc "Whether to replace a file that already exists on the local system." - defaultto true - end - - newparam(:validate_cmd) do - desc "Validates file." - end - - newparam(:ensure_newline) do - desc "Whether to ensure there is a newline after each fragment." - defaultto false - end - - autorequire(:concat_fragment) do - catalog.resources.collect do |r| - if r.is_a?(Puppet::Type.type(:concat_fragment)) && r[:tag] == self[:tag] - r.name - end - end.compact - end - - # Autorequire the file we are generating below - autorequire(:file) do - [self[:path]] - end - - def should_content - return @generated_content if @generated_content - @generated_content = "" - content_fragments = [] - - resources = catalog.resources.select do |r| - r.is_a?(Puppet::Type.type(:concat_fragment)) && r[:tag] == self[:tag] - end - - resources.each do |r| - content_fragments << ["#{r[:order]}___#{r[:name]}", fragment_content(r)] - end - - if self[:order] == 'numeric' - sorted = content_fragments.sort do |a, b| - def decompound(d) - d.split('___').map { |v| v =~ /^\d+$/ ? v.to_i : v } - end - - decompound(a[0]) <=> decompound(b[0]) - end - else - sorted = content_fragments.sort do |a, b| - def decompound(d) - d.split('___').first - end - - decompound(a[0]) <=> decompound(b[0]) - end - end - - @generated_content = sorted.map { |cf| cf[1] }.join - - @generated_content - end - - def fragment_content(r) - if r[:content].nil? == false - fragment_content = r[:content] - elsif r[:source].nil? == false - @source = nil - Array(r[:source]).each do |source| - if Puppet::FileServing::Metadata.indirection.find(source) - @source = source - break - end - end - self.fail "Could not retrieve source(s) #{r[:source].join(", ")}" unless @source - tmp = Puppet::FileServing::Content.indirection.find(@source, :environment => catalog.environment) - fragment_content = tmp.content unless tmp.nil? - end - - if self[:ensure_newline] - fragment_content<<"\n" unless fragment_content =~ /\n$/ - end - - fragment_content - end - - def generate - file_opts = { - :ensure => self[:ensure] == :absent ? :absent : :file, - } - - [:path, :owner, :group, :mode, :replace, :backup].each do |param| - unless self[param].nil? - file_opts[param] = self[param] - end - end - - [Puppet::Type.type(:file).new(file_opts)] - end - - def eval_generate - content = should_content - - if !content.nil? and !content.empty? - catalog.resource("File[#{self[:path]}]")[:content] = content - end - [] - end -end diff --git a/lib/puppet/type/concat_fragment.rb b/lib/puppet/type/concat_fragment.rb deleted file mode 100644 index 4567f2fb..00000000 --- a/lib/puppet/type/concat_fragment.rb +++ /dev/null @@ -1,64 +0,0 @@ -Puppet::Type.newtype(:concat_fragment) do - @doc = "Create a concat fragment to be used by concat. - the `concat_fragment` type creates a file fragment to be collected by concat based on the tag. - The example is based on exported resources. - - Example: - @@concat_fragment { \"uniqe_name_${::fqdn}\": - tag => 'unique_name', - order => 10, # Optional. Default to 10 - content => 'some content' # OR - content => template('template.erb') # OR - source => 'puppet:///path/to/file' - } - " - - newparam(:name, :namevar => true) do - desc "Unique name" - end - - newparam(:target) do - desc "Target" - end - - newparam(:content) do - desc "Content" - end - - newparam(:source) do - desc "Source" - end - - newparam(:order) do - desc "Order" - defaultto '10' - validate do |val| - fail Puppet::ParseError, '$order is not a string or integer.' if !(val.is_a? String or val.is_a? Integer) - fail Puppet::ParseError, "Order cannot contain '/', ':', or '\n'." if val.to_s =~ /[:\n\/]/ - end - end - - newparam(:tag) do - desc "Tag name to be used by concat to collect all concat_fragments by tag name" - end - - autorequire(:file) do - unless catalog.resource("Concat_file[#{self[:target]}]") - warning "Target Concat_file[#{self[:target]}] not found in the catalog" - end - end - - validate do - # Check if target is set - fail Puppet::ParseError, "Target not set" if self[:target].nil? - - # Check if tag is set - fail Puppet::ParseError, "Tag not set" if self[:tag].nil? - - # Check if either source or content is set. raise error if none is set - fail Puppet::ParseError, "Set either 'source' or 'content'" if self[:source].nil? && self[:content].nil? - - # Check if both are set, if so rais error - fail Puppet::ParseError, "Can't use 'source' and 'content' at the same time" if !self[:source].nil? && !self[:content].nil? - end -end diff --git a/manifests/fragment.pp b/manifests/fragment.pp index 7084d9ea..7b13b004 100644 --- a/manifests/fragment.pp +++ b/manifests/fragment.pp @@ -1,6 +1,6 @@ # == Define: concat::fragment # -# Creates a concat_fragment in the catalogue +# Puts a file fragment into a directory previous setup using concat # # === Options: # @@ -13,44 +13,121 @@ # [*order*] # By default all files gets a 10_ prefix in the directory you can set it to # anything else using this to influence the order of the content in the file +# [*ensure*] +# Present/Absent or destination to a file to include another file +# [*mode*] +# Deprecated +# [*owner*] +# Deprecated +# [*group*] +# Deprecated +# [*backup*] +# Deprecated # define concat::fragment( $target, - $ensure = undef, $content = undef, $source = undef, $order = '10', + $ensure = undef, + $mode = undef, + $owner = undef, + $group = undef, + $backup = undef ) { validate_string($target) - - if $ensure != undef { - warning('The $ensure parameter to concat::fragment is deprecated and has no effect.') - } - validate_string($content) if !(is_string($source) or is_array($source)) { fail('$source is not a string or an Array.') } - if !(is_string($order) or is_integer($order)) { fail('$order is not a string or integer.') } elsif (is_string($order) and $order =~ /[:\n\/]/) { fail("Order cannot contain '/', ':', or '\n'.") } + if $mode { + warning('The $mode parameter to concat::fragment is deprecated and has no effect') + } + if $owner { + warning('The $owner parameter to concat::fragment is deprecated and has no effect') + } + if $group { + warning('The $group parameter to concat::fragment is deprecated and has no effect') + } + if $backup { + warning('The $backup parameter to concat::fragment is deprecated and has no effect') + } - if ! ($content or $source) { - crit('No content, source or symlink specified') - } elsif ($content and $source) { - fail("Can't use 'source' and 'content' at the same time") + $my_backup = concat_getparam(Concat[$target], 'backup') + $_backup = $my_backup ? { + '' => undef, + default => $my_backup } - $safe_target_name = regsubst($target, '[/:\n\s\(\)]', '_', 'GM') + if $ensure == undef { + $my_ensure = concat_getparam(Concat[$target], 'ensure') + } else { + if ! ($ensure in [ 'present', 'absent' ]) { + warning('Passing a value other than \'present\' or \'absent\' as the $ensure parameter to concat::fragment is deprecated. If you want to use the content of a file as a fragment please use the $source parameter.') + } + $my_ensure = $ensure + } - concat_fragment { $name: - target => $target, - tag => $safe_target_name, - order => $order, - content => $content, + include concat::setup + + $safe_name = regsubst($name, '[/:\n]', '_', 'GM') + $safe_target_name = regsubst($target, '[/:\n]', '_', 'GM') + $concatdir = $concat::setup::concatdir + $fragdir = "${concatdir}/${safe_target_name}" + $fragowner = $concat::setup::fragment_owner + $fraggroup = $concat::setup::fragment_group + $fragmode = $concat::setup::fragment_mode + + # The file type's semantics are problematic in that ensure => present will + # not over write a pre-existing symlink. We are attempting to provide + # backwards compatiblity with previous concat::fragment versions that + # supported the file type's ensure => /target syntax + + # be paranoid and only allow the fragment's file resource's ensure param to + # be file, absent, or a file target + $safe_ensure = $my_ensure ? { + '' => 'file', + undef => 'file', + 'file' => 'file', + 'present' => 'file', + 'absent' => 'absent', + default => $my_ensure, + } + + # if it looks line ensure => /target syntax was used, fish that out + if ! ($my_ensure in ['', 'present', 'absent', 'file' ]) { + $ensure_target = $my_ensure + } else { + $ensure_target = undef + } + + # the file type's semantics only allows one of: ensure => /target, content, + # or source + if ($ensure_target and $source) or + ($ensure_target and $content) or + ($source and $content) { + fail('You cannot specify more than one of $content, $source, $ensure => /target') + } + + if ! ($content or $source or $ensure_target) { + crit('No content, source or symlink specified') + } + + file { "${fragdir}/fragments/${order}_${safe_name}": + ensure => $safe_ensure, + owner => $fragowner, + group => $fraggroup, + mode => $fragmode, source => $source, + content => $content, + backup => $_backup, + replace => true, + alias => "concat_fragment_${name}", + notify => Exec["concat_${target}"] } } diff --git a/manifests/init.pp b/manifests/init.pp index da02842b..077a29bd 100644 --- a/manifests/init.pp +++ b/manifests/init.pp @@ -16,29 +16,42 @@ # Who will own the file # [*mode*] # The mode of the final file +# [*force*] +# Enables creating empty files if no fragments are present # [*warn*] # Adds a normal shell style comment top of the file indicating that it is -# built by puppet. -# Before 2.0.0, this parameter would add a newline at the end of the warn -# message. To improve flexibilty, this was removed. Please add it explicitely -# if you need it. +# built by puppet +# [*force*] # [*backup*] # Controls the filebucketing behavior of the final file and see File type # reference for its use. Defaults to 'puppet' # [*replace*] # Whether to replace a file that already exists on the local system # [*order*] -# Select whether to order associated fragments by 'alpha' or 'numeric'. -# Defaults to 'alpha'. # [*ensure_newline*] -# Specifies whether to ensure there's a new line at the end of each fragment. -# Valid options: 'true' and 'false'. Default value: 'false'. -# [*validate_cmd*] -# Specifies a validation command to apply to the destination file. -# Requires Puppet version 3.5 or newer. Valid options: a string to be passed -# to a file resource. Default value: undefined. +# [*gnu*] +# Deprecated +# +# === Actions: +# * Creates fragment directories if it didn't exist already +# * Executes the concatfragments.rb script to build the final file, this +# script will create directory/fragments.concat. Execution happens only +# when: +# * The directory changes +# * fragments.concat != final destination, this means rebuilds will happen +# whenever someone changes or deletes the final file. Checking is done +# using /usr/bin/cmp. +# * The Exec gets notified by something else - like the concat::fragment +# define +# * Copies the file over to the final destination using a file resource +# +# === Aliases: +# +# * The exec can notified using Exec["concat_/path/to/file"] or +# Exec["concat_/path/to/directory"] +# * The final file can be referenced as File["/path/to/file"] or +# File["concat_/path/to/file"] # - define concat( $ensure = 'present', $path = $name, @@ -46,12 +59,13 @@ $group = undef, $mode = '0644', $warn = false, - $force = undef, + $force = false, $backup = 'puppet', $replace = true, $order = 'alpha', $ensure_newline = false, $validate_cmd = undef, + $gnu = undef ) { validate_re($ensure, '^present$|^absent$') validate_absolute_path($path) @@ -61,68 +75,194 @@ if ! (is_string($warn) or $warn == true or $warn == false) { fail('$warn is not a string or boolean') } - if ! is_bool($backup) and ! is_string($backup) { + validate_bool($force) + if ! concat_is_bool($backup) and ! is_string($backup) { fail('$backup must be string or bool!') } validate_bool($replace) validate_re($order, '^alpha$|^numeric$') validate_bool($ensure_newline) - if $validate_cmd and ! is_string($validate_cmd) { fail('$validate_cmd must be a string') } - - if $force != undef { - warning('The $force parameter to concat is deprecated and has no effect.') + if $gnu { + warning('The $gnu parameter to concat is deprecated and has no effect') } - $safe_name = regsubst($name, '[/:\n\s\(\)]', '_', 'G') - $default_warn_message = "# This file is managed by Puppet. DO NOT EDIT.\n" + include concat::setup + $safe_name = regsubst($name, '[/:]', '_', 'G') + $concatdir = $concat::setup::concatdir + $fragdir = "${concatdir}/${safe_name}" + $concat_name = 'fragments.concat.out' + $script_command = $concat::setup::script_command + $default_warn_message = '# This file is managed by Puppet. DO NOT EDIT.' + $bool_warn_message = 'Using stringified boolean values (\'true\', \'yes\', \'on\', \'false\', \'no\', \'off\') to represent boolean true/false as the $warn parameter to concat is deprecated and will be treated as the warning message in a future release' + + # lint:ignore:quoted_booleans case $warn { true: { $warn_message = $default_warn_message - $_append_header = true + } + 'true', 'yes', 'on': { + warning($bool_warn_message) + $warn_message = $default_warn_message } false: { $warn_message = '' - $_append_header = false + } + 'false', 'no', 'off': { + warning($bool_warn_message) + $warn_message = '' } default: { $warn_message = $warn - $_append_header = true } } + # lint:endignore + + $warnmsg_escaped = regsubst($warn_message, '\'', '\'\\\'\'', 'G') + $warnflag = $warnmsg_escaped ? { + '' => '', + default => "-w '${warnmsg_escaped}'" + } + + $forceflag = $force ? { + true => '-f', + false => '', + } + + $orderflag = $order ? { + 'numeric' => '-n', + 'alpha' => '', + } + + $newlineflag = $ensure_newline ? { + true => '-l', + false => '', + } + + File { + backup => $backup, + } + + # reset poisoned Exec defaults + Exec { + user => undef, + group => undef, + } if $ensure == 'present' { - concat_file { $name: - tag => $safe_name, - path => $path, - owner => $owner, - group => $group, - mode => $mode, - replace => $replace, - backup => $backup, - order => $order, - ensure_newline => $ensure_newline, - validate_cmd => $validate_cmd, - } - - if $_append_header { - concat_fragment { "${name}_header": - target => $name, - tag => $safe_name, - content => $warn_message, - order => '0', + file { $fragdir: + ensure => directory, + mode => '0750', + } + + file { "${fragdir}/fragments": + ensure => directory, + mode => '0750', + force => true, + ignore => ['.svn', '.git', '.gitignore'], + notify => Exec["concat_${name}"], + purge => true, + recurse => true, + } + + file { "${fragdir}/fragments.concat": + ensure => present, + mode => '0640', + } + + file { "${fragdir}/${concat_name}": + ensure => present, + mode => '0640', + } + + file { $name: + ensure => present, + owner => $owner, + group => $group, + mode => $mode, + replace => $replace, + path => $path, + alias => "concat_${name}", + source => "${fragdir}/${concat_name}", + backup => $backup, + } + + # Only newer versions of puppet 3.x support the validate_cmd parameter + if $validate_cmd { + File[$name] { + validate_cmd => $validate_cmd, } } + + # remove extra whitespace from string interpolation to make testing easier + $command = strip(regsubst("${script_command} -o \"${fragdir}/${concat_name}\" -d \"${fragdir}\" ${warnflag} ${forceflag} ${orderflag} ${newlineflag}", '\s+', ' ', 'G')) + + # make sure ruby is in the path for PE + if defined('$is_pe') and $::is_pe { + if $::kernel == 'windows' { + $command_path = "${::env_windows_installdir}/bin:${::path}" + } else { + $command_path = "/opt/puppet/bin:${::path}" + } + } else { + $command_path = $::path + } + + # if puppet is running as root, this exec should also run as root to allow + # the concatfragments.rb script to potentially be installed in path that + # may not be accessible by a target non-root owner. + exec { "concat_${name}": + alias => "concat_${fragdir}", + command => $command, + notify => File[$name], + subscribe => File[$fragdir], + unless => "${command} -t", + path => $command_path, + require => [ + File[$fragdir], + File["${fragdir}/fragments"], + File["${fragdir}/fragments.concat"], + ], + } } else { - concat_file { $name: - ensure => $ensure, - tag => $safe_name, - path => $path, + file { [ + $fragdir, + "${fragdir}/fragments", + "${fragdir}/fragments.concat", + "${fragdir}/${concat_name}" + ]: + ensure => absent, + force => true, + } + + file { $path: + ensure => absent, backup => $backup, } + + # lint:ignore:quoted_booleans + $absent_exec_command = $::kernel ? { + 'windows' => 'cmd.exe /c exit 0', + default => 'true', + } + # lint:endignore + + $absent_exec_path = $::kernel ? { + 'windows' => $::path, + default => '/bin:/usr/bin', + } + + # Need to have an unless here for idempotency. + exec { "concat_${name}": + alias => "concat_${fragdir}", + command => $absent_exec_command, + unless => $absent_exec_command, + path => $absent_exec_path, + } } } +# vim:sw=2:ts=2:expandtab:textwidth=79 diff --git a/manifests/setup.pp b/manifests/setup.pp new file mode 100644 index 00000000..c5aedd82 --- /dev/null +++ b/manifests/setup.pp @@ -0,0 +1,64 @@ +# === Class: concat::setup +# +# Sets up the concat system. This is a private class. +# +# [$concatdir] +# is where the fragments live and is set on the fact concat_basedir. +# Since puppet should always manage files in $concatdir and they should +# not be deleted ever, /tmp is not an option. +# +# It also copies out the concatfragments.{sh,rb} file to ${concatdir}/bin +# +class concat::setup { + if $caller_module_name != $module_name { + warning("${name} is deprecated as a public API of the ${module_name} module and should no longer be directly included in the manifest.") + } + + if $::concat_basedir { + $concatdir = $::concat_basedir + } else { + fail ('$concat_basedir not defined. Try running again with pluginsync=true on the [master] and/or [main] section of your node\'s \'/etc/puppet/puppet.conf\'.') + } + + # owner,group and mode of fragment files (on windows owner and access rights should + # be inherited from concatdir and not explicitly set to avoid problems) + $fragment_owner = $::osfamily ? { 'windows' => undef, default => $::id } + $fragment_mode = $::osfamily ? { 'windows' => undef, default => '0640' } + # test on gid fact availability to support older facter versions + if defined('$gid') and $::gid and $::osfamily != 'Windows' { + $fragment_group = $::gid + } else { + $fragment_group = undef + } + + $script_name = 'concatfragments.rb' + + $script_path = "${concatdir}/bin/${script_name}" + + $default_owner = $::osfamily ? { 'windows' => undef, default => $::id } + + $default_group = $default_owner ? { 'root' => '0', default => undef } + + $script_mode = $::osfamily ? { 'windows' => undef, default => '0755' } + + $script_command = $::osfamily? { + 'windows' => "ruby.exe '${script_path}'", + 'openbsd' => "/usr/local/bin/ruby21 '${script_path}'", + default => $script_path + } + + file { $script_path: + ensure => file, + owner => $default_owner, + group => $default_group, + mode => $script_mode, + source => "puppet:///modules/concat/${script_name}", + } + + file { [ $concatdir, "${concatdir}/bin" ]: + ensure => directory, + owner => $default_owner, + group => $default_group, + mode => '0755', + } +} diff --git a/metadata.json b/metadata.json index 31d9144d..1d459926 100644 --- a/metadata.json +++ b/metadata.json @@ -1,6 +1,6 @@ { "name": "puppetlabs-concat", - "version": "2.0.1", + "version": "1.2.3", "author": "Puppet Labs", "summary": "Construct files from multiple fragments.", "license": "Apache-2.0", @@ -97,7 +97,7 @@ "requirements": [ { "name": "pe", - "version_requirement": ">= 3.7.0 < 4.0.0" + "version_requirement": "3.x" }, { "name": "puppet", diff --git a/spec/acceptance/backup_spec.rb b/spec/acceptance/backup_spec.rb index 0a737a55..1d3a5dfe 100644 --- a/spec/acceptance/backup_spec.rb +++ b/spec/acceptance/backup_spec.rb @@ -26,7 +26,7 @@ it 'applies the manifest twice with "Filebucketed" stdout and no stderr' do apply_manifest(pp, :catch_failures => true) do |r| - expect(r.stdout).to match(/Filebucketed #{basedir}\/file to puppet with sum 0140c31db86293a1a1e080ce9b91305f/) + expect(r.stdout).to match(/Filebucketed #{basedir}\/file to puppet with sum 0140c31db86293a1a1e080ce9b91305f/) # sum is for file contents of 'old contents' end apply_manifest(pp, :catch_changes => true) end diff --git a/spec/acceptance/concat_spec.rb b/spec/acceptance/concat_spec.rb index f84a38fd..c5c97277 100644 --- a/spec/acceptance/concat_spec.rb +++ b/spec/acceptance/concat_spec.rb @@ -38,6 +38,56 @@ apply_manifest(pp, :catch_failures => true) apply_manifest(pp, :catch_changes => true) end + + describe file("#{vardir}/concat") do + it { should be_directory } + it { should be_owned_by username } + it("should be mode", :unless => (fact('osfamily') == 'AIX' or fact('osfamily') == 'windows')) { + should be_mode 755 + } + end + describe file("#{vardir}/concat/bin") do + it { should be_directory } + it { should be_owned_by username } + it("should be mode", :unless => (fact('osfamily') == 'AIX' or fact('osfamily') == 'windows')) { + should be_mode 755 + } + end + describe file("#{vardir}/concat/bin/#{scriptname}") do + it { should be_file } + it { should be_owned_by username } + it("should be mode", :unless => (fact('osfamily') == 'AIX' or fact('osfamily') == 'windows')) { + should be_mode 755 + } + end + describe file("#{vardir}/concat/#{safe_basedir}_file") do + it { should be_directory } + it { should be_owned_by username } + it("should be mode", :unless => (fact('osfamily') == 'AIX' or fact('osfamily') == 'windows')) { + should be_mode 750 + } + end + describe file("#{vardir}/concat/#{safe_basedir}_file/fragments") do + it { should be_directory } + it { should be_owned_by username } + it("should be mode", :unless => (fact('osfamily') == 'AIX' or fact('osfamily') == 'windows')) { + should be_mode 750 + } + end + describe file("#{vardir}/concat/#{safe_basedir}_file/fragments.concat") do + it { should be_file } + it { should be_owned_by username } + it("should be mode", :unless => (fact('osfamily') == 'AIX' or fact('osfamily') == 'windows')) { + should be_mode 640 + } + end + describe file("#{vardir}/concat/#{safe_basedir}_file/fragments.concat.out") do + it { should be_file } + it { should be_owned_by username } + it("should be mode", :unless => (fact('osfamily') == 'AIX' or fact('osfamily') == 'windows')) { + should be_mode 640 + } + end end context 'owner/group root' do @@ -83,6 +133,20 @@ should match '2' } end + describe file("#{vardir}/concat/#{safe_basedir}_file/fragments/01_1") do + it { should be_file } + it { should be_owned_by username } + it("should be mode", :unless => (fact('osfamily') == 'AIX' or fact('osfamily') == 'windows')) { + should be_mode 640 + } + end + describe file("#{vardir}/concat/#{safe_basedir}_file/fragments/02_2") do + it { should be_file } + it { should be_owned_by username } + it("should be mode", :unless => (fact('osfamily') == 'AIX' or fact('osfamily') == 'windows')) { + should be_mode 640 + } + end end context 'ensure' do @@ -149,39 +213,5 @@ it { should_not be_file } end end - context 'works when set to present with path that has special characters' do - filename = fact('osfamily') == 'windows' ? 'file(1)' : 'file(1:2)' - - before(:all) do - pp = <<-EOS - file { '#{basedir}': - ensure => directory, - } - EOS - apply_manifest(pp) - end - pp=" - concat { '#{filename}': - ensure => present, - path => '#{basedir}/#{filename}', - mode => '0644', - } - concat::fragment { '1': - target => '#{filename}', - content => '1', - order => '01', - } - " - - it_behaves_like 'successfully_applied', pp - - describe file("#{basedir}/#{filename}") do - it { should be_file } - it("should be mode", :unless => (fact('osfamily') == 'AIX' or fact('osfamily') == 'windows')) { - should be_mode 644 - } - its(:content) { should match '1' } - end - end end end diff --git a/spec/acceptance/concurrency_spec.rb b/spec/acceptance/concurrency_spec.rb deleted file mode 100644 index fcffdbd1..00000000 --- a/spec/acceptance/concurrency_spec.rb +++ /dev/null @@ -1,37 +0,0 @@ -require 'spec_helper_acceptance' - -describe 'with file recursive purge' do - basedir = default.tmpdir('concat') - context 'should still create concat file' do - pp = <<-EOS - file { '#{basedir}/bar': - ensure => directory, - purge => true, - recurse => true, - } - - concat { "foobar": - ensure => 'present', - path => '#{basedir}/bar/foobar', - } - - concat::fragment { 'foo': - target => 'foobar', - content => 'foo', - } - EOS - - it 'applies the manifest twice with no stderr' do - apply_manifest(pp, :catch_failures => true) - apply_manifest(pp, :catch_changes => true) - end - - describe file("#{basedir}/bar/foobar") do - it { should be_file } - its(:content) { - should match 'foo' - } - end - end -end - diff --git a/spec/acceptance/deprecation_warnings_spec.rb b/spec/acceptance/deprecation_warnings_spec.rb new file mode 100644 index 00000000..11133ea9 --- /dev/null +++ b/spec/acceptance/deprecation_warnings_spec.rb @@ -0,0 +1,238 @@ +require 'spec_helper_acceptance' + +describe 'deprecation warnings' do + basedir = default.tmpdir('concat') + + shared_examples 'has_warning' do |pp, w| + it 'applies the manifest twice with a stderr regex' do + expect(apply_manifest(pp, :catch_failures => true).stderr).to match(/#{Regexp.escape(w)}/m) + expect(apply_manifest(pp, :catch_changes => true).stderr).to match(/#{Regexp.escape(w)}/m) + end + end + + context 'concat gnu parameter' do + pp = <<-EOS + concat { '#{basedir}/file': + gnu => 'foo', + } + concat::fragment { 'foo': + target => '#{basedir}/file', + content => 'bar', + } + EOS + w = 'The $gnu parameter to concat is deprecated and has no effect' + + it_behaves_like 'has_warning', pp, w + end + + context 'concat warn parameter =>' do + ['true', 'yes', 'on'].each do |warn| + context warn do + pp = <<-EOS + concat { '#{basedir}/file': + warn => '#{warn}', + } + concat::fragment { 'foo': + target => '#{basedir}/file', + content => 'bar', + } + EOS + w = 'Using stringified boolean values (\'true\', \'yes\', \'on\', \'false\', \'no\', \'off\') to represent boolean true/false as the $warn parameter to concat is deprecated and will be treated as the warning message in a future release' + + it_behaves_like 'has_warning', pp, w + + describe file("#{basedir}/file") do + it { should be_file } + its(:content) { + should match '# This file is managed by Puppet. DO NOT EDIT.' + should match 'bar' + } + end + end + end + + ['false', 'no', 'off'].each do |warn| + context warn do + pp = <<-EOS + concat { '#{basedir}/file': + warn => '#{warn}', + } + concat::fragment { 'foo': + target => '#{basedir}/file', + content => 'bar', + } + EOS + w = 'Using stringified boolean values (\'true\', \'yes\', \'on\', \'false\', \'no\', \'off\') to represent boolean true/false as the $warn parameter to concat is deprecated and will be treated as the warning message in a future release' + + it_behaves_like 'has_warning', pp, w + + describe file("#{basedir}/file") do + it { should be_file } + its(:content) { + should_not match '# This file is managed by Puppet. DO NOT EDIT.' + should match 'bar' + } + end + end + end + end + + context 'concat::fragment ensure parameter', :unless => fact('osfamily') == 'windows' do + context 'target file exists' do + before(:all) do + pp = <<-EOS + file { '#{basedir}': + ensure => directory, + } + file { '#{basedir}/file1': + content => "file1 contents\n", + } + EOS + apply_manifest(pp) + end + + pp = <<-EOS + concat { '#{basedir}/file': } + concat::fragment { 'foo': + target => '#{basedir}/file', + ensure => '#{basedir}/file1', + } + EOS + w = 'Passing a value other than \'present\' or \'absent\' as the $ensure parameter to concat::fragment is deprecated. If you want to use the content of a file as a fragment please use the $source parameter.' + + it_behaves_like 'has_warning', pp, w + + describe file("#{basedir}/file") do + it { should be_file } + its(:content) { should match 'file1 contents' } + end + + describe 'the fragment can be changed from a symlink to a plain file', :unless => (fact("osfamily") == "windows") do + pp = <<-EOS + concat { '#{basedir}/file': } + concat::fragment { 'foo': + target => '#{basedir}/file', + content => 'new content', + } + EOS + + it 'applies the manifest twice with no stderr' do + apply_manifest(pp, :catch_failures => true) + apply_manifest(pp, :catch_changes => true) + end + + describe file("#{basedir}/file") do + it { should be_file } + its(:content) { + should match 'new content' + should_not match 'file1 contents' + } + end + end + end # target file exists + + context 'target does not exist', :unless => fact('osfamily') == 'windows' do + pp = <<-EOS + concat { '#{basedir}/file': } + concat::fragment { 'foo': + target => '#{basedir}/file', + ensure => '#{basedir}/file1', + } + EOS + w = 'Passing a value other than \'present\' or \'absent\' as the $ensure parameter to concat::fragment is deprecated. If you want to use the content of a file as a fragment please use the $source parameter.' + + it_behaves_like 'has_warning', pp, w + + describe file("#{basedir}/file") do + it { should be_file } + end + + describe 'the fragment can be changed from a symlink to a plain file', :unless => (fact('osfamily') == 'windows') do + pp = <<-EOS + concat { '#{basedir}/file': } + concat::fragment { 'foo': + target => '#{basedir}/file', + content => 'new content', + } + EOS + + it 'applies the manifest twice with no stderr' do + apply_manifest(pp, :catch_failures => true) + apply_manifest(pp, :catch_changes => true) + end + + describe file("#{basedir}/file") do + it { should be_file } + its(:content) { should match 'new content' } + end + end + end # target file exists + + end # concat::fragment ensure parameter + + context 'concat::fragment mode parameter' do + pp = <<-EOS + concat { '#{basedir}/file': } + concat::fragment { 'foo': + target => '#{basedir}/file', + content => 'bar', + mode => 'bar', + } + EOS + w = 'The $mode parameter to concat::fragment is deprecated and has no effect' + + it_behaves_like 'has_warning', pp, w + end + + context 'concat::fragment owner parameter' do + pp = <<-EOS + concat { '#{basedir}/file': } + concat::fragment { 'foo': + target => '#{basedir}/file', + content => 'bar', + owner => 'bar', + } + EOS + w = 'The $owner parameter to concat::fragment is deprecated and has no effect' + + it_behaves_like 'has_warning', pp, w + end + + context 'concat::fragment group parameter' do + pp = <<-EOS + concat { '#{basedir}/file': } + concat::fragment { 'foo': + target => '#{basedir}/file', + content => 'bar', + group => 'bar', + } + EOS + w = 'The $group parameter to concat::fragment is deprecated and has no effect' + + it_behaves_like 'has_warning', pp, w + end + + context 'concat::fragment backup parameter' do + pp = <<-EOS + concat { '#{basedir}/file': } + concat::fragment { 'foo': + target => '#{basedir}/file', + content => 'bar', + backup => 'bar', + } + EOS + w = 'The $backup parameter to concat::fragment is deprecated and has no effect' + + it_behaves_like 'has_warning', pp, w + end + + context 'include concat::setup' do + pp = <<-EOS + include concat::setup + EOS + w = 'concat::setup is deprecated as a public API of the concat module and should no longer be directly included in the manifest.' + + it_behaves_like 'has_warning', pp, w + end + +end diff --git a/spec/acceptance/empty_spec.rb b/spec/acceptance/empty_spec.rb new file mode 100644 index 00000000..4ab6e1e6 --- /dev/null +++ b/spec/acceptance/empty_spec.rb @@ -0,0 +1,23 @@ +require 'spec_helper_acceptance' + +describe 'concat force empty parameter' do + basedir = default.tmpdir('concat') + context 'should run successfully' do + pp = <<-EOS + concat { '#{basedir}/file': + mode => '0644', + force => true, + } + EOS + + it 'applies the manifest twice with no stderr' do + apply_manifest(pp, :catch_failures => true) + apply_manifest(pp, :catch_changes => true) + end + + describe file("#{basedir}/file") do + it { should be_file } + its(:content) { should_not match /1\n2/ } + end + end +end diff --git a/spec/acceptance/fragment_source_spec.rb b/spec/acceptance/fragment_source_spec.rb index b208979a..a174e02b 100644 --- a/spec/acceptance/fragment_source_spec.rb +++ b/spec/acceptance/fragment_source_spec.rb @@ -145,10 +145,11 @@ EOS it 'applies the manifest with resource failures' do - expect(apply_manifest(pp, :catch_failures => true).stderr).to match(/Failed to generate additional resources using 'eval_generate'/) + apply_manifest(pp, :expect_failures => true) end describe file("#{basedir}/fail_no_source") do #FIXME: Serverspec::Type::File doesn't support exists? for some reason. so... hack. + it { should_not be_file } it { should_not be_directory } end end diff --git a/spec/acceptance/nodesets/default.yml b/spec/acceptance/nodesets/default.yml index 4e2cb809..b8268515 100644 --- a/spec/acceptance/nodesets/default.yml +++ b/spec/acceptance/nodesets/default.yml @@ -1,10 +1,10 @@ HOSTS: - centos-65-x64: + centos-66-x64: roles: - master platform: el-6-x86_64 - box : centos-65-x64-vbox436-nocm - box_url : http://puppet-vagrant-boxes.puppetlabs.com/centos-65-x64-virtualbox-nocm.box + box : puppetlabs/centos-6.6-64-nocm + box_url : http://puppet-vagrant-boxes.puppetlabs.com/centos-6.6-64-nocm hypervisor : vagrant CONFIG: - type: foss + type: git diff --git a/spec/acceptance/order_spec.rb b/spec/acceptance/order_spec.rb index b746b1c9..c158dd88 100644 --- a/spec/acceptance/order_spec.rb +++ b/spec/acceptance/order_spec.rb @@ -37,7 +37,7 @@ end describe 'alpha' do - it_behaves_like 'sortby', 'alpha', /string1string10string2/ + it_behaves_like 'sortby', 'alpha', /string10string1string2/ end describe 'numeric' do @@ -83,7 +83,7 @@ end end describe 'alpha' do - it_should_behave_like 'order_by', 'alpha', /string3string2string1/ + it_should_behave_like 'order_by', 'alpha', /string2string1string3/ end describe 'numeric' do it_should_behave_like 'order_by', 'numeric', /string3string2string1/ diff --git a/spec/acceptance/replace_spec.rb b/spec/acceptance/replace_spec.rb index 68d7383f..bd597ed1 100644 --- a/spec/acceptance/replace_spec.rb +++ b/spec/acceptance/replace_spec.rb @@ -226,7 +226,7 @@ end end - # XXX + # XXX concat's force param currently enables the creation of empty files # when there are no fragments, and the replace param will only replace # files and symlinks, not directories. The semantics either need to be # changed, extended, or a new param introduced to control directory @@ -234,6 +234,7 @@ context 'should succeed', :pending => 'not yet implemented' do pp = <<-EOS concat { '#{basedir}/file': + force => true, } concat::fragment { '1': diff --git a/spec/acceptance/warn_header_spec.rb b/spec/acceptance/warn_spec.rb similarity index 98% rename from spec/acceptance/warn_header_spec.rb rename to spec/acceptance/warn_spec.rb index b73414e3..2788607c 100644 --- a/spec/acceptance/warn_header_spec.rb +++ b/spec/acceptance/warn_spec.rb @@ -71,7 +71,7 @@ context '# foo should overide default warning message' do pp = <<-EOS concat { '#{basedir}/file': - warn => "# foo\n", + warn => '# foo', } concat::fragment { '1': diff --git a/spec/acceptance/warnings_spec.rb b/spec/acceptance/warnings_spec.rb deleted file mode 100644 index 89b3c3c5..00000000 --- a/spec/acceptance/warnings_spec.rb +++ /dev/null @@ -1,61 +0,0 @@ -require 'spec_helper_acceptance' - -describe 'warnings' do - basedir = default.tmpdir('concat') - - shared_examples 'has_warning' do |pp, w| - it 'applies the manifest twice with a stderr regex' do - expect(apply_manifest(pp, :catch_failures => true).stderr).to match(/#{Regexp.escape(w)}/m) - expect(apply_manifest(pp, :catch_changes => true).stderr).to match(/#{Regexp.escape(w)}/m) - end - end - - context 'concat force parameter deprecation' do - pp = <<-EOS - concat { '#{basedir}/file': - force => false, - } - concat::fragment { 'foo': - target => '#{basedir}/file', - content => 'bar', - } - EOS - w = 'The $force parameter to concat is deprecated and has no effect.' - - it_behaves_like 'has_warning', pp, w - end - - context 'concat::fragment ensure parameter deprecation' do - context 'target file exists' do - pp = <<-EOS - concat { '#{basedir}/file': - } - concat::fragment { 'foo': - target => '#{basedir}/file', - ensure => false, - content => 'bar', - } - EOS - w = 'The $ensure parameter to concat::fragment is deprecated and has no effect.' - - it_behaves_like 'has_warning', pp, w - end - end - - context 'concat::fragment target not found' do - context 'target not found' do - pp = <<-EOS - concat { 'file': - path => '#{basedir}/file', - } - concat::fragment { 'foo': - target => '#{basedir}/file', - content => 'bar', - } - EOS - w = 'not found in the catalog' - - it_behaves_like 'has_warning', pp, w - end - end -end diff --git a/spec/spec_helper_acceptance.rb b/spec/spec_helper_acceptance.rb index 1293aeb9..da994f8d 100644 --- a/spec/spec_helper_acceptance.rb +++ b/spec/spec_helper_acceptance.rb @@ -5,10 +5,7 @@ unless ENV['RS_PROVISION'] == 'no' or ENV['BEAKER_provision'] == 'no' # This will install the latest available package on el and deb based # systems fail on windows and osx, and install via gem on other *nixes - foss_opts = { - :default_action => 'gem_install', - :version => (ENV['PUPPET_VERSION'] || '3.8.1'), - } + foss_opts = {:default_action => 'gem_install'} if default.is_pe?; then install_pe; @@ -19,14 +16,14 @@ hosts.each do |host| on hosts, "mkdir -p #{host['distmoduledir']}" if host['platform'] =~ /sles-1/i || host['platform'] =~ /solaris-1/i - get_deps = <<-EOS + get_stdlib = <<-EOS package{'wget':} - exec{'download-stdlib': + exec{'download': command => "wget -P /root/ https://forgeapi.puppetlabs.com/v3/files/puppetlabs-stdlib-4.5.1.tar.gz --no-check-certificate", path => ['/opt/csw/bin/','/usr/bin/'] } EOS - apply_manifest_on(host, get_deps) + apply_manifest_on(host, get_stdlib) # have to use force otherwise it checks ssl cert even though it is a local file on host, puppet('module install /root/puppetlabs-stdlib-4.5.1.tar.gz --force --ignore-dependencies'), {:acceptable_exit_codes => [0, 1]} elsif host['platform'] =~ /windows/i diff --git a/spec/unit/classes/concat_setup_spec.rb b/spec/unit/classes/concat_setup_spec.rb new file mode 100644 index 00000000..c6ff93e0 --- /dev/null +++ b/spec/unit/classes/concat_setup_spec.rb @@ -0,0 +1,98 @@ +require 'spec_helper' + +describe 'concat::setup', :type => :class do + + shared_examples 'setup' do |concatdir| + concatdir = '/foo' if concatdir.nil? + + let(:facts) do + { + :concat_basedir => concatdir, + :caller_module_name => 'Test', + :osfamily => 'Debian', + :id => 'root', + :is_pe => false, + } + end + + it do + should contain_file("#{concatdir}/bin/concatfragments.rb").with({ + :mode => '0755', + :owner => 'root', + :group => 0, + :source => 'puppet:///modules/concat/concatfragments.rb', + }) + end + + [concatdir, "#{concatdir}/bin"].each do |file| + it do + should contain_file(file).with({ + :ensure => 'directory', + :mode => '0755', + :owner => 'root', + :group => 0, + }) + end + end + end + + context 'facts' do + context 'concat_basedir =>' do + context '/foo' do + it_behaves_like 'setup', '/foo' + end + end + end # facts + + context 'deprecated as a public class' do + it 'should create a warning' do + skip('rspec-puppet support for testing warning()') + end + end + + context "on osfamily Solaris" do + concatdir = '/foo' + let(:facts) do + { + :concat_basedir => concatdir, + :caller_module_name => 'Test', + :osfamily => 'Solaris', + :id => 'root', + :is_pe => false, + } + end + + it do + should contain_file("#{concatdir}/bin/concatfragments.rb").with({ + :ensure => 'file', + :owner => 'root', + :group => 0, + :mode => '0755', + :source => 'puppet:///modules/concat/concatfragments.rb', + }) + end + end # on osfamily Solaris + + context "on osfamily windows" do + concatdir = '/foo' + let(:facts) do + { + :concat_basedir => concatdir, + :caller_module_name => 'Test', + :osfamily => 'windows', + :id => 'batman', + :is_pe => false, + } + end + + it do + should contain_file("#{concatdir}/bin/concatfragments.rb").with({ + :ensure => 'file', + :owner => nil, + :group => nil, + :mode => nil, + :source => 'puppet:///modules/concat/concatfragments.rb', + }) + end + end # on osfamily windows +end diff --git a/spec/unit/defines/concat_fragment_spec.rb b/spec/unit/defines/concat_fragment_spec.rb index 317b02cd..6cf3e429 100644 --- a/spec/unit/defines/concat_fragment_spec.rb +++ b/spec/unit/defines/concat_fragment_spec.rb @@ -9,21 +9,50 @@ :content => nil, :source => nil, :order => 10, + :ensure => 'present', }.merge(params) + safe_name = title.gsub(/[\/\n]/, '_') + safe_target_name = p[:target].gsub(/[\/\n]/, '_') + concatdir = '/var/lib/puppet/concat' + fragdir = "#{concatdir}/#{safe_target_name}" id = 'root' gid = 'root' + if p[:ensure] == 'absent' + safe_ensure = p[:ensure] + else + safe_ensure = 'file' + end let(:title) { title } + let(:facts) do + { + :concat_basedir => concatdir, + :id => id, + :gid => gid, + :osfamily => 'Debian', + :path => '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', + :is_pe => false, + } + end let(:params) { params } let(:pre_condition) do "concat{ '#{p[:target]}': }" end it do + should contain_class('concat::setup') should contain_concat(p[:target]) - should contain_concat_file(p[:target]) - should contain_concat_fragment(title) + should contain_file("#{fragdir}/fragments/#{p[:order]}_#{safe_name}").with({ + :ensure => safe_ensure, + :owner => id, + :group => gid, + :mode => '0640', + :source => p[:source], + :content => p[:content], + :alias => "concat_fragment_#{title}", + :backup => 'puppet', + }) end end @@ -31,7 +60,6 @@ ['0', '1', 'a', 'z'].each do |title| it_behaves_like 'fragment', title, { :target => '/etc/motd', - :content => "content for #{title}" } end end # title @@ -41,21 +69,42 @@ context target do it_behaves_like 'fragment', target, { :target => '/etc/motd', - :content => "content for #{target}" } end end context 'false' do let(:title) { 'motd_header' } + let(:facts) {{ :concat_basedir => '/tmp', :is_pe => false }} let(:params) {{ :target => false }} it 'should fail' do - expect { catalogue }.to raise_error(Puppet::Error, /is not a string/) + expect { should }.to raise_error(Puppet::Error, /is not a string/) end end end # target => + context 'ensure =>' do + ['present', 'absent'].each do |ens| + context ens do + it_behaves_like 'fragment', 'motd_header', { + :ensure => ens, + :target => '/etc/motd', + } + end + end + + context 'any value other than \'present\' or \'absent\'' do + let(:title) { 'motd_header' } + let(:facts) {{ :concat_basedir => '/tmp', :is_pe => false }} + let(:params) {{ :ensure => 'invalid', :target => '/etc/motd' }} + + it 'should create a warning' do + skip('rspec-puppet support for testing warning()') + end + end + end # ensure => + context 'content =>' do ['', 'ashp is our hero'].each do |content| context content do @@ -68,10 +117,11 @@ context 'false' do let(:title) { 'motd_header' } + let(:facts) {{ :concat_basedir => '/tmp', :is_pe => false }} let(:params) {{ :content => false, :target => '/etc/motd' }} it 'should fail' do - expect { catalogue }.to raise_error(Puppet::Error, /is not a string/) + expect { should }.to raise_error(Puppet::Error, /is not a string/) end end end # content => @@ -88,28 +138,193 @@ context 'false' do let(:title) { 'motd_header' } + let(:facts) {{ :concat_basedir => '/tmp', :is_pe => false }} let(:params) {{ :source => false, :target => '/etc/motd' }} it 'should fail' do - expect { catalogue }.to raise_error(Puppet::Error, /is not a string or an Array/) + expect { should }.to raise_error(Puppet::Error, /is not a string or an Array/) end end end # source => + context 'order =>' do + ['', '42', 'a', 'z'].each do |order| + context '\'\'' do + it_behaves_like 'fragment', 'motd_header', { + :order => order, + :target => '/etc/motd', + } + end + end + + context 'false' do + let(:title) { 'motd_header' } + let(:facts) {{ :concat_basedir => '/tmp', :is_pe => false }} + let(:params) {{ :order => false, :target => '/etc/motd' }} + + it 'should fail' do + expect { should }.to raise_error(Puppet::Error, /is not a string or integer/) + end + end + + context '123:456' do + let(:title) { 'motd_header' } + let(:facts) {{ :concat_basedir => '/tmp', :is_pe => false }} + let(:params) {{ :order => '123:456', :target => '/etc/motd' }} + + it 'should fail' do + expect { should }.to raise_error(Puppet::Error, /cannot contain/) + end + end + context '123/456' do + let(:title) { 'motd_header' } + let(:facts) {{ :concat_basedir => '/tmp', :is_pe => false }} + let(:params) {{ :order => '123/456', :target => '/etc/motd' }} + + it 'should fail' do + expect { should }.to raise_error(Puppet::Error, /cannot contain/) + end + end + context '123\n456' do + let(:title) { 'motd_header' } + let(:facts) {{ :concat_basedir => '/tmp', :is_pe => false }} + let(:params) {{ :order => "123\n456", :target => '/etc/motd' }} + + it 'should fail' do + expect { should }.to raise_error(Puppet::Error, /cannot contain/) + end + end + end # order => + context 'more than one content source' do + error_msg = 'You cannot specify more than one of $content, $source, $ensure => /target' + + context 'ensure => target and source' do + let(:title) { 'motd_header' } + let(:facts) do + { + :concat_basedir => '/tmp', + :osfamily => 'Debian', + :id => 'root', + :is_pe => false, + :gid => 'root', + } + end + let(:params) do + { + :target => '/etc/motd', + :ensure => '/foo', + :source => '/bar', + } + end + + it 'should fail' do + expect { should }.to raise_error(Puppet::Error, /#{Regexp.escape(error_msg)}/m) + end + end + + context 'ensure => target and content' do + let(:title) { 'motd_header' } + let(:facts) do + { + :concat_basedir => '/tmp', + :osfamily => 'Debian', + :id => 'root', + :is_pe => false, + :gid => 'root', + } + end + let(:params) do + { + :target => '/etc/motd', + :ensure => '/foo', + :content => 'bar', + } + end + + it 'should fail' do + expect { should }.to raise_error(Puppet::Error, /#{Regexp.escape(error_msg)}/m) + end + end + context 'source and content' do let(:title) { 'motd_header' } + let(:facts) do + { + :concat_basedir => '/tmp', + :osfamily => 'Debian', + :id => 'root', + :is_pe => false, + :gid => 'root', + } + end let(:params) do { - :target => '/etc/motd', + :target => '/etc/motd', :source => '/foo', :content => 'bar', } end it 'should fail' do - expect { catalogue }.to raise_error(Puppet::Error, /Can't use 'source' and 'content' at the same time/) + expect { should }.to raise_error(Puppet::Error, /#{Regexp.escape(error_msg)}/m) end end + end # more than one content source + + describe 'deprecated parameter' do + context 'mode =>' do + context '1755' do + it_behaves_like 'fragment', 'motd_header', { + :mode => '1755', + :target => '/etc/motd', + } + + it 'should create a warning' do + skip('rspec-puppet support for testing warning()') + end + end + end # mode => + + context 'owner =>' do + context 'apenny' do + it_behaves_like 'fragment', 'motd_header', { + :owner => 'apenny', + :target => '/etc/motd', + } + + it 'should create a warning' do + skip('rspec-puppet support for testing warning()') + end + end + end # owner => + + context 'group =>' do + context 'apenny' do + it_behaves_like 'fragment', 'motd_header', { + :group => 'apenny', + :target => '/etc/motd', + } + + it 'should create a warning' do + skip('rspec-puppet support for testing warning()') + end + end + end # group => + + context 'backup =>' do + context 'foo' do + it_behaves_like 'fragment', 'motd_header', { + :backup => 'foo', + :target => '/etc/motd', + } + + it 'should create a warning' do + skip('rspec-puppet support for testing warning()') + end + end + end # backup => + end # deprecated params + end diff --git a/spec/unit/defines/concat_spec.rb b/spec/unit/defines/concat_spec.rb index b2254f4e..115a0f55 100644 --- a/spec/unit/defines/concat_spec.rb +++ b/spec/unit/defines/concat_spec.rb @@ -14,13 +14,19 @@ :group => nil, :mode => '0644', :warn => false, + :force => false, :backup => 'puppet', :replace => true, + :order => 'alpha', + :ensure_newline => false, + :validate_cmd => nil, }.merge(params) safe_name = title.gsub('/', '_') + concatdir = '/var/lib/puppet/concat' + fragdir = "#{concatdir}/#{safe_name}" concat_name = 'fragments.concat.out' - default_warn_message = "# This file is managed by Puppet. DO NOT EDIT.\n" + default_warn_message = '# This file is managed by Puppet. DO NOT EDIT.' file_defaults = { :backup => p[:backup], @@ -30,6 +36,7 @@ let(:params) { params } let(:facts) do { + :concat_basedir => concatdir, :id => id, :osfamily => 'Debian', :path => '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', @@ -40,23 +47,117 @@ if p[:ensure] == 'present' it do - should contain_concat(title).with(file_defaults.merge({ + should contain_file(fragdir).with(file_defaults.merge({ + :ensure => 'directory', + :mode => '0750', + })) + end + + it do + should contain_file("#{fragdir}/fragments").with(file_defaults.merge({ + :ensure => 'directory', + :mode => '0750', + :force => true, + :ignore => ['.svn', '.git', '.gitignore'], + :purge => true, + :recurse => true, + })) + end + + [ + "#{fragdir}/fragments.concat", + "#{fragdir}/#{concat_name}", + ].each do |file| + it do + should contain_file(file).with(file_defaults.merge({ + :ensure => 'present', + :mode => '0640', + })) + end + end + + it do + should contain_file(title).with(file_defaults.merge({ :ensure => 'present', :owner => p[:owner], :group => p[:group], :mode => p[:mode], + :replace => p[:replace], :path => p[:path], + :alias => "concat_#{title}", + :source => "#{fragdir}/#{concat_name}", + :validate_cmd => p[:validate_cmd], :backup => p[:backup], - :replace => p[:replace], })) end + + cmd = "#{concatdir}/bin/concatfragments.rb " + + "-o \"#{concatdir}/#{safe_name}/fragments.concat.out\" " + + "-d \"#{concatdir}/#{safe_name}\"" + + # flag order: fragdir, warnflag, forceflag, orderflag, newlineflag + if p.has_key?(:warn) + case p[:warn] + when TrueClass + message = default_warn_message + when 'true', 'yes', 'on' + # should generate a stringified boolean warning + message = default_warn_message + when FalseClass + message = nil + when 'false', 'no', 'off' + # should generate a stringified boolean warning + message = nil + else + message = p[:warn] + end + + unless message.nil? + cmd += " -w \'#{message}\'" + end + end + + cmd += " -f" if p[:force] + cmd += " -n" if p[:order] == 'numeric' + cmd += " -l" if p[:ensure_newline] == true + + it do + should contain_exec("concat_#{title}").with({ + :alias => "concat_#{fragdir}", + :command => cmd, + :unless => "#{cmd} -t", + }) + end else + [ + fragdir, + "#{fragdir}/fragments", + "#{fragdir}/fragments.concat", + "#{fragdir}/#{concat_name}", + ].each do |file| + it do + should contain_file(file).with(file_defaults.merge({ + :ensure => 'absent', + :force => true, + })) + end + end + it do - should contain_concat(title).with(file_defaults.merge({ + should contain_file(title).with(file_defaults.merge({ :ensure => 'absent', :backup => p[:backup], })) end + + it do + should contain_exec("concat_#{title}").with({ + :alias => "concat_#{fragdir}", + :command => 'true', + :unless => 'true', + :path => '/bin:/usr/bin', + }) + end end end @@ -74,14 +175,14 @@ context title do let(:title) { title } it 'should fail' do - expect { catalogue }.to raise_error(Puppet::Error, /is not an absolute path/) + expect { should }.to raise_error(Puppet::Error, /is not an absolute path/) end end end end context 'with path param' do - ['/foo', 'foo', 'foo/bar'].each do |title| + ['./foo', 'foo', 'foo/bar'].each do |title| context title do it_behaves_like 'concat', title, { :path => '/etc/foo.bar' } end @@ -104,7 +205,7 @@ let(:title) { '/etc/foo.bar' } let(:params) {{ :ensure => 'invalid' }} it 'should fail' do - expect { catalogue }.to raise_error(Puppet::Error, /#{Regexp.escape('does not match "^present$|^absent$"')}/) + expect { should }.to raise_error(Puppet::Error, /#{Regexp.escape('does not match "^present$|^absent$"')}/) end end end # ensure => @@ -119,7 +220,7 @@ let(:title) { '/etc/foo.bar' } let(:params) {{ :path => path }} it 'should fail' do - expect { catalogue }.to raise_error(Puppet::Error, /is not an absolute path/) + expect { should }.to raise_error(Puppet::Error, /is not an absolute path/) end end end @@ -134,7 +235,7 @@ let(:title) { '/etc/foo.bar' } let(:params) {{ :owner => false }} it 'should fail' do - expect { catalogue }.to raise_error(Puppet::Error, /is not a string/) + expect { should }.to raise_error(Puppet::Error, /is not a string/) end end end # owner => @@ -148,7 +249,7 @@ let(:title) { '/etc/foo.bar' } let(:params) {{ :group => false }} it 'should fail' do - expect { catalogue }.to raise_error(Puppet::Error, /is not a string/) + expect { should }.to raise_error(Puppet::Error, /is not a string/) end end end # group => @@ -162,7 +263,7 @@ let(:title) { '/etc/foo.bar' } let(:params) {{ :mode => false }} it 'should fail' do - expect { catalogue }.to raise_error(Puppet::Error, /is not a string/) + expect { should }.to raise_error(Puppet::Error, /is not a string/) end end end # mode => @@ -190,11 +291,27 @@ let(:title) { '/etc/foo.bar' } let(:params) {{ :warn => 123 }} it 'should fail' do - expect { catalogue }.to raise_error(Puppet::Error, /is not a string or boolean/) + expect { should }.to raise_error(Puppet::Error, /is not a string or boolean/) end end end # warn => + context 'force =>' do + [true, false].each do |force| + context force do + it_behaves_like 'concat', '/etc/foo.bar', { :force => force } + end + end + + context '123' do + let(:title) { '/etc/foo.bar' } + let(:params) {{ :force => 123 }} + it 'should fail' do + expect { should }.to raise_error(Puppet::Error, /is not a boolean/) + end + end + end # force => + context 'backup =>' do context 'reverse' do it_behaves_like 'concat', '/etc/foo.bar', { :backup => 'reverse' } @@ -212,7 +329,7 @@ let(:title) { '/etc/foo.bar' } let(:params) {{ :backup => [] }} it 'should fail' do - expect { catalogue }.to raise_error(Puppet::Error, /backup must be string or bool/) + expect { should }.to raise_error(Puppet::Error, /backup must be string or bool/) end end end # backup => @@ -228,7 +345,7 @@ let(:title) { '/etc/foo.bar' } let(:params) {{ :replace => 123 }} it 'should fail' do - expect { catalogue }.to raise_error(Puppet::Error, /is not a boolean/) + expect { should }.to raise_error(Puppet::Error, /is not a boolean/) end end end # replace => @@ -244,7 +361,7 @@ let(:title) { '/etc/foo.bar' } let(:params) {{ :order => 'invalid' }} it 'should fail' do - expect { catalogue }.to raise_error(Puppet::Error, /#{Regexp.escape('does not match "^alpha$|^numeric$"')}/) + expect { should }.to raise_error(Puppet::Error, /#{Regexp.escape('does not match "^alpha$|^numeric$"')}/) end end end # order => @@ -260,7 +377,7 @@ let(:title) { '/etc/foo.bar' } let(:params) {{ :ensure_newline => 123 }} it 'should fail' do - expect { catalogue }.to raise_error(Puppet::Error, /is not a boolean/) + expect { should }.to raise_error(Puppet::Error, /is not a boolean/) end end end # ensure_newline => @@ -275,9 +392,24 @@ let(:title) { '/etc/foo.bar' } let(:params) {{ :validate_cmd => cmd }} it 'should fail' do - expect { catalogue }.to raise_error(Puppet::Error, /\$validate_cmd must be a string/) + expect { should }.to raise_error(Puppet::Error, /\$validate_cmd must be a string/) end end end end # validate_cmd => + + describe 'deprecated parameter' do + context 'gnu =>' do + context 'foo' do + it_behaves_like 'concat', '/etc/foo.bar', { :gnu => 'foo'} + + it 'should create a warning' do + skip('rspec-puppet support for testing warning()') + end + end + end + end + end + +# vim:sw=2:ts=2:expandtab:textwidth=79 diff --git a/spec/unit/facts/concat_basedir_spec.rb b/spec/unit/facts/concat_basedir_spec.rb new file mode 100644 index 00000000..41bc90f1 --- /dev/null +++ b/spec/unit/facts/concat_basedir_spec.rb @@ -0,0 +1,18 @@ +require 'spec_helper' + +describe 'concat_basedir', :type => :fact do + before(:each) { Facter.clear } + + context 'Puppet[:vardir] ==' do + it '/var/lib/puppet' do + Puppet.stubs(:[]).with(:vardir).returns('/var/lib/puppet') + Facter.fact(:concat_basedir).value.should == '/var/lib/puppet/concat' + end + + it '/home/apenny/.puppet/var' do + Puppet.stubs(:[]).with(:vardir).returns('/home/apenny/.puppet/var') + Facter.fact(:concat_basedir).value.should == '/home/apenny/.puppet/var/concat' + end + end + +end diff --git a/tests/init.pp b/tests/init.pp index c7f19346..fd214271 100644 --- a/tests/init.pp +++ b/tests/init.pp @@ -1,5 +1,6 @@ concat { '/tmp/concat': ensure => present, + force => true, owner => 'root', group => 'root', mode => '0644', From 6ad7f74b28ea43350855bd8ed6633547efe6ad0c Mon Sep 17 00:00:00 2001 From: Colleen Murphy Date: Mon, 23 Mar 2015 21:12:37 -0700 Subject: [PATCH 50/76] Update spec tests for rspec-puppet 2.0 Update tests for rspec-puppet 2.0 This patch changes `should` to `catalogue` for raise_error matchers since `should` has the same problems as `subject`, where it is not evaluated the way it used to be. We also unpin rspec-puppet. Guard validate_cmd test for older puppet versions Test group membership only on newer puppet The defined() function doesn't seem to work properly with puppet 3.4 and rspec. defined() works on its own, rspec works on its own, but together they determine that $gid is not defined and cause errors here. Work around it by ignoring this check for older puppet version. --- manifests/init.pp | 6 ++++ spec/unit/defines/concat_fragment_spec.rb | 30 ++++++++++------ spec/unit/defines/concat_spec.rb | 44 ++++++++++++----------- 3 files changed, 48 insertions(+), 32 deletions(-) diff --git a/manifests/init.pp b/manifests/init.pp index 077a29bd..b161e00f 100644 --- a/manifests/init.pp +++ b/manifests/init.pp @@ -104,14 +104,18 @@ true: { $warn_message = $default_warn_message } + # lint:ignore:quoted_booleans 'true', 'yes', 'on': { + # lint:endignore warning($bool_warn_message) $warn_message = $default_warn_message } false: { $warn_message = '' } + # lint:ignore:quoted_booleans 'false', 'no', 'off': { + # lint:endignore warning($bool_warn_message) $warn_message = '' } @@ -246,7 +250,9 @@ # lint:ignore:quoted_booleans $absent_exec_command = $::kernel ? { 'windows' => 'cmd.exe /c exit 0', + # lint:ignore:quoted_booleans default => 'true', + # lint:endignore } # lint:endignore diff --git a/spec/unit/defines/concat_fragment_spec.rb b/spec/unit/defines/concat_fragment_spec.rb index 6cf3e429..40866a86 100644 --- a/spec/unit/defines/concat_fragment_spec.rb +++ b/spec/unit/defines/concat_fragment_spec.rb @@ -46,13 +46,21 @@ should contain_file("#{fragdir}/fragments/#{p[:order]}_#{safe_name}").with({ :ensure => safe_ensure, :owner => id, - :group => gid, :mode => '0640', :source => p[:source], :content => p[:content], :alias => "concat_fragment_#{title}", :backup => 'puppet', }) + # The defined() function doesn't seem to work properly with puppet 3.4 and rspec. + # defined() works on its own, rspec works on its own, but together they + # determine that $gid is not defined and cause errors here. Work around + # it by ignoring this check for older puppet version. + if Puppet::Util::Package.versioncmp(Puppet.version, '3.5.0') >= 0 + should contain_file("#{fragdir}/fragments/#{p[:order]}_#{safe_name}").with({ + :group => gid, + }) + end end end @@ -79,7 +87,7 @@ let(:params) {{ :target => false }} it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /is not a string/) + expect { catalogue }.to raise_error(Puppet::Error, /is not a string/) end end end # target => @@ -121,7 +129,7 @@ let(:params) {{ :content => false, :target => '/etc/motd' }} it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /is not a string/) + expect { catalogue }.to raise_error(Puppet::Error, /is not a string/) end end end # content => @@ -142,7 +150,7 @@ let(:params) {{ :source => false, :target => '/etc/motd' }} it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /is not a string or an Array/) + expect { catalogue }.to raise_error(Puppet::Error, /is not a string or an Array/) end end end # source => @@ -163,7 +171,7 @@ let(:params) {{ :order => false, :target => '/etc/motd' }} it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /is not a string or integer/) + expect { catalogue }.to raise_error(Puppet::Error, /is not a string or integer/) end end @@ -173,7 +181,7 @@ let(:params) {{ :order => '123:456', :target => '/etc/motd' }} it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /cannot contain/) + expect { catalogue }.to raise_error(Puppet::Error, /cannot contain/) end end context '123/456' do @@ -182,7 +190,7 @@ let(:params) {{ :order => '123/456', :target => '/etc/motd' }} it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /cannot contain/) + expect { catalogue }.to raise_error(Puppet::Error, /cannot contain/) end end context '123\n456' do @@ -191,7 +199,7 @@ let(:params) {{ :order => "123\n456", :target => '/etc/motd' }} it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /cannot contain/) + expect { catalogue }.to raise_error(Puppet::Error, /cannot contain/) end end end # order => @@ -219,7 +227,7 @@ end it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /#{Regexp.escape(error_msg)}/m) + expect { catalogue }.to raise_error(Puppet::Error, /#{Regexp.escape(error_msg)}/m) end end @@ -243,7 +251,7 @@ end it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /#{Regexp.escape(error_msg)}/m) + expect { catalogue }.to raise_error(Puppet::Error, /#{Regexp.escape(error_msg)}/m) end end @@ -267,7 +275,7 @@ end it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /#{Regexp.escape(error_msg)}/m) + expect { catalogue }.to raise_error(Puppet::Error, /#{Regexp.escape(error_msg)}/m) end end diff --git a/spec/unit/defines/concat_spec.rb b/spec/unit/defines/concat_spec.rb index 115a0f55..b5dd2530 100644 --- a/spec/unit/defines/concat_spec.rb +++ b/spec/unit/defines/concat_spec.rb @@ -175,7 +175,7 @@ context title do let(:title) { title } it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /is not an absolute path/) + expect { catalogue }.to raise_error(Puppet::Error, /is not an absolute path/) end end end @@ -205,7 +205,7 @@ let(:title) { '/etc/foo.bar' } let(:params) {{ :ensure => 'invalid' }} it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /#{Regexp.escape('does not match "^present$|^absent$"')}/) + expect { catalogue }.to raise_error(Puppet::Error, /#{Regexp.escape('does not match "^present$|^absent$"')}/) end end end # ensure => @@ -220,7 +220,7 @@ let(:title) { '/etc/foo.bar' } let(:params) {{ :path => path }} it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /is not an absolute path/) + expect { catalogue }.to raise_error(Puppet::Error, /is not an absolute path/) end end end @@ -235,7 +235,7 @@ let(:title) { '/etc/foo.bar' } let(:params) {{ :owner => false }} it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /is not a string/) + expect { catalogue }.to raise_error(Puppet::Error, /is not a string/) end end end # owner => @@ -249,7 +249,7 @@ let(:title) { '/etc/foo.bar' } let(:params) {{ :group => false }} it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /is not a string/) + expect { catalogue }.to raise_error(Puppet::Error, /is not a string/) end end end # group => @@ -263,7 +263,7 @@ let(:title) { '/etc/foo.bar' } let(:params) {{ :mode => false }} it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /is not a string/) + expect { catalogue }.to raise_error(Puppet::Error, /is not a string/) end end end # mode => @@ -291,7 +291,7 @@ let(:title) { '/etc/foo.bar' } let(:params) {{ :warn => 123 }} it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /is not a string or boolean/) + expect { catalogue }.to raise_error(Puppet::Error, /is not a string or boolean/) end end end # warn => @@ -307,7 +307,7 @@ let(:title) { '/etc/foo.bar' } let(:params) {{ :force => 123 }} it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /is not a boolean/) + expect { catalogue }.to raise_error(Puppet::Error, /is not a boolean/) end end end # force => @@ -329,7 +329,7 @@ let(:title) { '/etc/foo.bar' } let(:params) {{ :backup => [] }} it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /backup must be string or bool/) + expect { catalogue }.to raise_error(Puppet::Error, /backup must be string or bool/) end end end # backup => @@ -345,7 +345,7 @@ let(:title) { '/etc/foo.bar' } let(:params) {{ :replace => 123 }} it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /is not a boolean/) + expect { catalogue }.to raise_error(Puppet::Error, /is not a boolean/) end end end # replace => @@ -361,7 +361,7 @@ let(:title) { '/etc/foo.bar' } let(:params) {{ :order => 'invalid' }} it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /#{Regexp.escape('does not match "^alpha$|^numeric$"')}/) + expect { catalogue }.to raise_error(Puppet::Error, /#{Regexp.escape('does not match "^alpha$|^numeric$"')}/) end end end # order => @@ -377,22 +377,24 @@ let(:title) { '/etc/foo.bar' } let(:params) {{ :ensure_newline => 123 }} it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /is not a boolean/) + expect { catalogue }.to raise_error(Puppet::Error, /is not a boolean/) end end end # ensure_newline => context 'validate_cmd =>' do - context '/usr/bin/test -e %' do - it_behaves_like 'concat', '/etc/foo.bar', { :validate_cmd => '/usr/bin/test -e %' } - end + if Puppet::Util::Package::versioncmp(Puppet::version, '3.5.0') > 0 + context '/usr/bin/test -e %' do + it_behaves_like 'concat', '/etc/foo.bar', { :validate_cmd => '/usr/bin/test -e %' } + end - [ 1234, true ].each do |cmd| - context cmd do - let(:title) { '/etc/foo.bar' } - let(:params) {{ :validate_cmd => cmd }} - it 'should fail' do - expect { should }.to raise_error(Puppet::Error, /\$validate_cmd must be a string/) + [ 1234, true ].each do |cmd| + context cmd do + let(:title) { '/etc/foo.bar' } + let(:params) {{ :validate_cmd => cmd }} + it 'should fail' do + expect { catalogue }.to raise_error(Puppet::Error, /\$validate_cmd must be a string/) + end end end end From 52cb97c0896bf986b4115b2f884fd8a8e921b8fa Mon Sep 17 00:00:00 2001 From: Bryan Jen Date: Fri, 12 Jun 2015 11:21:24 -0700 Subject: [PATCH 51/76] Update CHANGELOG with yanked 2.x releases --- CHANGELOG.md | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 2e606413..4e9bdb3c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,27 @@ +======= +##2015-06-02 - Supported Release 2.0.1 [YANKED] +###Summary + +This is a bugfix release. + +####Bugfixes +- Fixes dependency graphing with concurrent modification of the same file. +- Fixes handling fragment target. +- Fixes the defaulted force behavior to handle empty concats correctly. + +======= +##2015-05-12 - Supported Release 2.0.0 [YANKED] +###Summary + +This is a major release. Includes re-implementation of concat to use native Type and Providers, resulting in significantly improved performance and testability. Also includes a bugfix to alpha ordering of fragments. + +####Features +- Re-implementation of concat to use native Type and Providers. + +####Bugfixes +- Fixes a bug in alpha ordering of fragments. + +======= ##2015-06-02 - Supported Release 1.2.3 ###Summary From 17b836bb21215682d5bdcdd870c18830655764df Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Rapha=C3=ABl=20Pinson?= Date: Wed, 10 Jun 2015 10:01:09 +0200 Subject: [PATCH 52/76] Compare $::is_pe as bool, not string stdlib now provides an $::is_pe fact, which defaults to "false". When compared as a string, it is evaluated to true. --- manifests/init.pp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/manifests/init.pp b/manifests/init.pp index b161e00f..855be23f 100644 --- a/manifests/init.pp +++ b/manifests/init.pp @@ -205,7 +205,7 @@ $command = strip(regsubst("${script_command} -o \"${fragdir}/${concat_name}\" -d \"${fragdir}\" ${warnflag} ${forceflag} ${orderflag} ${newlineflag}", '\s+', ' ', 'G')) # make sure ruby is in the path for PE - if defined('$is_pe') and $::is_pe { + if defined('$is_pe') and str2bool("${::is_pe}") { # lint:ignore:only_variable_string if $::kernel == 'windows' { $command_path = "${::env_windows_installdir}/bin:${::path}" } else { From 5bc2d58394ae5c3306552f00400716ad4a529a8a Mon Sep 17 00:00:00 2001 From: Drew Fisher Date: Sat, 20 Jun 2015 08:48:50 -0600 Subject: [PATCH 53/76] Add support for Solaris 12 --- README.md | 2 +- metadata.json | 3 ++- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 69b9a898..afec06dc 100644 --- a/README.md +++ b/README.md @@ -268,7 +268,7 @@ The `concat::setup` class should no longer be directly included in the manifest. ##Limitations -This module has been tested on [all PE-supported platforms](https://forge.puppetlabs.com/supported#compat-matrix), and no issues have been identified. +This module has been tested on [all PE-supported platforms](https://forge.puppetlabs.com/supported#compat-matrix), and no issues have been identified. Additionally, it is tested (but not supported) on Solaris 12. ##Development diff --git a/metadata.json b/metadata.json index 1d459926..6f6adab6 100644 --- a/metadata.json +++ b/metadata.json @@ -67,7 +67,8 @@ "operatingsystem": "Solaris", "operatingsystemrelease": [ "10", - "11" + "11", + "12" ] }, { From 7473415ea362b639f528ffa0caac858289d091e8 Mon Sep 17 00:00:00 2001 From: Hunter Haugen Date: Thu, 11 Jun 2015 15:28:34 -0700 Subject: [PATCH 54/76] Add helper to install puppet/pe/puppet-agent --- Gemfile | 1 + spec/spec_helper_acceptance.rb | 14 +++----------- 2 files changed, 4 insertions(+), 11 deletions(-) diff --git a/Gemfile b/Gemfile index 2b1b7cd8..bfe64b18 100644 --- a/Gemfile +++ b/Gemfile @@ -28,6 +28,7 @@ group :system_tests do gem 'beaker-rspec', :require => false end gem 'serverspec', :require => false + gem 'beaker-puppet_install_helper', :require => false end diff --git a/spec/spec_helper_acceptance.rb b/spec/spec_helper_acceptance.rb index da994f8d..80510464 100644 --- a/spec/spec_helper_acceptance.rb +++ b/spec/spec_helper_acceptance.rb @@ -1,20 +1,12 @@ require 'beaker-rspec/spec_helper' require 'beaker-rspec/helpers/serverspec' require 'acceptance/specinfra_stubs' +require 'beaker/puppet_install_helper' -unless ENV['RS_PROVISION'] == 'no' or ENV['BEAKER_provision'] == 'no' - # This will install the latest available package on el and deb based - # systems fail on windows and osx, and install via gem on other *nixes - foss_opts = {:default_action => 'gem_install'} - - if default.is_pe?; then - install_pe; - else - install_puppet(foss_opts); - end +run_puppet_install_helper +unless ENV['RS_PROVISION'] == 'no' or ENV['BEAKER_provision'] == 'no' hosts.each do |host| - on hosts, "mkdir -p #{host['distmoduledir']}" if host['platform'] =~ /sles-1/i || host['platform'] =~ /solaris-1/i get_stdlib = <<-EOS package{'wget':} From f6545611739a8b734b11b1732c6eb4956fb805dd Mon Sep 17 00:00:00 2001 From: Morgan Haskel Date: Mon, 22 Jun 2015 16:33:07 -0700 Subject: [PATCH 55/76] Default test vardir to the aio paths for *nix systems --- spec/acceptance/concat_spec.rb | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/spec/acceptance/concat_spec.rb b/spec/acceptance/concat_spec.rb index c5c97277..ad959cb3 100644 --- a/spec/acceptance/concat_spec.rb +++ b/spec/acceptance/concat_spec.rb @@ -6,11 +6,17 @@ groupname = 'system' scriptname = 'concatfragments.rb' vardir = default['puppetvardir'] + if vardir == '' + vardir = '/opt/puppetlabs/puppet/cache' + end when 'Darwin' username = 'root' groupname = 'wheel' scriptname = 'concatfragments.rb' vardir = default['puppetvardir'] + if vardir == '' + vardir = '/opt/puppetlabs/puppet/cache' + end when 'windows' username = 'Administrator' groupname = 'Administrators' @@ -22,11 +28,17 @@ groupname = 'root' scriptname = 'concatfragments.rb' vardir = default['puppetvardir'] + if vardir == '' + vardir = '/opt/puppetlabs/puppet/cache' + end else username = 'root' groupname = 'root' scriptname = 'concatfragments.rb' vardir = default['puppetvardir'] + if vardir == '' + vardir = '/opt/puppetlabs/puppet/cache' + end end describe 'basic concat test' do From 967880fc4a96546cf5a766494a050b677ad5c0a4 Mon Sep 17 00:00:00 2001 From: Morgan Haskel Date: Mon, 22 Jun 2015 16:50:09 -0700 Subject: [PATCH 56/76] check for nil --- spec/acceptance/concat_spec.rb | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/spec/acceptance/concat_spec.rb b/spec/acceptance/concat_spec.rb index ad959cb3..2f7d4e3b 100644 --- a/spec/acceptance/concat_spec.rb +++ b/spec/acceptance/concat_spec.rb @@ -6,7 +6,7 @@ groupname = 'system' scriptname = 'concatfragments.rb' vardir = default['puppetvardir'] - if vardir == '' + if vardir.nil? or vardir == '' vardir = '/opt/puppetlabs/puppet/cache' end when 'Darwin' @@ -14,7 +14,7 @@ groupname = 'wheel' scriptname = 'concatfragments.rb' vardir = default['puppetvardir'] - if vardir == '' + if vardir.nil? or vardir == '' vardir = '/opt/puppetlabs/puppet/cache' end when 'windows' @@ -28,7 +28,7 @@ groupname = 'root' scriptname = 'concatfragments.rb' vardir = default['puppetvardir'] - if vardir == '' + if vardir.nil? or vardir == '' vardir = '/opt/puppetlabs/puppet/cache' end else @@ -36,7 +36,7 @@ groupname = 'root' scriptname = 'concatfragments.rb' vardir = default['puppetvardir'] - if vardir == '' + if vardir.nil? or vardir == '' vardir = '/opt/puppetlabs/puppet/cache' end end From 839de048afd495acb4ed2d198f86b4368c66d475 Mon Sep 17 00:00:00 2001 From: Garrett Wollman Date: Fri, 10 Jul 2015 15:22:48 -0400 Subject: [PATCH 57/76] (#2208) FreeBSD: call /usr/local/bin/ruby explicitly On FreeBSD systems, the Ruby interpreter is normally installed in /usr/local/bin, which is not on the default PATH when running init scripts. Most users will have it in their PATH when logged in, which causes concat to fail when run from init (or "service puppet start") but not when run from directly the command line ("sudo puppet agent"). This follows the same strategy as OpenBSD, but avoid specifying the Ruby interpreter version explicitly, because FreeBSD supports three different versions of Ruby (with the default always being installed as /usr/local/bin/ruby). --- manifests/setup.pp | 1 + 1 file changed, 1 insertion(+) diff --git a/manifests/setup.pp b/manifests/setup.pp index c5aedd82..6d5d8b2f 100644 --- a/manifests/setup.pp +++ b/manifests/setup.pp @@ -44,6 +44,7 @@ $script_command = $::osfamily? { 'windows' => "ruby.exe '${script_path}'", 'openbsd' => "/usr/local/bin/ruby21 '${script_path}'", + 'freebsd' => "/usr/local/bin/ruby '${script_path}'", default => $script_path } From a72be83df351021b11da7d2a34ff2ed0c0c7b98d Mon Sep 17 00:00:00 2001 From: Morgan Haskel Date: Mon, 13 Jul 2015 13:48:22 -0700 Subject: [PATCH 58/76] Use the correct vardir in the tests --- spec/acceptance/concat_spec.rb | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/spec/acceptance/concat_spec.rb b/spec/acceptance/concat_spec.rb index 2f7d4e3b..2bceefc6 100644 --- a/spec/acceptance/concat_spec.rb +++ b/spec/acceptance/concat_spec.rb @@ -5,7 +5,7 @@ username = 'root' groupname = 'system' scriptname = 'concatfragments.rb' - vardir = default['puppetvardir'] + vardir = default.puppet['vardir'] if vardir.nil? or vardir == '' vardir = '/opt/puppetlabs/puppet/cache' end @@ -13,7 +13,7 @@ username = 'root' groupname = 'wheel' scriptname = 'concatfragments.rb' - vardir = default['puppetvardir'] + vardir = default.puppet['vardir'] if vardir.nil? or vardir == '' vardir = '/opt/puppetlabs/puppet/cache' end @@ -21,13 +21,13 @@ username = 'Administrator' groupname = 'Administrators' scriptname = 'concatfragments.rb' - result = on default, "echo #{default['puppetvardir']}" + result = on default, "echo #{default.puppet['vardir']}" vardir = result.raw_output.chomp when 'Solaris' username = 'root' groupname = 'root' scriptname = 'concatfragments.rb' - vardir = default['puppetvardir'] + vardir = host.puppet['vardir'] if vardir.nil? or vardir == '' vardir = '/opt/puppetlabs/puppet/cache' end @@ -35,7 +35,7 @@ username = 'root' groupname = 'root' scriptname = 'concatfragments.rb' - vardir = default['puppetvardir'] + vardir = default.puppet['vardir'] if vardir.nil? or vardir == '' vardir = '/opt/puppetlabs/puppet/cache' end From 408633fd960703c60d1ee5c9b3482b480b10afe3 Mon Sep 17 00:00:00 2001 From: Morgan Haskel Date: Mon, 13 Jul 2015 15:36:56 -0700 Subject: [PATCH 59/76] Missed a typo --- spec/acceptance/concat_spec.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/spec/acceptance/concat_spec.rb b/spec/acceptance/concat_spec.rb index 2bceefc6..72b4b22f 100644 --- a/spec/acceptance/concat_spec.rb +++ b/spec/acceptance/concat_spec.rb @@ -27,7 +27,7 @@ username = 'root' groupname = 'root' scriptname = 'concatfragments.rb' - vardir = host.puppet['vardir'] + vardir = default.puppet['vardir'] if vardir.nil? or vardir == '' vardir = '/opt/puppetlabs/puppet/cache' end From c129edffca64f2c159d04a66c7c672ee085ad421 Mon Sep 17 00:00:00 2001 From: Morgan Haskel Date: Wed, 15 Jul 2015 17:24:11 -0700 Subject: [PATCH 60/76] Use AIO ruby if available --- manifests/init.pp | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/manifests/init.pp b/manifests/init.pp index 855be23f..43261ddc 100644 --- a/manifests/init.pp +++ b/manifests/init.pp @@ -209,10 +209,12 @@ if $::kernel == 'windows' { $command_path = "${::env_windows_installdir}/bin:${::path}" } else { - $command_path = "/opt/puppet/bin:${::path}" + $command_path = "/opt/puppetlabs/puppet/bin:/opt/puppet/bin:${::path}" } - } else { + } elsif $::kernel == 'windows' { $command_path = $::path + } else { + $command_path = "/opt/puppetlabs/puppet/bin:${::path}" } # if puppet is running as root, this exec should also run as root to allow From db914f1988555629ddfc25a12d501152e1161449 Mon Sep 17 00:00:00 2001 From: Morgan Haskel Date: Tue, 21 Jul 2015 14:38:45 -0700 Subject: [PATCH 61/76] 1.2.4 prep --- CHANGELOG.md | 12 +++++++++++- metadata.json | 12 ++++++------ 2 files changed, 17 insertions(+), 7 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 68baf333..f24932cb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,14 @@ +##2015-07-21 - Supported Release 1.2.4 +###Summary + +This release includes some ruby path and lint fixes. + +####Bugfixes +- Use correct ruby path with PE and AIO +- Lint fixes +- Use correct ruby path on FreeBSD +- Test fixes + ##2015-06-02 - Supported Release 2.0.1 [YANKED] ###Summary @@ -14,7 +25,6 @@ This is a bugfix release. This release includes a README fix to document correct behavior of fragment target parameter. ####Bugfixes - - README Fix to correctly document how a fragment $target param should work. ##2015-05-12 - Supported Release 2.0.0 [YANKED] diff --git a/metadata.json b/metadata.json index 6f6adab6..3f1caed7 100644 --- a/metadata.json +++ b/metadata.json @@ -1,12 +1,15 @@ { "name": "puppetlabs-concat", - "version": "1.2.3", + "version": "1.2.4", "author": "Puppet Labs", "summary": "Construct files from multiple fragments.", "license": "Apache-2.0", "source": "https://github.com/puppetlabs/puppetlabs-concat", "project_page": "https://github.com/puppetlabs/puppetlabs-concat", "issues_url": "https://tickets.puppetlabs.com/browse/MODULES", + "dependencies": [ + {"name":"puppetlabs/stdlib","version_requirement":">= 3.2.0 < 5.0.0"} + ], "operatingsystem_support": [ { "operatingsystem": "RedHat", @@ -98,14 +101,11 @@ "requirements": [ { "name": "pe", - "version_requirement": "3.x" + "version_requirement": ">= 3.0.0 < 2015.3.0" }, { "name": "puppet", - "version_requirement": "3.x" + "version_requirement": ">= 3.0.0 < 5.0.0" } - ], - "dependencies": [ - {"name":"puppetlabs/stdlib","version_requirement":">= 3.2.0 < 5.0.0"} ] } From 05426bb26b05989deb4938a0c4fa5c4f9ede232b Mon Sep 17 00:00:00 2001 From: Morgan Haskel Date: Tue, 21 Jul 2015 15:03:21 -0700 Subject: [PATCH 62/76] Revert "Add support for Solaris 12" --- README.md | 2 +- metadata.json | 3 +-- 2 files changed, 2 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index afec06dc..69b9a898 100644 --- a/README.md +++ b/README.md @@ -268,7 +268,7 @@ The `concat::setup` class should no longer be directly included in the manifest. ##Limitations -This module has been tested on [all PE-supported platforms](https://forge.puppetlabs.com/supported#compat-matrix), and no issues have been identified. Additionally, it is tested (but not supported) on Solaris 12. +This module has been tested on [all PE-supported platforms](https://forge.puppetlabs.com/supported#compat-matrix), and no issues have been identified. ##Development diff --git a/metadata.json b/metadata.json index 6f6adab6..1d459926 100644 --- a/metadata.json +++ b/metadata.json @@ -67,8 +67,7 @@ "operatingsystem": "Solaris", "operatingsystemrelease": [ "10", - "11", - "12" + "11" ] }, { From d5c305c2bf03912987fe7eba0da31dccadb65805 Mon Sep 17 00:00:00 2001 From: Sebastian Reitenbach Date: Wed, 22 Jul 2015 20:11:40 +0200 Subject: [PATCH 63/76] Recent OpenBSD changed to ruby22 as default interpreter, that now is also used by Puppet, so adapt to that. Tested on OpenBSD 5.8-beta, and 5.7-snapshot. Using kernelversion instead of operatingsystemrelease, because operatingsystemrelease can be 5.7, 5.8, 5.7-snapshot, 5.8-beta etc. --- manifests/setup.pp | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/manifests/setup.pp b/manifests/setup.pp index 6d5d8b2f..1b67b059 100644 --- a/manifests/setup.pp +++ b/manifests/setup.pp @@ -43,7 +43,10 @@ $script_command = $::osfamily? { 'windows' => "ruby.exe '${script_path}'", - 'openbsd' => "/usr/local/bin/ruby21 '${script_path}'", + 'openbsd' => $::kernelversion ? { + '5.7' => "/usr/local/bin/ruby21 '${script_path}'", + default => "/usr/local/bin/ruby22 '${script_path}'" + }, 'freebsd' => "/usr/local/bin/ruby '${script_path}'", default => $script_path } From ded2316c64f7b048082b897222cc1f460d9fac62 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?J=C3=A9r=C3=B4me=20Vizcaino?= Date: Thu, 23 Jul 2015 20:00:49 +0200 Subject: [PATCH 64/76] Never backup fragments, no matter what $backup for concat resource is used --- manifests/fragment.pp | 8 +------ manifests/init.pp | 5 +--- spec/unit/defines/concat_fragment_spec.rb | 2 +- spec/unit/defines/concat_spec.rb | 29 ++++++++++------------- 4 files changed, 16 insertions(+), 28 deletions(-) diff --git a/manifests/fragment.pp b/manifests/fragment.pp index 7b13b004..1c8b614e 100644 --- a/manifests/fragment.pp +++ b/manifests/fragment.pp @@ -58,12 +58,6 @@ warning('The $backup parameter to concat::fragment is deprecated and has no effect') } - $my_backup = concat_getparam(Concat[$target], 'backup') - $_backup = $my_backup ? { - '' => undef, - default => $my_backup - } - if $ensure == undef { $my_ensure = concat_getparam(Concat[$target], 'ensure') } else { @@ -125,7 +119,7 @@ mode => $fragmode, source => $source, content => $content, - backup => $_backup, + backup => false, replace => true, alias => "concat_fragment_${name}", notify => Exec["concat_${target}"] diff --git a/manifests/init.pp b/manifests/init.pp index 43261ddc..b62ebfa6 100644 --- a/manifests/init.pp +++ b/manifests/init.pp @@ -146,10 +146,6 @@ false => '', } - File { - backup => $backup, - } - # reset poisoned Exec defaults Exec { user => undef, @@ -168,6 +164,7 @@ force => true, ignore => ['.svn', '.git', '.gitignore'], notify => Exec["concat_${name}"], + backup => false, purge => true, recurse => true, } diff --git a/spec/unit/defines/concat_fragment_spec.rb b/spec/unit/defines/concat_fragment_spec.rb index 40866a86..5c350d16 100644 --- a/spec/unit/defines/concat_fragment_spec.rb +++ b/spec/unit/defines/concat_fragment_spec.rb @@ -50,7 +50,7 @@ :source => p[:source], :content => p[:content], :alias => "concat_fragment_#{title}", - :backup => 'puppet', + :backup => false, }) # The defined() function doesn't seem to work properly with puppet 3.4 and rspec. # defined() works on its own, rspec works on its own, but together they diff --git a/spec/unit/defines/concat_spec.rb b/spec/unit/defines/concat_spec.rb index b5dd2530..57685023 100644 --- a/spec/unit/defines/concat_spec.rb +++ b/spec/unit/defines/concat_spec.rb @@ -28,10 +28,6 @@ concat_name = 'fragments.concat.out' default_warn_message = '# This file is managed by Puppet. DO NOT EDIT.' - file_defaults = { - :backup => p[:backup], - } - let(:title) { title } let(:params) { params } let(:facts) do @@ -47,21 +43,22 @@ if p[:ensure] == 'present' it do - should contain_file(fragdir).with(file_defaults.merge({ + should contain_file(fragdir).with({ :ensure => 'directory', :mode => '0750', - })) + }) end it do - should contain_file("#{fragdir}/fragments").with(file_defaults.merge({ + should contain_file("#{fragdir}/fragments").with({ :ensure => 'directory', :mode => '0750', :force => true, :ignore => ['.svn', '.git', '.gitignore'], + :backup => false, :purge => true, :recurse => true, - })) + }) end [ @@ -69,15 +66,15 @@ "#{fragdir}/#{concat_name}", ].each do |file| it do - should contain_file(file).with(file_defaults.merge({ + should contain_file(file).with({ :ensure => 'present', :mode => '0640', - })) + }) end end it do - should contain_file(title).with(file_defaults.merge({ + should contain_file(title).with({ :ensure => 'present', :owner => p[:owner], :group => p[:group], @@ -88,7 +85,7 @@ :source => "#{fragdir}/#{concat_name}", :validate_cmd => p[:validate_cmd], :backup => p[:backup], - })) + }) end cmd = "#{concatdir}/bin/concatfragments.rb " + @@ -136,18 +133,18 @@ "#{fragdir}/#{concat_name}", ].each do |file| it do - should contain_file(file).with(file_defaults.merge({ + should contain_file(file).with({ :ensure => 'absent', :force => true, - })) + }) end end it do - should contain_file(title).with(file_defaults.merge({ + should contain_file(title).with({ :ensure => 'absent', :backup => p[:backup], - })) + }) end it do From 6ba6cd2134b0d2f3a8cc6d55b3c2878ed78def0f Mon Sep 17 00:00:00 2001 From: Joshua Hoblitt Date: Thu, 30 Jul 2015 15:33:12 -0700 Subject: [PATCH 65/76] Revert "Backup option breaks concat" --- manifests/fragment.pp | 8 ++++++- manifests/init.pp | 5 +++- spec/unit/defines/concat_fragment_spec.rb | 2 +- spec/unit/defines/concat_spec.rb | 29 +++++++++++++---------- 4 files changed, 28 insertions(+), 16 deletions(-) diff --git a/manifests/fragment.pp b/manifests/fragment.pp index 1c8b614e..7b13b004 100644 --- a/manifests/fragment.pp +++ b/manifests/fragment.pp @@ -58,6 +58,12 @@ warning('The $backup parameter to concat::fragment is deprecated and has no effect') } + $my_backup = concat_getparam(Concat[$target], 'backup') + $_backup = $my_backup ? { + '' => undef, + default => $my_backup + } + if $ensure == undef { $my_ensure = concat_getparam(Concat[$target], 'ensure') } else { @@ -119,7 +125,7 @@ mode => $fragmode, source => $source, content => $content, - backup => false, + backup => $_backup, replace => true, alias => "concat_fragment_${name}", notify => Exec["concat_${target}"] diff --git a/manifests/init.pp b/manifests/init.pp index b62ebfa6..43261ddc 100644 --- a/manifests/init.pp +++ b/manifests/init.pp @@ -146,6 +146,10 @@ false => '', } + File { + backup => $backup, + } + # reset poisoned Exec defaults Exec { user => undef, @@ -164,7 +168,6 @@ force => true, ignore => ['.svn', '.git', '.gitignore'], notify => Exec["concat_${name}"], - backup => false, purge => true, recurse => true, } diff --git a/spec/unit/defines/concat_fragment_spec.rb b/spec/unit/defines/concat_fragment_spec.rb index 5c350d16..40866a86 100644 --- a/spec/unit/defines/concat_fragment_spec.rb +++ b/spec/unit/defines/concat_fragment_spec.rb @@ -50,7 +50,7 @@ :source => p[:source], :content => p[:content], :alias => "concat_fragment_#{title}", - :backup => false, + :backup => 'puppet', }) # The defined() function doesn't seem to work properly with puppet 3.4 and rspec. # defined() works on its own, rspec works on its own, but together they diff --git a/spec/unit/defines/concat_spec.rb b/spec/unit/defines/concat_spec.rb index 57685023..b5dd2530 100644 --- a/spec/unit/defines/concat_spec.rb +++ b/spec/unit/defines/concat_spec.rb @@ -28,6 +28,10 @@ concat_name = 'fragments.concat.out' default_warn_message = '# This file is managed by Puppet. DO NOT EDIT.' + file_defaults = { + :backup => p[:backup], + } + let(:title) { title } let(:params) { params } let(:facts) do @@ -43,22 +47,21 @@ if p[:ensure] == 'present' it do - should contain_file(fragdir).with({ + should contain_file(fragdir).with(file_defaults.merge({ :ensure => 'directory', :mode => '0750', - }) + })) end it do - should contain_file("#{fragdir}/fragments").with({ + should contain_file("#{fragdir}/fragments").with(file_defaults.merge({ :ensure => 'directory', :mode => '0750', :force => true, :ignore => ['.svn', '.git', '.gitignore'], - :backup => false, :purge => true, :recurse => true, - }) + })) end [ @@ -66,15 +69,15 @@ "#{fragdir}/#{concat_name}", ].each do |file| it do - should contain_file(file).with({ + should contain_file(file).with(file_defaults.merge({ :ensure => 'present', :mode => '0640', - }) + })) end end it do - should contain_file(title).with({ + should contain_file(title).with(file_defaults.merge({ :ensure => 'present', :owner => p[:owner], :group => p[:group], @@ -85,7 +88,7 @@ :source => "#{fragdir}/#{concat_name}", :validate_cmd => p[:validate_cmd], :backup => p[:backup], - }) + })) end cmd = "#{concatdir}/bin/concatfragments.rb " + @@ -133,18 +136,18 @@ "#{fragdir}/#{concat_name}", ].each do |file| it do - should contain_file(file).with({ + should contain_file(file).with(file_defaults.merge({ :ensure => 'absent', :force => true, - }) + })) end end it do - should contain_file(title).with({ + should contain_file(title).with(file_defaults.merge({ :ensure => 'absent', :backup => p[:backup], - }) + })) end it do From fede2461c1cf82267ba65b7f40b60b9d3e4d5ca1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?J=C3=A9r=C3=B4me=20Vizcaino?= Date: Thu, 23 Jul 2015 20:00:49 +0200 Subject: [PATCH 66/76] Never backup fragments, no matter what $backup for concat resource is used --- manifests/fragment.pp | 8 +------ manifests/init.pp | 5 +--- spec/unit/defines/concat_fragment_spec.rb | 2 +- spec/unit/defines/concat_spec.rb | 29 ++++++++++------------- 4 files changed, 16 insertions(+), 28 deletions(-) diff --git a/manifests/fragment.pp b/manifests/fragment.pp index 7b13b004..1c8b614e 100644 --- a/manifests/fragment.pp +++ b/manifests/fragment.pp @@ -58,12 +58,6 @@ warning('The $backup parameter to concat::fragment is deprecated and has no effect') } - $my_backup = concat_getparam(Concat[$target], 'backup') - $_backup = $my_backup ? { - '' => undef, - default => $my_backup - } - if $ensure == undef { $my_ensure = concat_getparam(Concat[$target], 'ensure') } else { @@ -125,7 +119,7 @@ mode => $fragmode, source => $source, content => $content, - backup => $_backup, + backup => false, replace => true, alias => "concat_fragment_${name}", notify => Exec["concat_${target}"] diff --git a/manifests/init.pp b/manifests/init.pp index 43261ddc..b62ebfa6 100644 --- a/manifests/init.pp +++ b/manifests/init.pp @@ -146,10 +146,6 @@ false => '', } - File { - backup => $backup, - } - # reset poisoned Exec defaults Exec { user => undef, @@ -168,6 +164,7 @@ force => true, ignore => ['.svn', '.git', '.gitignore'], notify => Exec["concat_${name}"], + backup => false, purge => true, recurse => true, } diff --git a/spec/unit/defines/concat_fragment_spec.rb b/spec/unit/defines/concat_fragment_spec.rb index 40866a86..5c350d16 100644 --- a/spec/unit/defines/concat_fragment_spec.rb +++ b/spec/unit/defines/concat_fragment_spec.rb @@ -50,7 +50,7 @@ :source => p[:source], :content => p[:content], :alias => "concat_fragment_#{title}", - :backup => 'puppet', + :backup => false, }) # The defined() function doesn't seem to work properly with puppet 3.4 and rspec. # defined() works on its own, rspec works on its own, but together they diff --git a/spec/unit/defines/concat_spec.rb b/spec/unit/defines/concat_spec.rb index b5dd2530..57685023 100644 --- a/spec/unit/defines/concat_spec.rb +++ b/spec/unit/defines/concat_spec.rb @@ -28,10 +28,6 @@ concat_name = 'fragments.concat.out' default_warn_message = '# This file is managed by Puppet. DO NOT EDIT.' - file_defaults = { - :backup => p[:backup], - } - let(:title) { title } let(:params) { params } let(:facts) do @@ -47,21 +43,22 @@ if p[:ensure] == 'present' it do - should contain_file(fragdir).with(file_defaults.merge({ + should contain_file(fragdir).with({ :ensure => 'directory', :mode => '0750', - })) + }) end it do - should contain_file("#{fragdir}/fragments").with(file_defaults.merge({ + should contain_file("#{fragdir}/fragments").with({ :ensure => 'directory', :mode => '0750', :force => true, :ignore => ['.svn', '.git', '.gitignore'], + :backup => false, :purge => true, :recurse => true, - })) + }) end [ @@ -69,15 +66,15 @@ "#{fragdir}/#{concat_name}", ].each do |file| it do - should contain_file(file).with(file_defaults.merge({ + should contain_file(file).with({ :ensure => 'present', :mode => '0640', - })) + }) end end it do - should contain_file(title).with(file_defaults.merge({ + should contain_file(title).with({ :ensure => 'present', :owner => p[:owner], :group => p[:group], @@ -88,7 +85,7 @@ :source => "#{fragdir}/#{concat_name}", :validate_cmd => p[:validate_cmd], :backup => p[:backup], - })) + }) end cmd = "#{concatdir}/bin/concatfragments.rb " + @@ -136,18 +133,18 @@ "#{fragdir}/#{concat_name}", ].each do |file| it do - should contain_file(file).with(file_defaults.merge({ + should contain_file(file).with({ :ensure => 'absent', :force => true, - })) + }) end end it do - should contain_file(title).with(file_defaults.merge({ + should contain_file(title).with({ :ensure => 'absent', :backup => p[:backup], - })) + }) end it do From b6f1ac1d359fdf3e1fe95d411a430c71cb449eb1 Mon Sep 17 00:00:00 2001 From: Joshua Hoblitt Date: Thu, 30 Jul 2015 15:52:54 -0700 Subject: [PATCH 67/76] (MODULES-1700) remove all reliance on resource defaults And be explicitly defensive about defaults leaking in from other parts of the manifest. --- manifests/init.pp | 14 ++++++++------ manifests/setup.pp | 2 ++ 2 files changed, 10 insertions(+), 6 deletions(-) diff --git a/manifests/init.pp b/manifests/init.pp index b62ebfa6..2998d69e 100644 --- a/manifests/init.pp +++ b/manifests/init.pp @@ -146,16 +146,11 @@ false => '', } - # reset poisoned Exec defaults - Exec { - user => undef, - group => undef, - } - if $ensure == 'present' { file { $fragdir: ensure => directory, mode => '0750', + backup => false, } file { "${fragdir}/fragments": @@ -172,11 +167,13 @@ file { "${fragdir}/fragments.concat": ensure => present, mode => '0640', + backup => false, } file { "${fragdir}/${concat_name}": ensure => present, mode => '0640', + backup => false, } file { $name: @@ -224,6 +221,8 @@ subscribe => File[$fragdir], unless => "${command} -t", path => $command_path, + user => undef, + group => undef, require => [ File[$fragdir], File["${fragdir}/fragments"], @@ -239,6 +238,7 @@ ]: ensure => absent, force => true, + backup => false, } file { $path: @@ -266,6 +266,8 @@ command => $absent_exec_command, unless => $absent_exec_command, path => $absent_exec_path, + user => undef, + group => undef, } } } diff --git a/manifests/setup.pp b/manifests/setup.pp index 1b67b059..f186ef37 100644 --- a/manifests/setup.pp +++ b/manifests/setup.pp @@ -57,6 +57,7 @@ group => $default_group, mode => $script_mode, source => "puppet:///modules/concat/${script_name}", + backup => false, } file { [ $concatdir, "${concatdir}/bin" ]: @@ -64,5 +65,6 @@ owner => $default_owner, group => $default_group, mode => '0755', + backup => false, } } From ca58571ab2fd3619a9ae516987689d051b3814ff Mon Sep 17 00:00:00 2001 From: Bryan Jen Date: Tue, 11 Aug 2015 16:46:12 -0700 Subject: [PATCH 68/76] (MODULES-1700) Re-factored original fix, to not default to backing up fragments. --- README.md | 4 +++ manifests/fragment.pp | 11 ++++++++- manifests/init.pp | 31 +++++++++++++---------- spec/unit/defines/concat_spec.rb | 42 +++++++++++++++++++++++--------- 4 files changed, 62 insertions(+), 26 deletions(-) diff --git a/README.md b/README.md index 69b9a898..a0942bbd 100644 --- a/README.md +++ b/README.md @@ -139,6 +139,10 @@ All the parameters listed below are optional. Specifies whether (and how) to back up the destination file before overwriting it. Your value gets passed on to Puppet's [native `file` resource](https://docs.puppetlabs.com/references/latest/type.html#file-attribute-backup) for execution. Valid options: 'true', 'false', or a string representing either a target filebucket or a filename extension beginning with ".". Default value: 'puppet'. +#####`backup_fragments` + +Specifies whether to backup concat fragments using the backup setting of the target concat file. Valid options: 'true' and 'false'. Default value: 'false'. + #####`ensure` Specifies whether the destination file should exist. Setting to 'absent' tells Puppet to delete the destination file if it exists, and negates the effect of any other parameters. Valid options: 'present' and 'absent'. Default value: 'present'. diff --git a/manifests/fragment.pp b/manifests/fragment.pp index 1c8b614e..1e8a5d42 100644 --- a/manifests/fragment.pp +++ b/manifests/fragment.pp @@ -58,6 +58,15 @@ warning('The $backup parameter to concat::fragment is deprecated and has no effect') } + # Checks the target concat resources for whether fragments should be backed up or not + # otherwise defaults to false. + $enable_backup = concat_getparam(Concat[$target], 'backup_fragments') + $target_backup = concat_getparam(Concat[$target], 'backup') + $_backup = $enable_backup ? { + true => $target_backup, + default => false + } + if $ensure == undef { $my_ensure = concat_getparam(Concat[$target], 'ensure') } else { @@ -119,7 +128,7 @@ mode => $fragmode, source => $source, content => $content, - backup => false, + backup => $_backup, replace => true, alias => "concat_fragment_${name}", notify => Exec["concat_${target}"] diff --git a/manifests/init.pp b/manifests/init.pp index 2998d69e..29930b1e 100644 --- a/manifests/init.pp +++ b/manifests/init.pp @@ -25,6 +25,9 @@ # [*backup*] # Controls the filebucketing behavior of the final file and see File type # reference for its use. Defaults to 'puppet' +# [*backup_fragments*] +# Enables backup of fragments using the backup setting of the target +# concat file. Defaults to 'false' # [*replace*] # Whether to replace a file that already exists on the local system # [*order*] @@ -53,19 +56,20 @@ # File["concat_/path/to/file"] # define concat( - $ensure = 'present', - $path = $name, - $owner = undef, - $group = undef, - $mode = '0644', - $warn = false, - $force = false, - $backup = 'puppet', - $replace = true, - $order = 'alpha', - $ensure_newline = false, - $validate_cmd = undef, - $gnu = undef + $ensure = 'present', + $path = $name, + $owner = undef, + $group = undef, + $mode = '0644', + $warn = false, + $force = false, + $backup = 'puppet', + $backup_fragments = false, + $replace = true, + $order = 'alpha', + $ensure_newline = false, + $validate_cmd = undef, + $gnu = undef ) { validate_re($ensure, '^present$|^absent$') validate_absolute_path($path) @@ -79,6 +83,7 @@ if ! concat_is_bool($backup) and ! is_string($backup) { fail('$backup must be string or bool!') } + validate_bool($backup_fragments) validate_bool($replace) validate_re($order, '^alpha$|^numeric$') validate_bool($ensure_newline) diff --git a/spec/unit/defines/concat_spec.rb b/spec/unit/defines/concat_spec.rb index 57685023..fc110d9d 100644 --- a/spec/unit/defines/concat_spec.rb +++ b/spec/unit/defines/concat_spec.rb @@ -8,18 +8,19 @@ # default param values p = { - :ensure => 'present', - :path => title, - :owner => nil, - :group => nil, - :mode => '0644', - :warn => false, - :force => false, - :backup => 'puppet', - :replace => true, - :order => 'alpha', - :ensure_newline => false, - :validate_cmd => nil, + :ensure => 'present', + :path => title, + :owner => nil, + :group => nil, + :mode => '0644', + :warn => false, + :force => false, + :backup => 'puppet', + :backup_fragments => false, + :replace => true, + :order => 'alpha', + :ensure_newline => false, + :validate_cmd => nil, }.merge(params) safe_name = title.gsub('/', '_') @@ -331,6 +332,23 @@ end end # backup => + context 'backup_fragments =>' do + context 'true' do + it_behaves_like 'concat', '/etc/foo.bar', { :backup_fragments => true } + end + + context 'false' do + it_behaves_like 'concat', '/etc/foo.bar', { :backup_fragments => false } + end + + context 'invalid' do + let(:title) { '/etc/foo.bar' } + let(:params) {{ :backup_fragments => "invalid" }} + it 'should fail' do + expect { catalogue }.to raise_error(Puppet::Error, /is not a boolean/) + end + end + end # backup_fragments => context 'replace =>' do [true, false].each do |replace| context replace do From ff1e65a901381349e3ed37591605e8baac24be21 Mon Sep 17 00:00:00 2001 From: Joshua Hoblitt Date: Wed, 29 Jul 2015 16:54:57 -0700 Subject: [PATCH 69/76] (MODULES-2303) add selinux related params to concat type --- README.md | 30 ++++++++++++++ manifests/init.pp | 67 +++++++++++++++++++++----------- spec/unit/defines/concat_spec.rb | 66 ++++++++++++++++++++++++++----- 3 files changed, 130 insertions(+), 33 deletions(-) diff --git a/README.md b/README.md index a0942bbd..e765cb1c 100644 --- a/README.md +++ b/README.md @@ -199,6 +199,36 @@ If you set this parameter to 'true', Puppet adds the following message: # This file is managed by Puppet. DO NOT EDIT. ~~~ +#####`selinux_ignore_defaults` + +See the `file` type's +[`selinux_ignore_defaults`](https://docs.puppetlabs.com/references/latest/type.html#file-attribute-selinux_ignore_defaults) +documentention. + +#####`selrange` + +See the `file` type's +[`selrange`](https://docs.puppetlabs.com/references/latest/type.html#file-attribute-selrange) +documentention. + +#####`selrole` + +See the `file` type's +[`selrole`](https://docs.puppetlabs.com/references/latest/type.html#file-attribute-selrole) +documentention. + +#####`seltype` + +See the `file` type's +[`seltype`](https://docs.puppetlabs.com/references/latest/type.html#file-attribute-seltype) +documentention. + +#####`seluser` + +See the `file` type's +[`seluser`](https://docs.puppetlabs.com/references/latest/type.html#file-attribute-seluser) +documentention. + ####`concat::fragment` diff --git a/manifests/init.pp b/manifests/init.pp index 29930b1e..b1bbb8ad 100644 --- a/manifests/init.pp +++ b/manifests/init.pp @@ -34,6 +34,10 @@ # [*ensure_newline*] # [*gnu*] # Deprecated +# [*selinux_ignore_defaults*] +# [*selrange*] +# [*selrole*] +# [*seltype*] # # === Actions: # * Creates fragment directories if it didn't exist already @@ -56,20 +60,25 @@ # File["concat_/path/to/file"] # define concat( - $ensure = 'present', - $path = $name, - $owner = undef, - $group = undef, - $mode = '0644', - $warn = false, - $force = false, - $backup = 'puppet', - $backup_fragments = false, - $replace = true, - $order = 'alpha', - $ensure_newline = false, - $validate_cmd = undef, - $gnu = undef + $ensure = 'present', + $path = $name, + $owner = undef, + $group = undef, + $mode = '0644', + $warn = false, + $force = false, + $backup = 'puppet', + $backup_fragments = false, + $replace = true, + $order = 'alpha', + $ensure_newline = false, + $validate_cmd = undef, + $gnu = undef, + $selinux_ignore_defaults = undef, + $selrange = undef, + $selrole = undef, + $seltype = undef, + $seluser = undef ) { validate_re($ensure, '^present$|^absent$') validate_absolute_path($path) @@ -93,6 +102,13 @@ if $gnu { warning('The $gnu parameter to concat is deprecated and has no effect') } + if $selinux_ignore_defaults { + validate_bool($selinux_ignore_defaults) + } + validate_string($selrange) + validate_string($selrole) + validate_string($seltype) + validate_string($seluser) include concat::setup @@ -182,15 +198,20 @@ } file { $name: - ensure => present, - owner => $owner, - group => $group, - mode => $mode, - replace => $replace, - path => $path, - alias => "concat_${name}", - source => "${fragdir}/${concat_name}", - backup => $backup, + ensure => present, + owner => $owner, + group => $group, + mode => $mode, + selinux_ignore_defaults => $selinux_ignore_defaults, + selrange => $selrange, + selrole => $selrole, + seltype => $seltype, + seluser => $seluser, + replace => $replace, + path => $path, + alias => "concat_${name}", + source => "${fragdir}/${concat_name}", + backup => $backup, } # Only newer versions of puppet 3.x support the validate_cmd parameter diff --git a/spec/unit/defines/concat_spec.rb b/spec/unit/defines/concat_spec.rb index fc110d9d..6501a9f5 100644 --- a/spec/unit/defines/concat_spec.rb +++ b/spec/unit/defines/concat_spec.rb @@ -76,16 +76,21 @@ it do should contain_file(title).with({ - :ensure => 'present', - :owner => p[:owner], - :group => p[:group], - :mode => p[:mode], - :replace => p[:replace], - :path => p[:path], - :alias => "concat_#{title}", - :source => "#{fragdir}/#{concat_name}", - :validate_cmd => p[:validate_cmd], - :backup => p[:backup], + :ensure => 'present', + :owner => p[:owner], + :group => p[:group], + :mode => p[:mode], + :replace => p[:replace], + :path => p[:path], + :alias => "concat_#{title}", + :source => "#{fragdir}/#{concat_name}", + :validate_cmd => p[:validate_cmd], + :backup => p[:backup], + :selinux_ignore_defaults => p[:selinux_ignore_defaults], + :selrange => p[:selrange], + :selrole => p[:selrole], + :seltype => p[:seltype], + :seluser => p[:seluser], }) end @@ -415,6 +420,47 @@ end end # validate_cmd => + context 'selinux_ignore_defaults =>' do + let(:title) { '/etc/foo.bar' } + + [true, false].each do |v| + context v do + it_behaves_like 'concat', '/etc/foo.bar', { :selinux_ignore_defaults => v } + end + end + + context '123' do + let(:title) { '/etc/foo.bar' } + let(:params) {{ :selinux_ignore_defaults => 123 }} + it 'should fail' do + expect { catalogue }.to raise_error(Puppet::Error, /is not a boolean/) + end + end + end # selinux_ignore_defaults => + + [ + :selrange, + :selrole, + :seltype, + :seluser, + ].each do |p| + context " #{p} =>" do + let(:title) { '/etc/foo.bar' } + + context 'foo' do + it_behaves_like 'concat', '/etc/foo.bar', { p => 'foo' } + end + + context 'false' do + let(:title) { '/etc/foo.bar' } + let(:params) {{ p => false }} + it 'should fail' do + expect { catalogue }.to raise_error(Puppet::Error, /is not a string/) + end + end + end # #{p} => + end + describe 'deprecated parameter' do context 'gnu =>' do context 'foo' do From 60051ae55d1d398f0439ae91e9fdaf0d944789b4 Mon Sep 17 00:00:00 2001 From: Joshua Hoblitt Date: Tue, 18 Aug 2015 00:24:32 -0700 Subject: [PATCH 70/76] add log/ to .gitignore --- .gitignore | 1 + 1 file changed, 1 insertion(+) diff --git a/.gitignore b/.gitignore index b5db85e0..874db461 100644 --- a/.gitignore +++ b/.gitignore @@ -7,3 +7,4 @@ spec/fixtures/ coverage/ .idea/ *.iml +log/ From 5efd30a0e193e1b95c3077c9610886df70735769 Mon Sep 17 00:00:00 2001 From: Ryan Whitehurst Date: Mon, 14 Sep 2015 15:56:20 -0700 Subject: [PATCH 71/76] (MODULES-2572) Fix acceptance test vagrant URLs The vagrant boxes at puppet-vagrant-boxes.puppetlabs.com were removed a few months ago in favor of those hosted at atlas.hashicorp.com. This switches the acceptance test nodesets to specify the correct URL which makes it possible to manually run the acceptance tests. --- spec/acceptance/nodesets/centos-59-x64.yml | 2 +- spec/acceptance/nodesets/centos-64-x64-pe.yml | 2 +- spec/acceptance/nodesets/centos-64-x64.yml | 2 +- spec/acceptance/nodesets/centos-65-x64.yml | 2 +- spec/acceptance/nodesets/debian-607-x64.yml | 2 +- spec/acceptance/nodesets/debian-70rc1-x64.yml | 2 +- spec/acceptance/nodesets/debian-73-x64.yml | 2 +- spec/acceptance/nodesets/default.yml | 2 +- spec/acceptance/nodesets/fedora-18-x64.yml | 2 +- spec/acceptance/nodesets/sles-11-x64.yml | 2 +- spec/acceptance/nodesets/sles-11sp1-x64.yml | 2 +- spec/acceptance/nodesets/ubuntu-server-10044-x64.yml | 2 +- spec/acceptance/nodesets/ubuntu-server-12042-x64.yml | 2 +- 13 files changed, 13 insertions(+), 13 deletions(-) diff --git a/spec/acceptance/nodesets/centos-59-x64.yml b/spec/acceptance/nodesets/centos-59-x64.yml index 2ad90b86..6d110a64 100644 --- a/spec/acceptance/nodesets/centos-59-x64.yml +++ b/spec/acceptance/nodesets/centos-59-x64.yml @@ -4,7 +4,7 @@ HOSTS: - master platform: el-5-x86_64 box : centos-59-x64-vbox4210-nocm - box_url : http://puppet-vagrant-boxes.puppetlabs.com/centos-59-x64-vbox4210-nocm.box + box_url : https://atlas.hashicorp.com/puppetlabs/boxes/centos-59-x64-vbox4210-nocm.box hypervisor : vagrant CONFIG: type: git diff --git a/spec/acceptance/nodesets/centos-64-x64-pe.yml b/spec/acceptance/nodesets/centos-64-x64-pe.yml index 7d9242f1..8e20d74e 100644 --- a/spec/acceptance/nodesets/centos-64-x64-pe.yml +++ b/spec/acceptance/nodesets/centos-64-x64-pe.yml @@ -6,7 +6,7 @@ HOSTS: - dashboard platform: el-6-x86_64 box : centos-64-x64-vbox4210-nocm - box_url : http://puppet-vagrant-boxes.puppetlabs.com/centos-64-x64-vbox4210-nocm.box + box_url : https://atlas.hashicorp.com/puppetlabs/boxes/centos-64-x64-vbox4210-nocm.box hypervisor : vagrant CONFIG: type: pe diff --git a/spec/acceptance/nodesets/centos-64-x64.yml b/spec/acceptance/nodesets/centos-64-x64.yml index 06398354..63a66373 100644 --- a/spec/acceptance/nodesets/centos-64-x64.yml +++ b/spec/acceptance/nodesets/centos-64-x64.yml @@ -4,7 +4,7 @@ HOSTS: - master platform: el-6-x86_64 box : centos-64-x64-vbox4210-nocm - box_url : http://puppet-vagrant-boxes.puppetlabs.com/centos-64-x64-vbox4210-nocm.box + box_url : https://atlas.hashicorp.com/puppetlabs/boxes/centos-64-x64-vbox4210-nocm.box hypervisor : vagrant CONFIG: type: git diff --git a/spec/acceptance/nodesets/centos-65-x64.yml b/spec/acceptance/nodesets/centos-65-x64.yml index 4e2cb809..e0677fae 100644 --- a/spec/acceptance/nodesets/centos-65-x64.yml +++ b/spec/acceptance/nodesets/centos-65-x64.yml @@ -4,7 +4,7 @@ HOSTS: - master platform: el-6-x86_64 box : centos-65-x64-vbox436-nocm - box_url : http://puppet-vagrant-boxes.puppetlabs.com/centos-65-x64-virtualbox-nocm.box + box_url : https://atlas.hashicorp.com/puppetlabs/boxes/centos-65-x64-virtualbox-nocm.box hypervisor : vagrant CONFIG: type: foss diff --git a/spec/acceptance/nodesets/debian-607-x64.yml b/spec/acceptance/nodesets/debian-607-x64.yml index 4c8be42d..cfbcb0d1 100644 --- a/spec/acceptance/nodesets/debian-607-x64.yml +++ b/spec/acceptance/nodesets/debian-607-x64.yml @@ -4,7 +4,7 @@ HOSTS: - master platform: debian-6-amd64 box : debian-607-x64-vbox4210-nocm - box_url : http://puppet-vagrant-boxes.puppetlabs.com/debian-607-x64-vbox4210-nocm.box + box_url : https://atlas.hashicorp.com/puppetlabs/boxes/debian-607-x64-vbox4210-nocm.box hypervisor : vagrant CONFIG: type: git diff --git a/spec/acceptance/nodesets/debian-70rc1-x64.yml b/spec/acceptance/nodesets/debian-70rc1-x64.yml index 19181c12..ead9c629 100644 --- a/spec/acceptance/nodesets/debian-70rc1-x64.yml +++ b/spec/acceptance/nodesets/debian-70rc1-x64.yml @@ -4,7 +4,7 @@ HOSTS: - master platform: debian-7-amd64 box : debian-70rc1-x64-vbox4210-nocm - box_url : http://puppet-vagrant-boxes.puppetlabs.com/debian-70rc1-x64-vbox4210-nocm.box + box_url : https://atlas.hashicorp.com/puppetlabs/boxes/debian-70rc1-x64-vbox4210-nocm.box hypervisor : vagrant CONFIG: type: git diff --git a/spec/acceptance/nodesets/debian-73-x64.yml b/spec/acceptance/nodesets/debian-73-x64.yml index 3e6a3a9d..55c874a2 100644 --- a/spec/acceptance/nodesets/debian-73-x64.yml +++ b/spec/acceptance/nodesets/debian-73-x64.yml @@ -4,7 +4,7 @@ HOSTS: - master platform: debian-7-amd64 box : debian-73-x64-virtualbox-nocm - box_url : http://puppet-vagrant-boxes.puppetlabs.com/debian-73-x64-virtualbox-nocm.box + box_url : https://atlas.hashicorp.com/puppetlabs/boxes/debian-73-x64-virtualbox-nocm.box hypervisor : vagrant CONFIG: log_level: debug diff --git a/spec/acceptance/nodesets/default.yml b/spec/acceptance/nodesets/default.yml index b8268515..b31fe275 100644 --- a/spec/acceptance/nodesets/default.yml +++ b/spec/acceptance/nodesets/default.yml @@ -4,7 +4,7 @@ HOSTS: - master platform: el-6-x86_64 box : puppetlabs/centos-6.6-64-nocm - box_url : http://puppet-vagrant-boxes.puppetlabs.com/centos-6.6-64-nocm + box_url : https://atlas.hashicorp.com/puppetlabs/boxes/centos-6.6-64-nocm hypervisor : vagrant CONFIG: type: git diff --git a/spec/acceptance/nodesets/fedora-18-x64.yml b/spec/acceptance/nodesets/fedora-18-x64.yml index 624b5371..89d57a38 100644 --- a/spec/acceptance/nodesets/fedora-18-x64.yml +++ b/spec/acceptance/nodesets/fedora-18-x64.yml @@ -4,7 +4,7 @@ HOSTS: - master platform: fedora-18-x86_64 box : fedora-18-x64-vbox4210-nocm - box_url : http://puppet-vagrant-boxes.puppetlabs.com/fedora-18-x64-vbox4210-nocm.box + box_url : https://atlas.hashicorp.com/puppetlabs/boxes/fedora-18-x64-vbox4210-nocm.box hypervisor : vagrant CONFIG: type: git diff --git a/spec/acceptance/nodesets/sles-11-x64.yml b/spec/acceptance/nodesets/sles-11-x64.yml index 41abe213..1681ae45 100644 --- a/spec/acceptance/nodesets/sles-11-x64.yml +++ b/spec/acceptance/nodesets/sles-11-x64.yml @@ -4,7 +4,7 @@ HOSTS: - master platform: sles-11-x64 box : sles-11sp1-x64-vbox4210-nocm - box_url : http://puppet-vagrant-boxes.puppetlabs.com/sles-11sp1-x64-vbox4210-nocm.box + box_url : https://atlas.hashicorp.com/puppetlabs/boxes/sles-11sp1-x64-vbox4210-nocm.box hypervisor : vagrant CONFIG: type: foss diff --git a/spec/acceptance/nodesets/sles-11sp1-x64.yml b/spec/acceptance/nodesets/sles-11sp1-x64.yml index 554c37a5..604d6f93 100644 --- a/spec/acceptance/nodesets/sles-11sp1-x64.yml +++ b/spec/acceptance/nodesets/sles-11sp1-x64.yml @@ -4,7 +4,7 @@ HOSTS: - master platform: sles-11-x86_64 box : sles-11sp1-x64-vbox4210-nocm - box_url : http://puppet-vagrant-boxes.puppetlabs.com/sles-11sp1-x64-vbox4210-nocm.box + box_url : https://atlas.hashicorp.com/puppetlabs/boxes/sles-11sp1-x64-vbox4210-nocm.box hypervisor : vagrant CONFIG: type: git diff --git a/spec/acceptance/nodesets/ubuntu-server-10044-x64.yml b/spec/acceptance/nodesets/ubuntu-server-10044-x64.yml index 5ca1514e..7f5e198b 100644 --- a/spec/acceptance/nodesets/ubuntu-server-10044-x64.yml +++ b/spec/acceptance/nodesets/ubuntu-server-10044-x64.yml @@ -4,7 +4,7 @@ HOSTS: - master platform: ubuntu-10.04-amd64 box : ubuntu-server-10044-x64-vbox4210-nocm - box_url : http://puppet-vagrant-boxes.puppetlabs.com/ubuntu-server-10044-x64-vbox4210-nocm.box + box_url : https://atlas.hashicorp.com/puppetlabs/boxes/ubuntu-server-10044-x64-vbox4210-nocm.box hypervisor : vagrant CONFIG: type: foss diff --git a/spec/acceptance/nodesets/ubuntu-server-12042-x64.yml b/spec/acceptance/nodesets/ubuntu-server-12042-x64.yml index d065b304..05bdde77 100644 --- a/spec/acceptance/nodesets/ubuntu-server-12042-x64.yml +++ b/spec/acceptance/nodesets/ubuntu-server-12042-x64.yml @@ -4,7 +4,7 @@ HOSTS: - master platform: ubuntu-12.04-amd64 box : ubuntu-server-12042-x64-vbox4210-nocm - box_url : http://puppet-vagrant-boxes.puppetlabs.com/ubuntu-server-12042-x64-vbox4210-nocm.box + box_url : https://atlas.hashicorp.com/puppetlabs/boxes/ubuntu-server-12042-x64-vbox4210-nocm.box hypervisor : vagrant CONFIG: type: foss From a6ebae9857e40e31179659401a64d81d57eac8d9 Mon Sep 17 00:00:00 2001 From: Ryan Whitehurst Date: Mon, 14 Sep 2015 16:15:57 -0700 Subject: [PATCH 72/76] Allow integer UID/GID for $owner/$group This allows specifying a numerical user ID or group ID for the $owner and $group parameters, respectively. Currently, only string values for the user name and group name are allowed, but these parameters are passed straight to the file resource that is created, which supports both string name and numeric ID. --- manifests/init.pp | 8 ++++++-- spec/unit/defines/concat_spec.rb | 12 ++++++++++-- 2 files changed, 16 insertions(+), 4 deletions(-) diff --git a/manifests/init.pp b/manifests/init.pp index 29930b1e..b729e99d 100644 --- a/manifests/init.pp +++ b/manifests/init.pp @@ -73,9 +73,13 @@ ) { validate_re($ensure, '^present$|^absent$') validate_absolute_path($path) - validate_string($owner) - validate_string($group) validate_string($mode) + if ! (is_string($owner) or is_integer($owner)) { + fail("\$owner must be a string or integer, got ${owner}") + } + if ! (is_string($group) or is_integer($group)) { + fail("\$group must be a string or integer, got ${group}") + } if ! (is_string($warn) or $warn == true or $warn == false) { fail('$warn is not a string or boolean') } diff --git a/spec/unit/defines/concat_spec.rb b/spec/unit/defines/concat_spec.rb index fc110d9d..896415ac 100644 --- a/spec/unit/defines/concat_spec.rb +++ b/spec/unit/defines/concat_spec.rb @@ -229,11 +229,15 @@ it_behaves_like 'concat', '/etc/foo.bar', { :owner => 'apenny' } end + context '1000' do + it_behaves_like 'concat', '/etc/foo.bar', { :owner => 1000 } + end + context 'false' do let(:title) { '/etc/foo.bar' } let(:params) {{ :owner => false }} it 'should fail' do - expect { catalogue }.to raise_error(Puppet::Error, /is not a string/) + expect { catalogue }.to raise_error(Puppet::Error, /\$owner must be a string or integer/) end end end # owner => @@ -243,11 +247,15 @@ it_behaves_like 'concat', '/etc/foo.bar', { :group => 'apenny' } end + context '1000' do + it_behaves_like 'concat', '/etc/foo.bar', { :group => 1000 } + end + context 'false' do let(:title) { '/etc/foo.bar' } let(:params) {{ :group => false }} it 'should fail' do - expect { catalogue }.to raise_error(Puppet::Error, /is not a string/) + expect { catalogue }.to raise_error(Puppet::Error, /\$group must be a string or integer/) end end end # group => From f527236eaaf72ffdf8327baaa22653b747f9c173 Mon Sep 17 00:00:00 2001 From: Johannes Linke Date: Thu, 5 Nov 2015 22:35:24 +0100 Subject: [PATCH 73/76] Fix line endings on windows 7eb14b77 removed files/concatfragments.sh and used concatfragments.rb instead. the fix in acf67807 therefore needs to be applied to this .rb file now. --- .gitattributes | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.gitattributes b/.gitattributes index 2e05fd47..b313555d 100644 --- a/.gitattributes +++ b/.gitattributes @@ -1 +1 @@ -*.sh eol=lf +*.rb eol=lf From a7789fa7e852e21cbe38967495d0d2eed6396876 Mon Sep 17 00:00:00 2001 From: Jeremy Kindy Date: Fri, 18 Sep 2015 10:50:33 -0400 Subject: [PATCH 74/76] MODULES-1678: Add show_diff functionality to concat and concat::fragment --- README.md | 4 ++++ manifests/fragment.pp | 21 +++++++++++---------- manifests/init.pp | 11 +++++++++-- spec/unit/defines/concat_fragment_spec.rb | 15 ++++++++------- spec/unit/defines/concat_spec.rb | 18 ++++++++++++++++++ tests/fragment.pp | 11 ++++++----- tests/init.pp | 11 ++++++----- 7 files changed, 62 insertions(+), 29 deletions(-) diff --git a/README.md b/README.md index e765cb1c..654f7560 100644 --- a/README.md +++ b/README.md @@ -184,6 +184,10 @@ Specifies a destination file for the combined fragments. Valid options: a string Specifies whether to overwrite the destination file if it already exists. Valid options: 'true' and 'false'. Default value: 'true'. +#####`show_diff` + +Specifies whether to set the show_diff parameter for the file resource. Useful for hiding secrets stored in hiera from insecure reporting methods. Valid options: 'true + #####`validate_cmd` Specifies a validation command to apply to the destination file. Requires Puppet version 3.5 or newer. Valid options: a string to be passed to a file resource. Default value: undefined. diff --git a/manifests/fragment.pp b/manifests/fragment.pp index 1e8a5d42..1a5b4782 100644 --- a/manifests/fragment.pp +++ b/manifests/fragment.pp @@ -122,15 +122,16 @@ } file { "${fragdir}/fragments/${order}_${safe_name}": - ensure => $safe_ensure, - owner => $fragowner, - group => $fraggroup, - mode => $fragmode, - source => $source, - content => $content, - backup => $_backup, - replace => true, - alias => "concat_fragment_${name}", - notify => Exec["concat_${target}"] + ensure => $safe_ensure, + owner => $fragowner, + group => $fraggroup, + mode => $fragmode, + source => $source, + content => $content, + backup => $_backup, + show_diff => false, + replace => true, + alias => "concat_fragment_${name}", + notify => Exec["concat_${target}"] } } diff --git a/manifests/init.pp b/manifests/init.pp index 48c21a36..fb593328 100644 --- a/manifests/init.pp +++ b/manifests/init.pp @@ -18,6 +18,9 @@ # The mode of the final file # [*force*] # Enables creating empty files if no fragments are present +# [*show_diff*] +# Use metaparam for files to show/hide diffs for reporting when using eyaml +# secrets. Defaults to true # [*warn*] # Adds a normal shell style comment top of the file indicating that it is # built by puppet @@ -67,6 +70,7 @@ $mode = '0644', $warn = false, $force = false, + $show_diff = true, $backup = 'puppet', $backup_fragments = false, $replace = true, @@ -93,6 +97,7 @@ fail('$warn is not a string or boolean') } validate_bool($force) + validate_bool($show_diff) if ! concat_is_bool($backup) and ! is_string($backup) { fail('$backup must be string or bool!') } @@ -216,6 +221,7 @@ alias => "concat_${name}", source => "${fragdir}/${concat_name}", backup => $backup, + show_diff => $show_diff, } # Only newer versions of puppet 3.x support the validate_cmd parameter @@ -272,8 +278,9 @@ } file { $path: - ensure => absent, - backup => $backup, + ensure => absent, + backup => $backup, + show_diff => $show_diff, } # lint:ignore:quoted_booleans diff --git a/spec/unit/defines/concat_fragment_spec.rb b/spec/unit/defines/concat_fragment_spec.rb index 5c350d16..c6e0cd0c 100644 --- a/spec/unit/defines/concat_fragment_spec.rb +++ b/spec/unit/defines/concat_fragment_spec.rb @@ -44,13 +44,14 @@ should contain_class('concat::setup') should contain_concat(p[:target]) should contain_file("#{fragdir}/fragments/#{p[:order]}_#{safe_name}").with({ - :ensure => safe_ensure, - :owner => id, - :mode => '0640', - :source => p[:source], - :content => p[:content], - :alias => "concat_fragment_#{title}", - :backup => false, + :ensure => safe_ensure, + :owner => id, + :mode => '0640', + :source => p[:source], + :content => p[:content], + :alias => "concat_fragment_#{title}", + :backup => false, + :show_diff => false, }) # The defined() function doesn't seem to work properly with puppet 3.4 and rspec. # defined() works on its own, rspec works on its own, but together they diff --git a/spec/unit/defines/concat_spec.rb b/spec/unit/defines/concat_spec.rb index 6b97c259..1804f238 100644 --- a/spec/unit/defines/concat_spec.rb +++ b/spec/unit/defines/concat_spec.rb @@ -15,6 +15,7 @@ :mode => '0644', :warn => false, :force => false, + :show_diff => true, :backup => 'puppet', :backup_fragments => false, :replace => true, @@ -85,6 +86,7 @@ :alias => "concat_#{title}", :source => "#{fragdir}/#{concat_name}", :validate_cmd => p[:validate_cmd], + :show_diff => p[:show_diff], :backup => p[:backup], :selinux_ignore_defaults => p[:selinux_ignore_defaults], :selrange => p[:selrange], @@ -323,6 +325,22 @@ end end # force => + context 'show_diff =>' do + [true, false].each do |show_diff| + context show_diff do + it_behaves_like 'concat', '/etc/foo.bar', { :show_diff => show_diff } + end + end + + context '123' do + let(:title) { '/etc/foo.bar' } + let(:params) {{ :show_diff => 123 }} + it 'should fail' do + expect { catalogue }.to raise_error(Puppet::Error, /is not a boolean/) + end + end + end # show_diff => + context 'backup =>' do context 'reverse' do it_behaves_like 'concat', '/etc/foo.bar', { :backup => 'reverse' } diff --git a/tests/fragment.pp b/tests/fragment.pp index a2dfaca2..16bf3b6e 100644 --- a/tests/fragment.pp +++ b/tests/fragment.pp @@ -1,9 +1,10 @@ concat { 'testconcat': - ensure => present, - path => '/tmp/concat', - owner => 'root', - group => 'root', - mode => '0664', + ensure => present, + path => '/tmp/concat', + owner => 'root', + group => 'root', + mode => '0664', + show_diff => true, } concat::fragment { '1': diff --git a/tests/init.pp b/tests/init.pp index fd214271..72d065c7 100644 --- a/tests/init.pp +++ b/tests/init.pp @@ -1,7 +1,8 @@ concat { '/tmp/concat': - ensure => present, - force => true, - owner => 'root', - group => 'root', - mode => '0644', + ensure => present, + force => true, + owner => 'root', + group => 'root', + mode => '0644', + show_diff => true, } From 6aebec53cf4a7b1416f79ddd931bd045dc62d87d Mon Sep 17 00:00:00 2001 From: Hunter Haugen Date: Thu, 19 Nov 2015 12:36:31 -0800 Subject: [PATCH 75/76] (maint) Fix line endings --- spec/acceptance/quoted_paths_spec.rb | 84 ++++++++++++++-------------- 1 file changed, 42 insertions(+), 42 deletions(-) diff --git a/spec/acceptance/quoted_paths_spec.rb b/spec/acceptance/quoted_paths_spec.rb index 042425f9..6f9f9fd5 100644 --- a/spec/acceptance/quoted_paths_spec.rb +++ b/spec/acceptance/quoted_paths_spec.rb @@ -1,42 +1,42 @@ -require 'spec_helper_acceptance' - -describe 'quoted paths' do - basedir = default.tmpdir('concat') - - before(:all) do - pp = <<-EOS - file { '#{basedir}': - ensure => directory, - } - file { '#{basedir}/concat test': - ensure => directory, - } - EOS - apply_manifest(pp) - end - - context 'path with blanks' do - pp = <<-EOS - concat { '#{basedir}/concat test/foo': - } - concat::fragment { '1': - target => '#{basedir}/concat test/foo', - content => 'string1', - } - concat::fragment { '2': - target => '#{basedir}/concat test/foo', - content => 'string2', - } - EOS - - it 'applies the manifest twice with no stderr' do - apply_manifest(pp, :catch_failures => true) - apply_manifest(pp, :catch_changes => true) - end - - describe file("#{basedir}/concat test/foo") do - it { should be_file } - its(:content) { should match /string1string2/ } - end - end -end +require 'spec_helper_acceptance' + +describe 'quoted paths' do + basedir = default.tmpdir('concat') + + before(:all) do + pp = <<-EOS + file { '#{basedir}': + ensure => directory, + } + file { '#{basedir}/concat test': + ensure => directory, + } + EOS + apply_manifest(pp) + end + + context 'path with blanks' do + pp = <<-EOS + concat { '#{basedir}/concat test/foo': + } + concat::fragment { '1': + target => '#{basedir}/concat test/foo', + content => 'string1', + } + concat::fragment { '2': + target => '#{basedir}/concat test/foo', + content => 'string2', + } + EOS + + it 'applies the manifest twice with no stderr' do + apply_manifest(pp, :catch_failures => true) + apply_manifest(pp, :catch_changes => true) + end + + describe file("#{basedir}/concat test/foo") do + it { should be_file } + its(:content) { should match /string1string2/ } + end + end +end From b77fa845e63b069c34b143c495f49c37ecb8f01b Mon Sep 17 00:00:00 2001 From: Helen Campbell Date: Mon, 23 Nov 2015 16:18:45 +0000 Subject: [PATCH 76/76] Release 1.2.5 --- CHANGELOG.md | 5 +++++ metadata.json | 4 ++-- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index f24932cb..422c5b17 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,8 @@ +##Supported Release 1.2.5 +###Summary + +Small release for support of newer PE versions. This increments the version of PE in the metadata.json file. + ##2015-07-21 - Supported Release 1.2.4 ###Summary diff --git a/metadata.json b/metadata.json index a7753b20..000b072b 100644 --- a/metadata.json +++ b/metadata.json @@ -1,6 +1,6 @@ { "name": "puppetlabs-concat", - "version": "1.2.4", + "version": "1.2.5", "author": "Puppet Labs", "summary": "Construct files from multiple fragments.", "license": "Apache-2.0", @@ -100,7 +100,7 @@ "requirements": [ { "name": "pe", - "version_requirement": ">= 3.0.0 < 2015.3.0" + "version_requirement": ">= 3.0.0 < 2015.4.0" }, { "name": "puppet",