parse_config and parse_config_file does not show fully info when test fails (or even if it succedes)

[nvtkaszpir]

Description

This is similar issue as #1093 but found out when using parse_config_file.

When using parse_config_file in the test and it fails then the message is missing detailed explanation what actually failed - unless you have insanely detailed tests.

InSpec and Platform Version

inspec 0.35.0
ubuntu 15.10 testing remotely on 14.04

Replication Case

in /tmp/inspec.php5-fpm-status.curl.txt write below code:

pool:                 www
process manager:      static
start time:           25/Sep/2016:08:13:37 +0000
start since:          8030
accepted conn:        5
listen queue:         0
max listen queue:     0
listen queue len:     128
idle processes:       15
active processes:     1
total processes:      16
max active processes: 1
max children reached: 0
slow requests:        0

Then use below test file

title 'php5-fpm status as static pool'

check_url = 'http://127.0.0.1/php_status'
check_auth = ''
process_manager = 'static'

control 'test-parse_config_file-999' do
  impact 1.0
  title 'php5-fpm service status via web - extended'
  desc 'php5-fpm service status via web - extended'

  # commeted out for github bug report
  #curl = 'curl ' + check_url
  #command(curl + ' -o /tmp/inspec.php5-fpm-status.curl.txt').stdout

  options = {
    assignment_re: /^\s*([^:]*?)\s*:\s*(.*?)\s*$/,
    multiple_values: false
  }
  # process the output
  describe parse_config_file('/tmp/inspec.php5-fpm-status.curl.txt', options) do
    its('pool') { should eq 'www'}
    its('process manager') { should eq process_manager }
    its('listen queue') { should be < 100 } # this will fail because output is string and not integer
    its('max children reached') { should eq 0 } # this will fail because output is string and not integer
  end

end

output:

  ✖  php5-fpm-status-extended: php5-fpm service status via web - extended (2 failed)
     ✖  expected: < 100
          got:   "0"
     ✖  
     expected: 0
          got: "0"

     (compared using ==)

     ✔  Parse Config File /tmp/inspec.php5-fpm-status.curl.txt pool should eq "www"
     ✔  Parse Config File /tmp/inspec.php5-fpm-status.curl.txt process manager should eq "static"

Looking at the test output we cannot see what exactly failed.

expected output:

  ✖  php5-fpm-status-extended: php5-fpm service status via web - extended (2 failed)
     ✖  listen queue 
          expected: < 100
          got:   "0"
     ✖  max children reached
          expected: 0
          got: "0"

     (compared using ==)

     ✔  Parse Config File /tmp/inspec.php5-fpm-status.curl.txt pool should eq "www"
     ✔  Parse Config File /tmp/inspec.php5-fpm-status.curl.txt process manager should eq "static"

Of course the issue is that test verifies integers to strings...

Possible Solutions

This is similar issue as #1093

Stacktrace

No stacktrace.

[nvtkaszpir]

This issue also happens when using parse_config with plain command output.

  curl = 'curl http://127.0.0.1/php_status'
  output = command(curl).stdout
  # php5-fpm status putput is in 'key:val' format, and we do not allow multiple values
  options = {
    assignment_re: /^\s*([^:]*?)\s*:\s*(.*?)\s*$/,
    multiple_values: false
  }
  # process the output
  describe parse_config(output, options) do
    its('pool') { should eq 'www'}
    its('process manager') { should eq process_manager }
    its('listen queue') { should be < 100 }
    its('max children reached') { should eq 0 }
  end
end

[nvtkaszpir]

In addition parse_config_file does not show output that it cannot find config file if it is missing. The output is just showing the test was skipped, without message why.

[nvtkaszpir]

also I think it should return more verbose output when using .parse['param'] like below:

control 'php5-fpm-status-extended' do
  impact 1.0
  title 'php5-fpm service status via web - extended'
  desc 'php5-fpm service status via web - extended'
  curl = 'curl http://127.0.0.1/php_status'

  output = command(curl).stdout
  # php5-fpm status putput is in 'key:val' format, and we do not allow multiple values
  options = {
    assignment_re: /^\s*([^:]*?)\s*:\s*(.*?)\s*$/,
    multiple_values: false
  }
  # process the output
  describe parse_config(output, options) do
    its('pool') { should eq 'www'}
    its('process manager') { should eq process_manager }
  end

  describe parse_config(output, options).params['listen queue'].to_i do
    it { should be < 100 }
  end
  describe parse_config(output, options).params['max children reached'].to_i do
    it { should eq 0 }
  end
  # cleanup of the temporary file
  #command('rm -f /tmp/inspec.php5-fpm-status.curl.txt').stdout

end

outputs:

  ✔  php5-fpm-status-extended: php5-fpm service status via web - extended
     ✔  Parse Config  pool should eq "www"
     ✔  Parse Config  process manager should eq "static"
     ✔  0 should be < 100
     ✔  0 should eq 0

The last two lines are mystery :D

[vjeffrey]

@nvtkaszpir just tested with newest inspec (1.1.0) and received this output:

does that meet your expectations for the output?
I believe this was fixed with #1096.

[nvtkaszpir]

looks good.

[vjeffrey]

thanks, closing the issue.