Why is the yml file required?

[mhedgpeth]

I'm learning the code and the tool and wanted to start out with a hello_world_spec.rb. When I did that, I got an error that the metadata.rb is not there. Looking at the code, it's really looking for a yml file. I suggest that this should be changed.

This exposes what I see to be a major architectural question: is inspec a testing framework that can be cataloged within a compliance language or is inspec a compliance framework that uses tests? I believe (from my recent conversation with Dominic) that it's the former.

If that's the case, then the workflow of learning inspec should start with the test. Start out with a hello world test that make sure 80 isn't running on one's laptop. That should be: create a file and write the test in four lines, then run inspec against that folder. It should run the tests!

Then step 2 is I want to add some compliance decorators to the tests. Doing that should work, with no other work!

Then step 3 is to package those tests into a policy. That should require the yml file so it can be catalogued correctly.

This way you get a really nice learning workflow and start people at the natural center of inspec as I see it: test -> control -> compliance policy

What do you think?

[chris-rock]

@mhedgpeth you describe inspec design. Therefore perfect match 👍

Double-check that you have the latest version installed

inspec version  
0.9.11

Step 1:

$ cat > port_test.rb << EOF
describe port(80) do
  it { should be_listening }
end
EOF
$ inspec exec port_test.rb 
F

Failures:

  1) Port  80 should be listening
     Failure/Error: with.run_specs(tests)
       expected `Port  80.listening?` to return true, got false
     # port_test.rb:2:in `block (2 levels) in load'
     # ./lib/inspec/runner_rspec.rb:55:in `run'

Finished in 0.13433 seconds (files took 0.47616 seconds to load)
1 example, 1 failure

Failed examples:

rspec  # Port  80 should be listening

Step 2:

Adapt the test and re-run it with inspec inspec exec port_test.rb

control 'port-80-rule' do 
  title 'port 80 is active'
  desc 'my web app should be running'
  impact 1.0
  describe port(80) do
    it { should be_listening }
  end
end

[chris-rock]

Step 3

mkdir -p port-profile/controls
mv port_test.rb port-profile/controls
cd port-profile 
cat > inspec.yml << EOF
name: port-profile
title: Web Server Tests
summary: Verifies that my Web Server is properly configured
version: 1.0.0
EOF

# check the profile
$ inspec check port-profile 
I, [2016-02-03T20:09:25.762733 #24237]  INFO -- : Checking profile in port-profile
W, [2016-02-03T20:09:25.762871 #24237]  WARN -- : Missing profile maintainer in inspec.yml
W, [2016-02-03T20:09:25.762899 #24237]  WARN -- : Missing profile copyright in inspec.yml
I, [2016-02-03T20:09:25.762955 #24237]  INFO -- : Found 1 rules.
I, [2016-02-03T20:09:25.762975 #24237]  INFO -- : Rule definitions OK.

# execute the profile
$ inspec exec port-profile 
F

Failures:

  1) Port  80 should be listening
     Failure/Error: with.run_specs(tests)
       expected `Port  80.listening?` to return true, got false
     # port-profile/controls/port_test.rb:6:in `block (3 levels) in load'
     # ./lib/inspec/runner_rspec.rb:55:in `run'

Finished in 0.07905 seconds (files took 0.3315 seconds to load)
1 example, 1 failure

Failed examples:

rspec  # Port  80 should be listening

# archive profile
inspec archive port-profile 
I, [2016-02-03T20:10:37.781275 #24265]  INFO -- : Checking profile in port-profile
W, [2016-02-03T20:10:37.781414 #24265]  WARN -- : Missing profile maintainer in inspec.yml
W, [2016-02-03T20:10:37.781431 #24265]  WARN -- : Missing profile copyright in inspec.yml
I, [2016-02-03T20:10:37.781463 #24265]  INFO -- : Found 1 rules.
I, [2016-02-03T20:10:37.781485 #24265]  INFO -- : Rule definitions OK.
I, [2016-02-03T20:10:37.781611 #24265]  INFO -- : Profile check finished. Generate archive /Users/chartmann/Development/compliance/inspec/port-profile.tar.gz.
I, [2016-02-03T20:10:37.786196 #24265]  INFO -- : Finished archive generation.

Now you are able to upload the profile to Chef Compliance.

[chris-rock]

Is that what you are thinking of? I would love to understand what went wrong, because that should not happen.

[mhedgpeth]

Chris, it was a pathing problem. I was using the ChefDK version but within the path of the code I pulled from master and made that mistake. Sorry about that. The steps above is exactly what I'm looking for and what Dominic and I were talking about at CfgMgmtCamp on Tuesday. I'm really excited about this tool.

[chris-rock]

anything we can do to improve the situation with ChefDK?

[mhedgpeth]

Oh you're fine, it was just a noob move on my part.

[chris-rock]

@mhedgpeth Thanks for reporting, if you experience any further issues, please keep us posted.

[arlimus]

Btw: I find the whole requirement on inspec.yml an interesting topic. If supporting inspec without that comes up as a requirement anywhere, please open an issue and let us know exactly you're doing.

Thank you @chris-rock for that awesome help! 👍

[mhedgpeth]

See my attempts here: https://github.com/mhedgpeth/inspec/tree/better_noob_documentation

I think the problem is that you really have 3 main things happening and the flow isn't easy and it's packed into one executable.

1. Infrastructure tests - I run with inspec test . in a folder where my tests are
2. Compliance tests - I run the same command but this is for the compliance/security workflow, not always the one we're going for
3. Portable policy - I run inspec verify . to check that the policy is valid or inspec json . to export it to JSON. This is where the yml file comes in and it's confusing if you're still at shared linux file handling + specinfra config + cleanup #1. Also it's not clear to me how you would export a policy and import it back in. Is that outside of the scope?

I think in the documentation it might be nice to see something like this:

In Chef --- this equivilent exists in inspec
Chef resource - single compliance test or compliance decorator
Chef recipe - test file/compliance file
Chef cookbook - policy