Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New postgres_ident_conf resource #1963

Merged
merged 23 commits into from
Jul 3, 2017
Merged

Conversation

aaronlippold
Copy link
Collaborator

This is a resource for parsing and processing the pg_ident.conf file for a postgres database.

rx294 and others added 10 commits June 23, 2017 02:50
Signed-off-by: Rony Xavier <rx294@nyu.edu>
Signed-off-by: Rony Xavier <rx294@nyu.edu>
… into al/pg_pg_ident

Signed-off-by: Rony Xavier <rx294@nyu.edu>
… into al/pg_pg_ident

Signed-off-by: Rony Xavier <rx294@nyu.edu>
Signed-off-by: Aaron Lippold <lippold@gmail.com>
Signed-off-by: Rony Xaiver <rx294@nyu.edu>
a file.

Signed-off-by: Aaron Lippold <lippold@gmail.com>
added test files and docs

Signed-off-by: Rony Xavier <rx294@nyu.edu>
Signed-off-by: Rony Xavier <rx294@nyu.edu>
Signed-off-by: Rony Xavier <rx294@nyu.edu>
Signed-off-by: Rony Xavier <rx294@nyu.edu>
Signed-off-by: Aaron Lippold <lippold@gmail.com>
Signed-off-by: Rony Xavier <rx294@nyu.edu>
Signed-off-by: Aaron Lippold <lippold@gmail.com>
Signed-off-by: Rony Xavier <rx294@nyu.edu>
Signed-off-by: Rony Xavier <rx294@nyu.edu>
Signed-off-by: Rony Xavier <rx294@nyu.edu>
Signed-off-by: Aaron Lippold <lippold@gmail.com>
Signed-off-by: Aaron Lippold <lippold@gmail.com>
Copy link
Contributor

@adamleff adamleff left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @aaronlippold - thanks for your PR. I've supplied you some feedback to address, please. :)

@@ -122,6 +122,7 @@ def self.validate_resource_dsl_version!(version)
require 'resources/packages'
require 'resources/parse_config'
require 'resources/passwd'
require 'resources/pg_ident_conf'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is global feedback for this whole PR, putting it here since it made sense...

I think the resource name should be postgres_ident_conf... we should avoid abbreviations wherever possible, and we already have other PostgreSQL-related resources that are prefixed with postgres_

# copyright: 2017
# author: Rony Xavier, rx294@nyu.edu
# author: Aaron Lippold, lippold@gmail.com
# license: All rights reserved
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove this line.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

check. will do

attr_reader :params, :conf_dir, :conf_file

def initialize(ident_conf_path = nil)
@conf_dir = inspec.postgres.conf_dir
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no need to set this variable that I can see, especially since you call inspec.postgres.conf_dir directly in the next line.

def initialize(ident_conf_path = nil)
@conf_dir = inspec.postgres.conf_dir
@conf_file = ident_conf_path || File.expand_path('pg_ident.conf', inspec.postgres.conf_dir)
@files_contents = {}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@files_contents is set to an empty hash here, but later when the variable is populated, it's populated with an array. This should likely either be set to nil or an empty array rather than a hash. Ultimately, since this variable isn't really used anywhere, I'd eliminate it completely.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agree we will change it to an array, i think it is just a hold over.

@content = nil
@params = nil
read_content
return skip_resource '`pg_ident_conf` is not yet supported on your OS' if inspec.os.windows?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would reverse lines 28 and 29... don't try and read the file content on Windows if we're just going to skip_resource anyways.

end

def parse_line(line)
x = line.split(' ')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While newer versions of Ruby will also include tab characters as an empty space when splitting like this, it's probably more accurate (and clearer to the reader) to do line.split(/\s+/)

end

def read_file(conf_file = @conf_file)
@files_contents = inspec.file(conf_file).content.lines
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't really see where this instance variable is used in a meaningful way, I think this whole method body can just be:

inspec.file(conf_file).content.lines

@@ -149,6 +149,11 @@ def md.directory?
# Test DH parameters, 2048 bit long safe prime, generator 2 for dh_params in PEM format
'dh_params.dh_pem' => mockfile.call('dh_params.dh_pem'),
'default.toml' => mockfile.call('default.toml'),
'/etc/postgresql/9.5/main/pg_ident.conf' => mockfile.call('pg_ident.conf'),
'C:/etc/postgresql/9.5/main/pg_ident.conf' => mockfile.call('pg_ident.conf'),
'/etc/postgresql/9.5/main' => mockfile.call('9.5.main'),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line, and 155, look like an error to me. Why would InSpec be trying to read in this directory? We shouldn't have to mock this.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, since I can't add a review comment on an empty file, the mock files:

  • test/unit/mock/files/var.9.5.main/.gitkeep
  • test/unit/mock/files/9.5.main/.gitkeep

... shouldn't be necessary either. Let's fix whatever is causing you to add these to this PR.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was added to cover an issue with appvoior given that it is trying to run the unit tests on Windows and failing because we didn't mock for the windows side given that the skip resource was not being respected with FilterTable. This was a work around until we address the open issue #1965 .

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is likely because of your direct call to inspec.postgres.conf_dir in the initialize method. Unit tests should not have to rely on OS or filesystem status as it's about testing the code/messages rather than a functional test.

You'll need to add a mock in your unit tests to cover this as adding these directory mockloader calls isn't appropriate.

It will look something like this (writing this off-the-cuff, haven't tested this):

describe 'blah' do
  let(:resource) { load_resource... }
  let(:inspec) { mock }
  let(:postgres) { mock }

  before do
    resource.stubs(:inspec).returns(inspec)
    inspec.stubs(:postgres).returns(postgres)
    postgres.stubs(:conf_dir).returns('/test/path/to/postgres')
  end

  it 'does something' do
    entries = resource.where ...
  end
end

This should stub/mock the call to inspec.postgres.conf_dir so that it always returns the string /test/path/to/postgres so it works regardless of operating system and whether or not PostgreSQL is installed on the machine performing the unit tests. And then I'd change the helper config to mock out /test/path/to/postgres/pg_ident.conf instead of the one in /etc/postgres... so that you're 100% certain your testing just your code, not some integration point on the system itself.

Search the tests in test/unit for additional examples of how to use mock and stubs. Let me know if you get hung up on this, but we have to square this away before this can be merged.

# copyright: 2017
# author: Aaron Lippold, lippold@gmail.com
# author: Rony Xavier, rx294@nyu.edu
# license: All rights reserved
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove this line.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

require 'helper'
require 'inspec/resource'

describe 'Inspec::Resources::PGIdentConf' do
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to see the tests account for a pg_ident file that has multiple entries to ensure the resource is parsing the file properly and handing back all the lines to FilterTable.

Just adding a second line and changing these must_match statements to match an array of contents would be sufficient for that.

Signed-off-by: Rony Xavier <rx294@nyu.edu>
Signed-off-by: Rony Xavier <rx294@nyu.edu>
@aaronlippold aaronlippold force-pushed the al/pg_pg_ident branch 2 times, most recently from fcb731f to 87e895a Compare June 27, 2017 14:22
… into al/pg_pg_ident

Signed-off-by: Aaron Lippold <lippold@gmail.com>
Copy link
Contributor

@adamleff adamleff left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aaronlippold and @rx294 this is getting closer. I've provided you the tip necessary to stub/mock the calls that are causing you Appveyor issues that we'll need to get squared away in order to eliminate the directory mocks you've added to the unit test helper.rb. Please let me know if you have any questions.

end

def read_content
@content = ''
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line, and line 56, are unnecessary. You already assign these to nil in the initialize method, and it's our mostly-standard practice to keep empty variables set to nil whenever possible.

@@ -149,6 +149,11 @@ def md.directory?
# Test DH parameters, 2048 bit long safe prime, generator 2 for dh_params in PEM format
'dh_params.dh_pem' => mockfile.call('dh_params.dh_pem'),
'default.toml' => mockfile.call('default.toml'),
'/etc/postgresql/9.5/main/pg_ident.conf' => mockfile.call('pg_ident.conf'),
'C:/etc/postgresql/9.5/main/pg_ident.conf' => mockfile.call('pg_ident.conf'),
'/etc/postgresql/9.5/main' => mockfile.call('9.5.main'),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is likely because of your direct call to inspec.postgres.conf_dir in the initialize method. Unit tests should not have to rely on OS or filesystem status as it's about testing the code/messages rather than a functional test.

You'll need to add a mock in your unit tests to cover this as adding these directory mockloader calls isn't appropriate.

It will look something like this (writing this off-the-cuff, haven't tested this):

describe 'blah' do
  let(:resource) { load_resource... }
  let(:inspec) { mock }
  let(:postgres) { mock }

  before do
    resource.stubs(:inspec).returns(inspec)
    inspec.stubs(:postgres).returns(postgres)
    postgres.stubs(:conf_dir).returns('/test/path/to/postgres')
  end

  it 'does something' do
    entries = resource.where ...
  end
end

This should stub/mock the call to inspec.postgres.conf_dir so that it always returns the string /test/path/to/postgres so it works regardless of operating system and whether or not PostgreSQL is installed on the machine performing the unit tests. And then I'd change the helper config to mock out /test/path/to/postgres/pg_ident.conf instead of the one in /etc/postgres... so that you're 100% certain your testing just your code, not some integration point on the system itself.

Search the tests in test/unit for additional examples of how to use mock and stubs. Let me know if you get hung up on this, but we have to square this away before this can be merged.

describe 'Inspec::Resources::PGIdentConf' do
describe 'PGIdentConf Paramaters' do
resource = load_resource('postgres_ident_conf')
it 'Verify postgres_ident_conf filtering by `system_username`' do
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While these are fine tests, this isn't exactly what I was trying to convey with my earlier review. 🙂 Your tests are testing more the .where functionality of FilterTable than you are your parsing engine.

A better more-slimmed-down test might look like this:

it 'parses the pg_ident.conf file correctly' do
  resource.map_name.must_equal ['omicron', 'ssl-test', 'pki-users']
  resource.system_username.must_equal ['bryanh', 'ann', 'robert']
  resource.pg_username.must_equal ['bryanh', 'ann', 'bob']
end

This eliminates any unnecessary calls to FilterTable and focuses solely on this resource's ability to parse your file and populate the FilterTable itself.

You also don't need the two describe statements here... just whittle it down to one and make it match the actual class name of the resource without quotes:

describe Inspec::Resources::PostgresIdentConf do
  it 'parses the pg_ident.conf file correctly' do
    # stuff here
  end
end

Additional describe statements can always be added later to cover individual helper methods, etc.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have made the corrections and added the tests.Please review them when you get a chance. Thank you again @adamleff !

@adamleff adamleff changed the title pg_pg_ident resource New postgres_ident_conf resource Jun 27, 2017
@adamleff adamleff added the Type: New Feature Adds new functionality label Jun 27, 2017
@rx294 rx294 force-pushed the al/pg_pg_ident branch 2 times, most recently from 73d19d8 to 6150438 Compare June 28, 2017 12:05
Copy link
Contributor

@adamleff adamleff left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good - thanks @aaronlippold and @rx294!

Copy link
Contributor

@chris-rock chris-rock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome addition @rx294 and @aaronlippold I just have a super mini change-request. Once done, LGTM

@@ -0,0 +1,77 @@
# encoding: utf-8
# copyright: 2017
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All files will be apache licensed, lets remove the empty copyright here

Signed-off-by: Aaron Lippold <lippold@gmail.com>
Signed-off-by: Aaron Lippold <lippold@gmail.com>
@aaronlippold
Copy link
Collaborator Author

@chris-rock changes made.

@chris-rock chris-rock merged commit 57864f1 into inspec:master Jul 3, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Type: New Feature Adds new functionality
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants