Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release 1.0 branch #49

Closed
wants to merge 93 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
93 commits
Select commit Hold shift + click to select a range
aff59c3
Create catalog-compile, start moving libraries around
Jan 3, 2017
c7f7336
More class renames
Jan 3, 2017
5fe57f2
Update paths and classes in tests
Jan 3, 2017
652bc32
Update more paths
Jan 3, 2017
4efc7a1
Change CLI class --catalog-only over to the API
Jan 3, 2017
9653424
Rename the API to v1
Jan 3, 2017
f47fc33
Update integration tests to match new API
Jan 3, 2017
8828b00
Add spec test for catalog compile API v1
Jan 3, 2017
dbb9b3d
Add pass-through top level API with methods
Jan 3, 2017
0149171
Move catalog-diff from CLI to new API class
Jan 3, 2017
cf82fcf
Add spec tests for catalog-diff and common
Jan 3, 2017
3d137ec
Merge remote-tracking branch 'origin/master' into kpaulisse-better-api
Jan 4, 2017
9b2cf04
Add spec test for --bootstrap-then-exit clause
Jan 4, 2017
559596f
Stub documentation
Jan 4, 2017
0300de9
Make octocatalog-diff config file part of the API
Jan 4, 2017
af85f11
Move CLI to the top level
Jan 4, 2017
7bc7749
Re-namespace the CLI classes
Jan 4, 2017
6f29688
Add integration test for the CLI
Jan 4, 2017
fb248b3
Call correct classes from CLI
Jan 4, 2017
6dd8ea2
Move octocatalog-diff/catalog-diff/cli -> lib/octocatalog-diff/cli
Jan 7, 2017
57290c8
Move cli/catalogs -> util/catalogs
Jan 7, 2017
6768f58
Merge pull request #47 from github/kpaulisse-split-cli
kpaulisse Jan 7, 2017
503b668
Merge branch 'kpaulisse-release-0-7' into kpaulisse-split-cli-catalogs
kpaulisse Jan 7, 2017
501cf24
Merge branch 'kpaulisse-split-cli-catalogs' into kpaulisse-better-api
Jan 7, 2017
757ce28
Merge branch 'kpaulisse-release-0-7' into kpaulisse-better-api
Jan 7, 2017
5d9e0f4
Merge pull request #48 from github/kpaulisse-split-cli-catalogs
kpaulisse Jan 7, 2017
67e88f1
Merge branch 'kpaulisse-release-0-7' into kpaulisse-better-api
Jan 7, 2017
6e03afe
Merge branch 'kpaulisse-yaml-diff' into kpaulisse-yaml-diff-0-7
Jan 7, 2017
d7f2661
Move and update filters option to new structure
Jan 7, 2017
3c8092d
Update documentation links
Jan 7, 2017
042c12a
Update link within this document as well
Jan 7, 2017
479ad83
Correct typo in link, oops
Jan 7, 2017
7262c47
Merge pull request #50 from github/kpaulisse-yaml-diff-0-7
kpaulisse Jan 7, 2017
4769985
Merge branch 'kpaulisse-release-0-7' into kpaulisse-better-api
kpaulisse Jan 7, 2017
e42f430
Start spec test for API v1 config
Jan 8, 2017
6449f4d
Add additional test coverage
Jan 8, 2017
89cb1de
Finish test coverage for OctocatalogDiff API config
Jan 8, 2017
f868a7a
Centralize errors to their own class
Jan 8, 2017
a4c95ca
Create documentation for `.config` API
Jan 8, 2017
7a1673f
Bigger headings, misc. wording
Jan 8, 2017
6aa1920
Move CatalogError, BootstrapError to error class
Jan 8, 2017
7de36fc
Update tests for error classes
Jan 8, 2017
880f279
Stub documentation
Jan 8, 2017
72e7264
Move fact source and retrieval errors to error class
Jan 8, 2017
6689007
Move PuppetDB errors to error class
Jan 8, 2017
2ede61f
Move catalog compilation errors to errors class
Jan 8, 2017
f6b5483
Move remaining errors
Jan 8, 2017
0f5e49f
Ruby variability, accept either error message
Jan 8, 2017
b104b30
Merge pull request #44 from github/kpaulisse-better-api
kpaulisse Jan 8, 2017
851171f
Merge branch 'kpaulisse-release-0-7' into kpaulisse-catalog-object-api
Jan 8, 2017
09fa9ea
Merge remote-tracking branch 'origin/kpaulisse-release-0-7' into kpau…
Jan 8, 2017
3ac4575
Create wrapper class around OctocatalogDiff::Catalog
Jan 8, 2017
8f54ac7
Initial commit of the Diff object
Jan 8, 2017
02954fa
Make catalog-diff return API objects
Jan 8, 2017
27e9a59
Update documentation for new objects
Jan 8, 2017
8482345
Add, test, and document change type methods
Jan 8, 2017
b553c88
Add documentation and test for structure
Jan 8, 2017
bc966c7
Document and test #old_value and #new_value
Jan 8, 2017
19be23b
Add new/old file/line tests and documentation
Jan 8, 2017
286d218
Add documentation and test for #to_h
Jan 8, 2017
9a4ec46
Finish coverage and documentation of `diff` API object
Jan 8, 2017
1b74f52
change_type -> diff_type
Jan 8, 2017
14f5d31
Wrapper spec tests for API catalog
Jan 8, 2017
e935ee7
Add test and stub documentation for catalog object
Jan 8, 2017
d4c97ca
Finish API catalog documentation
Jan 8, 2017
b4cfb75
Merge pull request #51 from github/kpaulisse-catalog-object-api
kpaulisse Jan 8, 2017
7a6199e
Create spec coverage for catalog-utils/git
Jan 8, 2017
bafc777
Error rename and coverage for bootstrap class
Jan 8, 2017
c2e254f
Add coverage of #run_bootstrap
Jan 8, 2017
e2e822b
Add additional test coverage for util/catalogs
Jan 8, 2017
2fc2b20
Finish test for catalogs
Jan 8, 2017
7a6ab3e
Merge pull request #52 from github/kpaulisse-test-coverage
kpaulisse Jan 8, 2017
3f79ac7
Add example for catalog compilation
Jan 9, 2017
ff2c178
Document some parameters for catalog
Jan 9, 2017
006c1a7
Add note in options reference about using parameters in the API
Jan 9, 2017
35eb733
Update catalog doc - you can use the config API
Jan 9, 2017
8010b54
Update the --catalog-only code to use the API catalog object
Jan 9, 2017
e569ea9
Switch from raw diff object to API diff object
Jan 9, 2017
28d3231
Update YAML integration to use `diff_match?` check
Jan 9, 2017
1c1c507
Update test to use `.diff_match?`
Jan 9, 2017
f47c864
Fix more tests to use `.diff_match?`
Jan 9, 2017
afd4d9c
Convert more integration tests to `.diff_match?`
Jan 9, 2017
900ecb4
Convert more integration tests to `.diff_match?`
Jan 9, 2017
816b2ea
Ignore errors from directory cleanup
Jan 9, 2017
67aa476
Merge pull request #53 from github/kpaulisse-catalog-api-doc
kpaulisse Jan 9, 2017
640152f
No test coverage of the remove_entry_secure issue
Jan 9, 2017
20ed698
Merge branch 'kpaulisse-release-0-7' into kpaulisse-catalog-api-use-i…
kpaulisse Jan 9, 2017
d5bd021
Rename file so it gets run and add test for `.config` method
Jan 9, 2017
2eae673
Merge pull request #54 from github/kpaulisse-catalog-api-use-internally
kpaulisse Jan 9, 2017
173c356
Merge branch 'kpaulisse-release-0-7' into kpaulisse-diff-api-use-inte…
kpaulisse Jan 9, 2017
1c2072f
Merge pull request #55 from github/kpaulisse-diff-api-use-internally
kpaulisse Jan 10, 2017
fe37f3e
Merge branch 'kpaulisse-release-0-7' into kpaulisse-spec-api-v1
kpaulisse Jan 10, 2017
cdd7031
Merge pull request #56 from github/kpaulisse-spec-api-v1
kpaulisse Jan 10, 2017
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Expand Up @@ -51,6 +51,7 @@ The example above reflects the changes in the Puppet catalog from switching an u
- [Requirements](/doc/requirements.md)
- [Limitations](/doc/limitations.md)
- [List of all command line options](/doc/optionsref.md)
- [API](/doc/dev/api.md)

### Project

Expand Down
55 changes: 7 additions & 48 deletions bin/octocatalog-diff
Expand Up @@ -20,56 +20,15 @@ end

config_test = ARGV.include?('--config-test')

paths = [
ENV['OCTOCATALOG_DIFF_CONFIG_FILE'],
File.join(Dir.pwd, '.octocatalog-diff.cfg.rb'),
File.join(ENV['HOME'], '.octocatalog-diff.cfg.rb'),
'/opt/puppetlabs/octocatalog-diff/octocatalog-diff.cfg.rb',
'/usr/local/etc/octocatalog-diff.cfg.rb',
'/etc/octocatalog-diff.cfg.rb'
]
logger = Logger.new(STDERR)
logger.level = Logger::DEBUG if config_test

cfgfile = nil

paths.each do |path|
next if path.nil?
puts "Looking for config in: #{path}" if config_test
next unless File.file?(path)
begin
cfgfile = path
puts "Loading config: #{path}" if config_test
require path
rescue => exc
STDERR.puts "#{exc.class} error with #{path}: #{exc.message}\n#{exc.backtrace}"
exit 1
end
break
end

options = {}

if cfgfile
begin
options = OctocatalogDiff::Config.config
rescue => exc
STDERR.puts "#{exc.class} error with #{cfgfile}: #{exc.message}\n#{exc.backtrace}"
exit 1
end
unless options.is_a?(Hash)
STDERR.puts "Configuration must be Hash not #{options.class}!"
exit 1
end
if config_test
options.each do |key, val|
puts ":#{key} => (#{val.class}) #{val.inspect}"
end
exit 0
end
elsif config_test
STDERR.puts 'No configuration files found'
exit 1
options = OctocatalogDiff::API::V1.config(logger: logger, test: config_test)
if config_test
logger.info 'Exiting now because --config-test was specified'
exit(0)
end

argv = ARGV.dup
exit_code = OctocatalogDiff::CatalogDiff::Cli.cli(argv, Logger.new(STDERR), options)
exit_code = OctocatalogDiff::Cli.cli(argv, Logger.new(STDERR), options)
exit(exit_code)
2 changes: 1 addition & 1 deletion doc/advanced-filter.md
Expand Up @@ -9,7 +9,7 @@ Please note that there are other options to ignore specified diffs, including:

Here is the list of available filters and an explanation of each:

- [YAML](#YAML) - Ignore whitespace/comment differences if YAML parses to the same object
- [YAML](/doc/advanced-filter.md#yaml) - Ignore whitespace/comment differences if YAML parses to the same object

## YAML

Expand Down
2 changes: 1 addition & 1 deletion doc/advanced-pe-enc.md
Expand Up @@ -39,7 +39,7 @@ Please see [Authentication token](https://docs.puppet.com/pe/latest/nc_forming_r

The referenced document contains links to generate a token with the `puppet-access` command.

Note that if you wish to hard-code an authentication token in your [configuration file](/doc/configuration.md), the internal variable key is `:pe_enc_token` and the content is a string containing the entire token. (The `--pe-enc-token-file` option simply reads the provided file and stores the content in the `:pe_enc_token` key. See [source](/lib/octocatalog-diff/catalog-diff/cli/options/pe_enc_token_file.rb).)
Note that if you wish to hard-code an authentication token in your [configuration file](/doc/configuration.md), the internal variable key is `:pe_enc_token` and the content is a string containing the entire token. (The `--pe-enc-token-file` option simply reads the provided file and stores the content in the `:pe_enc_token` key. See [source](/lib/octocatalog-diff/cli/options/pe_enc_token_file.rb).)

### SSL client keypair

Expand Down
5 changes: 5 additions & 0 deletions doc/dev/api.md
@@ -0,0 +1,5 @@
# octocatalog-diff API documentation

The octocatalog-diff API allows developers to construct and work with certain octocatalog-diff internals in their own projects.

The current API version is [API v1](/doc/dev/api/v1.md), which is available as of octocatalog-diff version 0.7.
35 changes: 35 additions & 0 deletions doc/dev/api/v1.md
@@ -0,0 +1,35 @@
# octocatalog-diff v1 API documentation

## API calls

#### catalog

`catalog` allows you to build a catalog using the octocatalog-diff compiler or to obtain a catalog from a Puppet server.

[Read more about the `catalog` call](/doc/dev/api/v1/calls/catalog.md)

#### catalog-diff

`catalog-diff` allows you compare two catalogs and obtain the differences between them. Catalogs can be built if necessary.

[Read more about the `catalog-diff` call](/doc/dev/api/v1/calls/catalog-diff.md)

#### config

`config` allows you read and parse an [octocatalog-diff configuration file](/doc/configuration.md).

[Read more about the `config` call](/doc/dev/api/v1/calls/config.md)

## Objects

#### OctocatalogDiff::API::V1::Catalog

The `OctocatalogDiff::API::V1::Catalog` object represents a compiled catalog and supports several methods to get information about the catalog.

[Read more about the `OctocatalogDiff::API::V1::Catalog` object](/doc/dev/api/v1/objects/catalog.md)

#### OctocatalogDiff::API::V1::Diff

The `OctocatalogDiff::API::V1::Diff` object represents a difference between two catalogs and supports several methods to get information about the difference.

[Read more about the `OctocatalogDiff::API::V1::Diff` object](/doc/dev/api/v1/objects/diff.md)
37 changes: 37 additions & 0 deletions doc/dev/api/v1/calls/catalog-diff.md
@@ -0,0 +1,37 @@
# octocatalog-diff v1 API documentation: catalog-diff

## Overview

`catalog-diff` allows you compare two catalogs and obtain the differences between them. Catalogs can be built if necessary.

This is analogous to using the default arguments with the octocatalog-diff command-line script.

```
catalog_diff_result = OctocatalogDiff::API::V1.catalog_diff(

)
```

## Options

**NOTE**: Additional options as described in the [options reference](/doc/optionsref.md) may also have an effect on catalog difference generation.

## Return value

The return value is a structure in the following format:

```
{
diffs: Array<OctocatalogDiff::API::V1::Diff>,
from: OctocatalogDiff::API::V1::Catalog,
to: OctocatalogDiff::API::V1::Catalog
}
```

It is possible to query this object as a hash (e.g. `result[:diffs]`) or using methods (e.g. `result.diffs`).

[Read more about the `OctocatalogDiff::API::V1::Catalog` object](/doc/dev/api/v1/objects/catalog.md)

[Read more about the `OctocatalogDiff::API::V1::Diff` object](/doc/dev/api/v1/objects/diff.md)

## Exceptions
115 changes: 115 additions & 0 deletions doc/dev/api/v1/calls/catalog.md
@@ -0,0 +1,115 @@
# octocatalog-diff v1 API documentation: catalog

## Overview

`catalog` returns an `OctocatalogDiff::Catalog` object built with the octocatalog-diff compiler, obtained from a Puppet server, or read in from a file. This is analogous to using the `--catalog-only` option with the octocatalog-diff command-line script.

```
catalog_obj = OctocatalogDiff::API::V1.catalog(<Hash>)
#=> OctocatalogDiff::API::V1::Catalog
```

The return value is an [`OctocatalogDiff::API::V1::Catalog`](/doc/dev/api/v1/objects/catalog.md) object.

For an example, see [catalog-builder-local-files.rb](/examples/api/v1/catalog-builder-local-files.rb).

## Parameters

The `catalog` method takes one argument, which is a Hash containing parameters.

The list of parameters here is not exhaustive. The `.catalog` method accepts most parameters described in [Configuration](/doc/configuration.md), [Building catalogs instead of diffing catalogs](/doc/advanced-catalog-only.md), and [Command line options reference](/doc/optionsref.md).

It is also possible to use the parameters from [OctocatalogDiff::API::V1.config](/doc/dev/api/v1/calls/config.md) for the catalog compilation. Simply combine the hash returned by `.config` with any additional keys, and pass the merged hash to the `.catalog` method.

### Global parameters

#### `:logger` (Logger, Optional)

Debugging and informational messages will be logged to this object as the catalog is built.

If no Logger object is passed, these messages will be silently discarded.

#### `:node` (String, Required)

The node name whose catalog is to be compiled or obtained. This should be the fully qualified domain name that matches the node's name as seen in Puppet.

### Computed catalog parameters

#### `:basedir` (String, Optional)

Directory that contains a git repository with the Puppet code. Use in conjunction with `:to_branch` to specify the branch name that should be checked out.

If your Puppet code is not in a git repository, or you already have the branch checked out via some other process, use `:bootstrapped_to_dir` instead.

#### `:bootstrap_script` (String, Optional)

Path to a script to run after checking out the selected branch of Puppet code from the git repository. See [Bootstrapping your Puppet checkout](/doc/advanced-bootstrap.md) for details.

#### `:bootstrapped_to_dir` (String, Optional)

Directory that is already prepared ("bootstrapped") and can have Puppet run against its contents to compile the catalog.

Often this means that you have done the following to this directory:

- Checked out the necessary branch of Puppet code from your version control system
- Installed any third party modules (e.g. with librarian-puppet or r10k)

#### `:enc` (String, Optional)

File path to your External Node Classifier.

See [Configuring octocatalog-diff to use ENC](/doc/configuration-enc.md) for details on using octocatalog-diff with an External Node Classifier.

#### `:fact_file` (String, Optional)

Path to the file that contains the facts for the node whose catalog you are going to compile.

Generally, must either specify the fact file, or [configure octocatalog-diff to use PuppetDB](/doc/configuration-puppetdb.md) to retrieve node facts.

#### `:hiera_config` (String, Optional)

Path to the Hiera configuration file (generally named `hiera.yaml`) for your Puppet installation. Please see [Configuring octocatalog-diff to use Hiera](/doc/configuration-hiera.md) for details on Hiera configuration.

#### `:hiera_path` (String, Optional)

Directory within your Puppet installation where Hiera data is stored. Please see [Configuring octocatalog-diff to use Hiera](/doc/configuration-hiera.md) for details on Hiera configuration.

If your Puppet setup is modeled after the [Puppet control repository template](https://github.com/puppetlabs/control-repo), the correct setting for `:hiera_path` is `'hieradata'`.

#### `:puppet_binary` (String, Required)

Path to the Puppet binary on your system.

Please refer to [Configuring octocatalog-diff to use Puppet](/doc/configuration-puppet.md) for details of connecting octocatalog-diff to your Puppet installation.

#### `:puppetdb_url` (String, Optional)

URL to PuppetDB. See [Configuring octocatalog-diff to use PuppetDB](/doc/configuration-puppetdb.md) for instructions on this setting, and other settings that may be needed in your environment.

#### `:to_branch` (String, Optional)

Branch name in git repository to use for Puppet code. This option must be used in conjunction with `:basedir` so that code can be located.

## Exceptions

The following exceptions may occur during the compilation of a catalog:

- `OctocatalogDiff::Errors::BootstrapError`

Bootstrapping failed.

- `OctocatalogDiff::Errors::CatalogError`

Catalog failed to compile. Please note that whenever possible, a `OctocatalogDiff::API::V1::Catalog` object is still constructed for a failed catalog, with `#valid?` returning false.

- `OctocatalogDiff::Errors::GitCheckoutError`

Git checkout failed.

- `OctocatalogDiff::Errors::PuppetVersionError`

The version of Puppet could not be determined, generally because the Puppet binary was not found, or does not respond as expected to `puppet version`.

- `OctocatalogDiff::Errors::ReferenceValidationError`

See [Catalog validation](/doc/advanced-catalog-validation.md).
37 changes: 37 additions & 0 deletions doc/dev/api/v1/calls/config.md
@@ -0,0 +1,37 @@
# octocatalog-diff v1 API documentation: config

## Overview

`config` reads and parses an [octocatalog-diff configuration file](/doc/configuration.md).

```
options = OctocatalogDiff::API::V1.config(
filename: "String",
logger: Logger,
test: <true|false>
)
```

## Options

- **`:filename`** (String, optional): Full path to configuration file to read. If not provided, the configuration file will be searched as described in [Configuration](/doc/configuration.md).

- **`:logger`** (Logger, optional): Logger object. If provided, debug messages and fatal errors will be logged to this object.

- **`:test`** (Boolean, optional): Test mode, defaults to false. If true, the value of the configuration settings will be logged to the logger (with priority DEBUG) and an exception will be raised if the configuration file cannot be located.

## Return value

If the configuration file is located and valid, the return value is a Hash consisting of the options defined in the configuration file.

If the configuration file cannot be found, the return value is an empty Hash (`{}`). Except, with `:test => true`, an exception will be raised.

## Exceptions

- `OctocatalogDiff::Errors::ConfigurationFileContentError`

Raised if the configuration file could not be evaluated. A more specific error message will help identify the cause. Possible causes include the file not being valid ruby, the file not containing the expected structure or methods, or the method returning something other than a Hash.

- `OctocatalogDiff::Errors::ConfigurationFileNotFoundError`

Raised if the configuration file could not be found, *and* `:test => true` was supplied. (Note, if no configuration file is found, but `:test => false`, no error is raised, and `{}` is returned.)