Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add cluster config validation #402

Open
wants to merge 12 commits into
base: master
from

Conversation

Projects
None yet
3 participants
@ericfranz
Copy link
Contributor

commented Sep 24, 2018

Fixes #401

This PR adds a rake task validate:clusters

@@ -59,9 +59,11 @@ gem 'rails_12factor', group: :production
gem 'mocha', '~> 1.1', group: :test
gem "spring", group: :test
gem 'autoprefixer-rails', '~> 8.4'
gem 'json-schema', '~> 2.8'

This comment has been minimized.

Copy link
@MorganRodgers

MorganRodgers Sep 25, 2018

Contributor

Why use a JSON schema validator when there are YAML validators available? For example: kwalify.

This comment has been minimized.

Copy link
@ericfranz

ericfranz Sep 25, 2018

Author Contributor

Oh. I didn't see this. http://www.kuwata-lab.com/kwalify/ looks interesting

This comment has been minimized.

Copy link
@AriettaZ

AriettaZ Sep 27, 2018

Contributor

I used https://www.jsonschema.net/ to generate JSON schema but there's nothing similar for YAML. To save time writing schema(that's hundreds of lines), I think JSON schema validator would be a better choice for now.


if !JSON::Validator.validate(schema, config_json)
puts "Test for '#{config}' FAILED!"
puts JSON::Validator.fully_validate(schema, config_json)

This comment has been minimized.

Copy link
@MorganRodgers

MorganRodgers Sep 25, 2018

Contributor

@ericfranz do we want this to fail on the first error, or run through all the configs and then either succeed or fail at the end?

This comment has been minimized.

Copy link
@ericfranz

ericfranz Sep 25, 2018

Author Contributor

I would expect a validation for an individual cluster config to output a validation failed with the reasons, or validation succeeded, for each cluster.

This comment has been minimized.

Copy link
@MorganRodgers

MorganRodgers Sep 25, 2018

Contributor

Then we'll want to add a failed flag that is set so that we can do something like raise 'One or more cluster configurations are invalid.' at the end.

With an explicit failed return code from the raise this could be tied into a CI system.

@AriettaZ

This comment has been minimized.

Copy link
Contributor

commented Sep 25, 2018

Will raise error at the end if at least one failed

azhu (add-validation *) dashboard $ bundle exec rake  validate:clusters
Validating cluster config: /etc/ood/config/clusters.d/quick.yml...
Test for '/etc/ood/config/clusters.d/quick.yml' FAILED!
The property '#/v2' did not contain a required property of 'login' in schema e1779fe7-65a5-5204-9622-c37479c04caf
The property '#/v2/custom' did not contain a required property of 'moab' in schema e1779fe7-65a5-5204-9622-c37479c04caf
The property '#/v2/custom' did not contain a required property of 'rsv_query' in schema e1779fe7-65a5-5204-9622-c37479c04caf
The property '#/v2/custom' did not contain a required property of 'ganglia' in schema e1779fe7-65a5-5204-9622-c37479c04caf
Validating cluster config: /etc/ood/config/clusters.d/ruby.yml...
Finished validating cluster config: /etc/ood/config/clusters.d/ruby.yml
Validating cluster config: /etc/ood/config/clusters.d/oakley.yml...
Finished validating cluster config: /etc/ood/config/clusters.d/oakley.yml
Validating cluster config: /etc/ood/config/clusters.d/quick_ruby.yml...
Test for '/etc/ood/config/clusters.d/quick_ruby.yml' FAILED!
The property '#/v2' did not contain a required property of 'login' in schema e1779fe7-65a5-5204-9622-c37479c04caf
The property '#/v2/custom' did not contain a required property of 'moab' in schema e1779fe7-65a5-5204-9622-c37479c04caf
The property '#/v2/custom' did not contain a required property of 'rsv_query' in schema e1779fe7-65a5-5204-9622-c37479c04caf
The property '#/v2/custom' did not contain a required property of 'ganglia' in schema e1779fe7-65a5-5204-9622-c37479c04caf
Validating cluster config: /etc/ood/config/clusters.d/quick_pitzer.yml...
Test for '/etc/ood/config/clusters.d/quick_pitzer.yml' FAILED!
The property '#/v2' did not contain a required property of 'login' in schema e1779fe7-65a5-5204-9622-c37479c04caf
The property '#/v2/custom' did not contain a required property of 'moab' in schema e1779fe7-65a5-5204-9622-c37479c04caf
The property '#/v2/custom' did not contain a required property of 'rsv_query' in schema e1779fe7-65a5-5204-9622-c37479c04caf
The property '#/v2/custom' did not contain a required property of 'ganglia' in schema e1779fe7-65a5-5204-9622-c37479c04caf
Validating cluster config: /etc/ood/config/clusters.d/pitzer.yml...
Finished validating cluster config: /etc/ood/config/clusters.d/pitzer.yml
Validating cluster config: /etc/ood/config/clusters.d/owens.yml...
Finished validating cluster config: /etc/ood/config/clusters.d/owens.yml
rake aborted!
One or more cluster configurations are invalid.
/users/PZS0002/azhu/ondemand/dev/dashboard/lib/tasks/validate.rake:14:in `block (3 levels) in <top (required)>'
/users/PZS0002/azhu/ondemand/dev/dashboard/vendor/bundle/ruby/gems/rake-12.3.1/exe/rake:27:in `<top (required)>'
Tasks: TOP => validate:clusters => validate:clusters:selector
(See full trace by running task with --trace)
azhu (add-validation *) dashboard $ bundle exec rake  validate:clusters:/etc/ood/config/clusters.d/quick_pitzer.yml
Validating cluster config: /etc/ood/config/clusters.d/quick_pitzer.yml...
Test for '/etc/ood/config/clusters.d/quick_pitzer.yml' FAILED!
The property '#/v2' did not contain a required property of 'login' in schema e1779fe7-65a5-5204-9622-c37479c04caf
The property '#/v2/custom' did not contain a required property of 'moab' in schema e1779fe7-65a5-5204-9622-c37479c04caf
The property '#/v2/custom' did not contain a required property of 'rsv_query' in schema e1779fe7-65a5-5204-9622-c37479c04caf
The property '#/v2/custom' did not contain a required property of 'ganglia' in schema e1779fe7-65a5-5204-9622-c37479c04caf
rake aborted!
One or more cluster configurations are invalid.
/users/PZS0002/azhu/ondemand/dev/dashboard/lib/tasks/validate.rake:74:in `block (4 levels) in <top (required)>'
/users/PZS0002/azhu/ondemand/dev/dashboard/vendor/bundle/ruby/gems/rake-12.3.1/exe/rake:27:in `<top (required)>'
Tasks: TOP => validate:clusters:/etc/ood/config/clusters.d/quick_pitzer.yml
(See full trace by running task with --trace) 

Pass Argument

// ganglia is spelled wrong in test.yml
azhu (add-validation *) dashboard $ bundle exec rake  validate:clusters[lib/tasks/test.yml]
Validating cluster config: /users/PZS0002/azhu/ondemand/dev/dashboard/lib/tasks/test.yml...
Test for '/users/PZS0002/azhu/ondemand/dev/dashboard/lib/tasks/test.yml' FAILED!
The property '#/v2/custom' did not contain a required property of 'ganglia' in schema e1779fe7-65a5-5204-9622-c37479c04caf
Finished validating cluster config: /users/PZS0002/azhu/ondemand/dev/dashboard/lib/tasks/test.yml
rake aborted!
One or more cluster configurations are invalid.
/users/PZS0002/azhu/ondemand/dev/dashboard/lib/tasks/validate.rake:58:in `block (3 levels) in <top (required)>'
/users/PZS0002/azhu/ondemand/dev/dashboard/vendor/bundle/ruby/gems/rake-12.3.1/exe/rake:27:in `<top (required)>'
Tasks: TOP => validate:clusters => validate:clusters:selector
(See full trace by running task with --trace)

Validate:clusters as a prerequiste of test:jobs

azhu (add-validation *) dashboard $ bundle exec rake test:jobs
Validating cluster config: /etc/ood/config/clusters.d/ruby.yml...
Finished validating cluster config: /etc/ood/config/clusters.d/ruby.yml
Testing cluster 'ruby'...
Finished testing cluster 'ruby'
Validating cluster config: /etc/ood/config/clusters.d/pitzer.yml...
Finished validating cluster config: /etc/ood/config/clusters.d/pitzer.yml
Testing cluster 'pitzer'...
Finished testing cluster 'pitzer'
Validating cluster config: /etc/ood/config/clusters.d/owens.yml...
Finished validating cluster config: /etc/ood/config/clusters.d/owens.yml
Testing cluster 'owens'...
Finished testing cluster 'owens'
Validating cluster config: /etc/ood/config/clusters.d/quick.yml...
Finished validating cluster config: /etc/ood/config/clusters.d/quick.yml
Testing cluster 'quick'...
Finished testing cluster 'quick'
Validating cluster config: /etc/ood/config/clusters.d/quick_ruby.yml...
Finished validating cluster config: /etc/ood/config/clusters.d/quick_ruby.yml
Testing cluster 'quick_ruby'...
Finished testing cluster 'quick_ruby'
Validating cluster config: /etc/ood/config/clusters.d/quick_pitzer.yml...
Finished validating cluster config: /etc/ood/config/clusters.d/quick_pitzer.yml
Testing cluster 'quick_pitzer'...
Finished testing cluster 'quick_pitzer'
Validating cluster config: /etc/ood/config/clusters.d/oakley.yml...
Finished validating cluster config: /etc/ood/config/clusters.d/oakley.yml
Testing cluster 'oakley'...
Finished testing cluster 'oakley'

When validate:clusters failed, test:jobs won't execute:

azhu (add-validation +) dashboard $ bundle exec rake test:jobs:quick
Validating cluster config: /etc/ood/config/clusters.d/quick.yml...
Test for '/etc/ood/config/clusters.d/quick.yml' FAILED!
The property '#/v2' did not contain a required property of 'login' in schema e1779fe7-65a5-5204-9622-c37479c04caf
The property '#/v2/custom' did not contain a required property of 'moab' in schema e1779fe7-65a5-5204-9622-c37479c04caf
The property '#/v2/custom' did not contain a required property of 'rsv_query' in schema e1779fe7-65a5-5204-9622-c37479c04caf
The property '#/v2/custom' did not contain a required property of 'ganglia' in schema e1779fe7-65a5-5204-9622-c37479c04caf
rake aborted!
One or more cluster configurations are invalid.
/users/PZS0002/azhu/ondemand/dev/dashboard/lib/tasks/validate.rake:74:in `block (4 levels) in <top (required)>'
/users/PZS0002/azhu/ondemand/dev/dashboard/vendor/bundle/ruby/gems/rake-12.3.1/exe/rake:27:in `<top (required)>'
Tasks: TOP => test:jobs:quick => validate:clusters:/etc/ood/config/clusters.d/quick.yml
(See full trace by running task with --trace)

AriettaZ added some commits Sep 27, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.