Permalink
Browse files

Merge branch 'next'

This marks the end of the agile iteration from 11/3-11/10.
  • Loading branch information...
2 parents af8e4d6 + 755b6c1 commit 308dcbbfc892ccbaf13b3be4d997c6cdcb6b44af Paul Berry committed Nov 11, 2010
Showing with 743 additions and 467 deletions.
  1. +1 −1 .gitignore
  2. +25 −9 README.markdown
  3. +0 −5 app/controllers/statuses_controller.rb
  4. +4 −0 app/helpers/application_helper.rb
  5. +12 −58 app/models/node.rb
  6. +3 −1 app/models/node_class.rb
  7. +6 −2 app/models/node_group.rb
  8. +5 −14 app/models/report.rb
  9. +6 −9 app/models/status.rb
  10. +4 −4 app/views/node_classes/show.html.haml
  11. +25 −22 app/views/node_groups/show.html.haml
  12. +0 −12 app/views/nodes/_node_classes.html.haml
  13. +0 −7 app/views/nodes/_node_groups.html.haml
  14. +11 −5 app/views/nodes/_nodes.html.haml
  15. +11 −34 app/views/nodes/show.html.haml
  16. +4 −0 app/views/puppet/util/logs/_log.haml
  17. +32 −1 app/views/reports/_report.html.haml
  18. +4 −1 app/views/reports/_report_status_icon.html.haml
  19. +20 −0 app/views/shared/_classes.html.haml
  20. +20 −0 app/views/shared/_groups.html.haml
  21. +3 −2 app/views/shared/_node_manager_sidebar.html.haml
  22. +29 −0 app/views/shared/_parameters.html.haml
  23. +0 −11 app/views/statuses/_overview.html.haml
  24. +5 −2 app/views/statuses/_run_failure.html.haml
  25. +0 −1 app/views/statuses/overview.html.haml
  26. +24 −7 bin/external_node
  27. +4 −0 config/environment.rb
  28. +0 −2 config/routes.rb
  29. +4 −0 config/settings-sample.yml
  30. +56 −0 db/migrate/20101109001012_add_status_to_reports.rb
  31. +6 −6 db/schema.rb
  32. +38 −5 ext/passenger/dashboard-vhost.conf
  33. +0 −7 lib/node_group_cycle_error.rb
  34. +131 −9 lib/node_group_graph.rb
  35. +16 −5 lib/puppet/report.rb
  36. +12 −22 lib/puppet_https.rb
  37. +60 −46 lib/tasks/install.rake
  38. +12 −1 lib/tasks/mail_patches.rake
  39. BIN public/images/icons/changed.png
  40. BIN public/images/icons/{failure.png → failed.png}
  41. BIN public/images/icons/success.png
  42. BIN public/images/icons/unchanged.png
  43. +8 −5 public/javascripts/application.js
  44. +3 −0 public/stylesheets/application.css
  45. +5 −0 public/stylesheets/sass/application.scss
  46. +8 −0 public/stylesheets/tables.css
  47. +1 −1 spec/controllers/nodes_controller_spec.rb
  48. +0 −4 spec/controllers/statuses_controller_spec.rb
  49. +19 −6 spec/lib/puppet/report_spec.rb
  50. +1 −1 spec/models/node_group_spec.rb
  51. +65 −107 spec/models/node_spec.rb
  52. +31 −9 spec/models/report_spec.rb
  53. +6 −6 spec/support/factories.rb
  54. +0 −14 spec/views/nodes/_node_classes.html.haml_spec.rb
  55. +1 −1 spec/views/reports/_report_status_icon.html.haml_spec.rb
  56. +1 −1 spec/views/reports/_report_status_td.html.haml_spec.rb
  57. +1 −1 spec/views/reports/show.html.haml_spec.rb
View
@@ -14,7 +14,7 @@ db/*.sqlite3
db/dump.sql
doc/api
doc/app
-log/*.log
+log/*.log*
log/*.pid
public/javascripts/all.js
public/stylesheets/all.css
View
@@ -182,7 +182,9 @@ There are many ways to run a Ruby web application like the Puppet Dashboard, we
./script/server -p 8080
-2. **Passenger**: This plugin for [Apache](http://httpd.apache.org/) or [Nginx](http://nginx.org/) makes it easy to run multiple Ruby web apps quickly and efficiently using multiple instances -- it's great for production use. If used along with Ruby Enterprise Edition, it can dramatically reduce the memory required to run Ruby web applications. For further information, please see [Passenger/](http://www.modrails.com/) and [Ruby Enterprise Edition](http://www.rubyenterpriseedition.com/) and the example apache configuration in `ext/passenger/dashboard-vhost.conf`.
+2. **Passenger**: This plugin for [Apache](http://httpd.apache.org/) or [Nginx](http://nginx.org/) makes it easy to run multiple Ruby web apps quickly and efficiently using multiple instances -- it's great for production use. If used along with Ruby Enterprise Edition, it can dramatically reduce the memory required to run Ruby web applications. For further information, including installation and configuration instructions please see [Passenger/](http://www.modrails.com/) and [Ruby Enterprise Edition](http://www.rubyenterpriseedition.com/) and the example apache configuration in `ext/passenger/dashboard-vhost.conf`.
+
+You will need to change the Passenger path depending on the version of Passenger you have installed. Also, DocumentRoot, ServerName, and Directory must be changed to match your Puppet Dashboard install directory.
3. **Thin**: This fast and reliable server can run multiple instances of the Puppet Dashboard application behind a proxy like [Apache](http://httpd.apache.org/) or [Nginx](http://nginx.org/) to appear as a single website -- it's great for production use. For further information, please see [Thin](http://code.macournoyer.com/thin/).
@@ -220,15 +222,26 @@ The Puppet Dashboard can collect reports from your Puppet Master as they're crea
puppetmasterd --configprint libdir
-2. Create a directory for custom report libraries, e.g. run the following command, but replace `LIBDIR` with the path you found in step 1:
+2. If a puppet agent is also running on the puppet master:
+
+ a. Determine if the puppet agent is using pluginsync. You can find this out by running the following command:
+
+ puppetd --configprint pluginsync
+
+ b. If the value of pluginsync is true, you will need to ensure that the puppet agent uses a different libdir than the puppet master. You can do this by putting the following lines in your `puppet.conf`:
+
+ [puppetd]
+ libdir = $vardir/agent_lib
+
+3. Create a directory for custom report libraries, e.g. run the following command, but replace `LIBDIR` with the path you found in step 1:
mkdir -p LIBDIR/puppet/reports/
For example, if `puppetmasterd --configprint libdir` prints `/var/lib/puppet/lib`, then you should run:
mkdir -p /var/lib/puppet/lib/puppet/reports/
-3. Create a custom report processor file on your Puppet Master by copying the Puppet Dashboard's `ext/puppet/puppet_dashboard.rb` file to `LIBDIR/puppet/reports`. E.g.,
+4. Create a custom report processor file on your Puppet Master by copying the Puppet Dashboard's `ext/puppet/puppet_dashboard.rb` file to `LIBDIR/puppet/reports`. E.g.,
cp ext/puppet/puppet_dashboard.rb LIBDIR/puppet/reports
@@ -287,14 +300,16 @@ The Puppet Dashboard can act as an external node classification tool, which will
*NOTE:* The `bin/external_node` program connects to the Puppet Dashboard at `localhost` on port `3000`. If your Puppet Dashboard is running on a different host or node, please modify this file.
+ *NOTE:* If you have Dashboard set up to use HTTPS, change the DASHBOARD_URL in `external_node` to the `https` prefix and the correct port number (443, by default). You may also need to change the CERT_PATH, and PKEY_PATH variables if your puppet master's hostname is not `puppet` or if your ssldir is not `/etc/puppet/ssl`.
+
2. Restart the `puppetmasterd` process.
Security
--------
*WARNING:* The Puppet Dashboard provides access to sensitive information and can make changes to your Puppet-managed infrastructure. You must restrict access to it to protect it!
-The Puppet Dashboard does not currently provide authentication, authorization or encryption -- although work on these is in progress.
+The Puppet Dashboard does not currently provide authentication or authorization -- although work on these is in progress.
Third-party tools that can help secure a Puppet Dashboard include:
@@ -306,12 +321,13 @@ Third-party tools that can help secure a Puppet Dashboard include:
http://username:password@hostname
+4. HTTPS (SSL) Encryption is supported when running Dashboard under Apache and Passenger. The example configuration in `ext/passenger/dashboard-vhost.conf` includes a commented-out vhost configured to use SSL. You may need to change the Apache directives SSLCertificateFile, SSLCertificateKeyFile, SSLCACertificateFile, and SSLCARevocationFile to the paths of the files created by the `cert` rake tasks. (See `Generating certs and connecting to the puppet master` for how to create these files)
+
Performance
-----------
The Puppet Dashboard slows down as it manages more data. Here are ways to make it run faster, from easiest to hardest:
-* Clear the Ruby on Rails logs. You can zero all these logs by running `rake log:clear`, or use a tool like `logrotate` to archive old files in the `logs` directory.
* Optimize your database by running `rake RAILS_ENV=production db:raw:optimize` from your Puppet Dashboard directory, this will reorganize and reanalyze your database for faster queries.
* Run the application in `production` mode, e.g. by running `./script/server -e production`. The default `development` mode is significantly slower because it doesn't cache and logs more details.
* Run the application using multiple processes to handle more concurrent requests. You can use Phusion Passenger, or clusters of Thin or Unicorn servers to serve multiple concurrent requests.
@@ -325,7 +341,7 @@ Debugging
The Puppet Dashboard may not start or may display warnings if misconfigured or if it encounters an error. Details about these errors are recorded to log files that will help diagnose and resolve the problem.
-You can find the logs in the `log` subdirectory of the Puppet Dashboard install, which will probably be in `/usr/share/puppet-dashboard/log/{environment}.log` if you installed from a package.
+You can find the logs in the `log` subdirectory of the Puppet Dashboard install, which will probably be in `/usr/share/puppet-dashboard/log/{environment}.log` if you installed from a package. You may want to customize your log rotation in `config/environment.rb`, if you would like to devote more or less disk to archival of logs.
If you installed from source, it will be wherever you cloned your git repository.
@@ -368,13 +384,13 @@ Generating certs and connecting to the puppet master
In order to connect to the puppet master (to retrieve node facts), the Dashboard must be configured with the correct SSL certificates. To do this, run the following commands:
- rake create_key_pair
+ rake cert:create_key_pair
- rake cert_request
+ rake cert:request
Then instruct the master to sign the certificate request (using "puppet cert"), and then run the command:
- rake cert_retrieve
+ rake cert:retrieve
You will also need to configure auth.conf on the master to allow Dashboard to connect to the facts terminus:
@@ -1,5 +0,0 @@
-class StatusesController < ApplicationController
- def overview
- render :layout => !request.xhr?
- end
-end
@@ -156,6 +156,10 @@ def puppet_log_sorter(collection)
end
end
+ def wrap_on_slashes(str)
+ (h str).gsub("/","/<wbr />")
+ end
+
# Return HTML describing the search if one is present in params[:q].
def describe_search_if_present
if params[:q].present?
View
@@ -22,7 +22,7 @@ def self.per_page; 20 end # Pagination
named_scope :search, lambda{|q| q.blank? ? {} : {:conditions => ['name LIKE ?', "%#{q}%"]} }
# ordering scopes for has_scope
- named_scope :by_latest_report, proc { |order|
+ named_scope :by_latest_report, proc { |order|
direction = {1 => 'ASC', 0 => 'DESC'}[order]
direction ? {:order => "reported_at #{direction}"} : {}
}
@@ -33,8 +33,6 @@ def self.per_page; 20 end # Pagination
fires :updated, :on => :update
fires :removed, :on => :destroy
- # RH:TODO: Denormalize last report status into nodes table.
-
# Return nodes based on their currentness and successfulness.
#
# The terms are:
@@ -47,11 +45,12 @@ def self.per_page; 20 end # Pagination
# * non-current and successful: Return any nodes that ever had a successful report.
# * non-current and failing: Return any nodes that ever had a failing report.
named_scope :by_currentness_and_successfulness, lambda {|currentness, successfulness|
+ operator = successfulness ? '!=' : '='
if currentness
- { :conditions => ['nodes.success = ?', successfulness] }
+ { :conditions => ["nodes.status #{operator} 'failed' AND nodes.last_report_id is not NULL"] }
else
{
- :conditions => ['reports.success = ?', successfulness],
+ :conditions => ["reports.status #{operator} 'failed'"],
:joins => :reports,
:group => 'nodes.id',
}
@@ -69,10 +68,11 @@ def self.per_page; 20 end # Pagination
named_scope :no_longer_reporting, :conditions => ['reported_at < ?', NO_LONGER_REPORTING_CUTOFF.ago]
def self.count_by_currentness_and_successfulness(currentness, successfulness)
+ operator = successfulness ? '!=' : '='
if currentness
self.by_currentness_and_successfulness(currentness, successfulness).count
else
- Report.count_by_sql(['SELECT COUNT(node_id) FROM (SELECT DISTINCT node_id FROM reports WHERE success = ?) as tmp', successfulness])
+ Report.count_by_sql(["SELECT COUNT(node_id) FROM (SELECT DISTINCT node_id FROM reports WHERE status #{operator} 'failed') as tmp"])
end
end
@@ -104,24 +104,8 @@ def to_param
name.to_s
end
- def available_node_classes
- @available_node_classes ||= NodeClass.all(:order => :name) - node_classes - inherited_classes
- end
-
- def available_node_groups
- @available_node_groups ||= NodeGroup.all(:order => :name) - node_groups
- end
-
- def inherited_classes
- (node_group_list - [self]).map(&:node_classes).flatten.uniq
- end
-
- def all_classes
- node_classes | inherited_classes
- end
-
def configuration
- { 'name' => name, 'classes' => all_classes.collect(&:name), 'parameters' => compiled_parameters }
+ { 'name' => name, 'classes' => all_node_classes.collect(&:name), 'parameters' => parameter_list }
end
def to_yaml(opts={})
@@ -132,40 +116,10 @@ def timeline_events
TimelineEvent.for_node(self)
end
- # This wrapper method is just used to cache the result of the recursive method
- def compiled_parameters(allow_conflicts=false)
- unless @compiled_parameters
- @compiled_parameters, @conflicts = compile_subgraph_parameters(self, node_group_graph)
- @conflicts.each do |key|
- errors.add(:parameters,key)
- end
- end
- raise ParameterConflictError unless allow_conflicts or @conflicts.empty?
- @compiled_parameters
- end
+ # Placeholder attributes
- # Walks the graph of node groups for the given node, compiling parameters by
- # merging down (preferring parameters specified in node groups that are
- # nearer). Raises a ParameterConflictError if parameters at the same distance
- # from the node have the same name.
- def compile_subgraph_parameters(group,subgraph)
- children = subgraph.map do |child,child_subgraph|
- compile_subgraph_parameters(child,child_subgraph)
- end
- # Pick-up conflicts that our children had
- conflicts = children.map(&:last).inject(Set.new,&:merge)
- params = group.parameters.to_hash
- inherited = {}
- # Now collect our inherited params and their conflicts
- children.map(&:first).map {|h| [*h]}.flatten.each_slice(2) do |key,value|
- conflicts.add(key) if inherited[key] && inherited[key] != value
- inherited[key] = value
- end
- # Resolve all possible conflicts
- conflicts.each do |key|
- conflicts.delete(key) if params[key]
- end
- [params.reverse_merge(inherited), conflicts]
+ def environment
+ 'production'
end
def status_class
@@ -208,9 +162,9 @@ def assign_last_report(report=nil)
report ||= find_last_report
unless self.last_report == report
- self.last_report = report
+ self.last_report = report
self.reported_at = report ? report.time : nil
- self.success = report ? report.success? : false
+ self.status = report ? report.status : 'unchanged'
# FIXME #update_without_callbacks doesn't update the object, and #save! is creating unwanted timeline events.
### node.send :update_without_callbacks # do not create a timeline event
View
@@ -1,10 +1,12 @@
class NodeClass < ActiveRecord::Base
def self.per_page; 50 end # Pagination
+ include NodeGroupGraph
+
has_many :node_group_class_memberships, :dependent => :destroy
has_many :node_class_memberships, :dependent => :destroy
- has_many :node_groups, :through => :node_group_class_memberships
+ has_many :node_group_children, :class_name => "NodeGroup", :through => :node_group_class_memberships, :source => :node_group
has_many :nodes, :through => :node_class_memberships
validates_presence_of :name
View
@@ -12,8 +12,12 @@ def self.per_page; 50 end # Pagination
has_many :node_group_edges_out, :class_name => "NodeGroupEdge", :foreign_key => 'from_id', :dependent => :destroy
has_many :node_group_edges_in, :class_name => "NodeGroupEdge", :foreign_key => 'to_id', :dependent => :destroy
- # TODO Want to add a list of groups have edges into us, may want to rename node_groups
- has_many :node_groups, :through => :node_group_edges_out, :source => :to
+ has_many :node_group_children, :class_name => "NodeGroup", :through => :node_group_edges_in, :source => :from
+ has_many :node_group_parents, :class_name => "NodeGroup", :through => :node_group_edges_out, :source => :to
+
+ # Alias for compatibility with Node
+ alias :node_groups :node_group_parents
+ alias :node_groups= :node_group_parents=
has_parameters
View
@@ -4,7 +4,6 @@ def self.per_page; 20 end # Pagination
before_validation :ensure_valid_format
before_validation :process_report
- validate :report_contains_metrics
validates_presence_of :host
validates_presence_of :time
validates_uniqueness_of :host, :scope => :time, :allow_nil => true
@@ -30,10 +29,6 @@ def report
rep
end
- def status
- failed? ? 'failure' : 'success'
- end
-
def metrics
return unless report && report.metrics
@metrics ||= report.metrics.with_indifferent_access
@@ -50,6 +45,8 @@ def total_time
def config_retrieval_time
if value = metric_value(:time, :config_retrieval)
TOTAL_TIME_FORMAT % value
+ else
+ TOTAL_TIME_FORMAT % 0
end
end
@@ -71,9 +68,9 @@ def process_report
end
def set_attributes
- self.success = !report.failed?
- self.time = report.time
- self.host = report.host
+ self.status = failed? ? 'failed' : changed? ? 'changed' : 'unchanged'
+ self.time = report.time
+ self.host = report.host
end
def assign_to_node
@@ -89,10 +86,4 @@ def update_node(force=false)
def replace_last_report
node.assign_last_report if node
end
-
- def report_contains_metrics
- has_metrics = report.metrics.present?
- errors.add_to_base("The report contains no metrics") unless has_metrics
- has_metrics
- end
end
View
@@ -1,9 +1,10 @@
class Status
- attr_reader :failed, :total, :percent, :start
+ attr_reader :changed, :unchanged, :failed, :total, :start
def initialize(datum)
+ @changed = datum["changed"].to_i
+ @unchanged = datum["unchanged"].to_i
@failed = datum["failed"].to_i
@total = datum["total"].to_i
- @percent = datum["percent"].to_f
@start = datum["start"].to_time
end
@@ -15,11 +16,6 @@ def self.recent(options={})
by_interval options.merge(:start => 1.hour.ago)
end
- def self.sparkline
- # return [12.5, 10.5, 13.4, 11.4, 13.2, 12.3, 13.4, 14.3]
- by_interval(:limit => 20).map(&:percent)
- end
-
# Default time in seconds for the interval
INTERVAL_CUTOFF = 30.days
@@ -47,9 +43,10 @@ def self.by_interval(options={})
sql = <<-SQL
SELECT
- COUNT(*) - SUM(success) as failed,
COUNT(*) as total,
- SUM(success) / COUNT(*) * 100 as percent,
+ SUM(CASE status when "unchanged" then 1 else 0 end) as unchanged,
+ SUM(CASE status when "changed" then 1 else 0 end) as changed,
+ SUM(CASE status when "failed" then 1 else 0 end) as failed,
#{date} as start
FROM reports
SQL
@@ -9,12 +9,12 @@
%li= link_to 'Destroy', @node_class, :confirm => 'Are you sure?', :method => :delete, :class => "delete button"
.item
- - if @node_class.nodes.present?
+ - if @node_class.all_nodes.present?
.section
- = render 'statuses/run_failure', :nodes => @node_class.nodes
+ = render 'statuses/run_failure', :nodes => @node_class.all_nodes
.section
%h3 Nodes in this class
- - if @node_class.nodes.present?
- = render 'nodes/nodes', :nodes => @node_class.nodes
+ - if @node_class.all_nodes.present?
+ = render 'nodes/nodes', :nodes => @node_class.all_nodes, :container => @node_class
- else
= describe_no_matches_for :nodes, :class
Oops, something went wrong.

0 comments on commit 308dcbb

Please sign in to comment.