Permalink
Browse files

Initial commit. See readme for info.

  • Loading branch information...
0 parents commit ee995157786dd59632c73596a8a2cd8953b399de Chris Heald committed Mar 24, 2009
Showing with 248 additions and 0 deletions.
  1. +19 −0 LICENSE
  2. +19 −0 README
  3. +184 −0 app/metal/scrap.rb
  4. +5 −0 init.rb
  5. +12 −0 sample.html
  6. +9 −0 scrap.yml.example
19 LICENSE
@@ -0,0 +1,19 @@
+Copyright (c) 2008 Chris Heald
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.
19 README
@@ -0,0 +1,19 @@
+Scrap is a Rails Metal endpoint designed to expose various garbage and memory-related metrics about your app. It may be particularly useful in tracking down memory leaks.
+
+To use it, simply install the plugin. This will provide a new url, <code>/stats/scrap</code>, which will report a number of metrics about your app.
+
+h2. Config
+
+If present, Scrap will use a config/scrap.yml file. See the provided example file for a list of the configuration options accepted.
+
+* max requests: How many requests to keep a record of. Older requests will be pushed out of the queue when the limit has been reached. Default is 150.
+* max_objects: How many objects/deltas to show. Default is 50.
+* classes: A hash of class names to do object counting on. Values may be "true" which prints the object count with a default set of options, or it may be a hash consisting of the following:
+** print_objects: boolean - toggles the output of a representation of each instance of the type.
+** show_fields: array - list of fields to show per instance. This actually invokes the "attributes" method of the object, so it's really only useful for ActiveRecord objects.
+** small: boolean - if false, will not print counts in h3 tags. Default is true.
+** min: integer - minimum count, if set, that an object must have to appear in the delta or top objects list. Default is nil.
+
+h2. Other considerations
+
+Scrap will take advantage of many of the wonderful metrics provided by recent versions of Ruby Enterprise Edition. It will work with other versions of Ruby, but you'll get a lot more info out of it if you're running REE 1.8.6-20090201 or later.
184 app/metal/scrap.rb
@@ -0,0 +1,184 @@
+class Scrap < Rails::Rack::Metal
+ PATH = "/stats/scrap".freeze
+ COMMIFY_REGEX = /(\d)(?=(\d\d\d)+(?!\d))/
+
+ @@gc_stats = {}
+ @@last_gc_run = nil
+ @@last_gc_mem = nil
+ @@requests_processed = 0
+ @@request_list = []
+ @@alive_at = nil
+ @@gc_stats_enabled = nil
+ @@config = nil
+
+ def self.config
+ @@config ||= YAML::load open(File.join(Rails.root, "config", "scrap.yml")).read
+ rescue Errno::ENOENT
+ @@config = {}
+ rescue
+ puts "[scrap] scrap.yml: #{$!.message}"
+ @@config = {}
+ end
+
+ def self.call(env)
+ if !@@gc_stats_enabled then
+ GC.enable_stats if GC.respond_to? :enable_stats
+ @@gc_stats_enabled = true
+ end
+ @@requests_processed += 1
+ @@last_gc_run ||= @@alive_at ||= Time.now.to_f
+ @@last_gc_mem ||= get_usage
+
+ req = sprintf("[%-10.2fMB] %s %s", get_usage, env["REQUEST_METHOD"], env["PATH_INFO"])
+ req << "<pre>#{ObjectSpace.statistics}</pre>" if ObjectSpace.respond_to? :statistics
+ @@request_list.unshift req
+ @@request_list.pop if @@request_list.length > (config["max_requests"] || 150)
+
+ if env["PATH_INFO"] == PATH
+ gc_stats
+ else
+ NotFoundResponse
+ end
+ end
+
+ def self.gc_stats
+ collected = nil
+ puts "Respond to? #{ObjectSpace.respond_to? :live_objects}"
+ if ObjectSpace.respond_to? :live_objects then
+ live = ObjectSpace.live_objects
+ GC.start
+ collected = live - ObjectSpace.live_objects
+ else
+ GC.start
+ end
+ GC.start
+ usage = get_usage
+
+ mem_delta = usage - @@last_gc_mem
+ time_delta = Time.now.to_f - @@last_gc_run
+
+ s = "<title>[#{$$}] Garbage Report</title>"
+ s << "<style> body { font-family: monospace; color: #222; } td { border-bottom: 1px solid #eee; padding: 1px 9px; } td.t { background: #fafafa; } tr:hover td { background: #fafaf0; border-color: #e0e0dd; } h1,h2,h3 { border-bottom: 1px solid #ddd; font-family: sans-serif; } </style>"
+
+ s << "<h1>Scrap - PID #{$$}</h1>"
+
+ s << "<table>"
+ s << sprintf("<tr><td class='t'>Memory usage:</td><td>%2.2fMB</td></tr>", usage)
+ s << sprintf("<tr><td class='t'>Delta:</td><td>%2.2fMB</td></tr>", mem_delta)
+ s << sprintf("<tr><td class='t'>Last Scrap req:</td><td>%2.2f seconds ago</td></tr>", time_delta)
+ s << sprintf("<tr><td class='t'>Requests processed:</td><td>%s</td></tr>", @@requests_processed)
+ s << sprintf("<tr><td class='t'>Alive for:</td><td>%2.2f seconds</td></tr>", Time.now.to_f - @@alive_at)
+ if GC.respond_to? :time then
+ s << sprintf("<tr><td class='t'>Time spent in GC:</td><td>%2.2f seconds</td></tr>", GC.time / 1000000.0)
+ end
+ if collected
+ s << sprintf("<tr><td class='t'>Collected objects:</td><td>%2d</td></tr>", collected)
+ s << sprintf("<tr><td class='t'>Live objects:</td><td>%2d</td></tr>", ObjectSpace.live_objects)
+ end
+ s << "</table>"
+
+ s << "<h3>Top #{config["max_objects"]} deltas since last request</h3>"
+ s << "<table border='0'>"
+ memcheck(config["max_objects"], Object, :deltas).each do |v|
+ next if v.last == 0
+ s << "<tr><td class='t'>#{v.first}</td><td>#{sprintf("%s%s", v.last >= 0 ? "+" : "-", commify(v.last))}</td></tr>"
+ end
+ s << "</table>"
+
+ s << "<h3>Top #{config["max_objects"]} objects</h3>"
+ s << "<table border='0'>"
+ memcheck(config["max_objects"]).each do |v|
+ s << "<tr><td class='t'>#{v.first}</td><td>#{commify v.last}</td></tr>"
+ end
+ s << "</table>"
+
+ (config["classes"] || {}).each do |klass, val|
+ puts val.inspect
+ opts = val === true ? {"print_objects" => true} : val
+ add_os(klass.constantize, s, opts)
+ end
+
+ s << "<h3>Request history</h3>"
+ @@request_list.each do |req|
+ s << req
+ s << "<br />"
+ end
+
+ @@last_gc_run = Time.now.to_f
+ @@last_gc_mem = usage
+ @@requests_processed = 0
+ [200, {"Content-Type" => "text/html"}, s]
+ end
+
+ def self.get_usage
+ usage = 0
+ begin
+ usage = `cat /proc/#{$$}/stat`.split(" ")[22].to_i / (1024 * 1024).to_f
+ rescue
+ # pass
+ end
+ return usage
+ end
+
+ def self.add_os(c, s, options = {})
+ print_objects = options["print_objects"]
+ small = options["small"]
+ min = options["min"]
+ show_fields = options["show_fields"]
+
+ ct = ObjectSpace.each_object(c) {}
+ return if min and ct < min
+
+ if small
+ s << "#{c} (#{ct})<br />"
+ else
+ s << "<h3>#{c} (#{ct})</h3>"
+ end
+
+ return if !print_objects
+ s << "<table>"
+ val = ObjectSpace.each_object(c) do |m|
+ s << "<tr><td class='t'>" << "&lt;#{m.class.to_s}:#{sprintf("0x%.8x", m.object_id)}&gt;</td>"
+ if show_fields then
+ show_fields.each do |field|
+ v = m.attributes[field.to_s]
+ if v.blank?
+ s << "<td>&nbsp;</td>"
+ else
+ s << "<td>#{field}: #{v}</td>"
+ end
+ end
+ end
+ s << "</tr>"
+ end
+ s << "</table>"
+ end
+
+ def self.memcheck(top, klass = Object, mode = :normal)
+ top ||= 50
+ os = Hash.new(0)
+ ObjectSpace.each_object(klass) do |o|
+ begin;
+ # If this is true, it's an association proxy, and we don't want to invoke method_missing on it,
+ # as it will result in a load of the association proxy from the DB, doing extra DB work and
+ # potentially creating a lot of AR objects. Hackalicious.
+ next if o.respond_to? :proxy_respond_to?
+ os[o.class.to_s] += 1 if o.respond_to? :class
+ rescue; end
+ end
+ if mode == :deltas then
+ os2 = Hash.new(0)
+ os.each do |k, v|
+ os2[k] = v - (@@gc_stats[k] || 0)
+ @@gc_stats[k] = v
+ end
+ sorted = os2.sort_by{|k,v| -v }.first(top)
+ else
+ sorted = os.sort_by{|k,v| -v }.first(top)
+ end
+ end
+
+ def self.commify(i)
+ i.to_s.gsub(COMMIFY_REGEX, "\\1,")
+ end
+end
5 init.rb
@@ -0,0 +1,5 @@
+begin
+ Rails::Rack::Metal::metal_paths << File.join(File.dirname(__FILE__), "app", "metal")
+rescue NameError
+ puts "[WARNING] Scrap requires Rails 2.3 or better. Not booting."
+end
12 sample.html
@@ -0,0 +1,12 @@
+<html><head>
+<meta http-equiv="content-type" content="text/html; charset=UTF-8"><title>[9752] Garbage Report</title><style> body { font-family: monospace; color: #222; } td { border-bottom: 1px solid #eee; padding: 1px 9px; } td.t { background: #fafafa; } tr:hover td { background: #fafaf0; border-color: #e0e0dd; } h1,h2,h3 { border-bottom: 1px solid #ddd; font-family: sans-serif; } </style></head><body><h1>Scrap - PID 9752</h1><table><tbody><tr><td class="t">Memory usage:</td><td>59.01MB</td></tr><tr><td class="t">Delta:</td><td>0.00MB</td></tr><tr><td class="t">Last Scrap req:</td><td>0.39 seconds ago</td></tr><tr><td class="t">Requests processed:</td><td>1</td></tr><tr><td class="t">Alive for:</td><td>0.39 seconds</td></tr><tr><td class="t">Time spent in GC:</td><td>0.18 seconds</td></tr><tr><td class="t">Collected objects:</td><td>309826</td></tr><tr><td class="t">Live objects:</td><td>696773</td></tr></tbody></table><h3>Top deltas since last request</h3><table border="0"><tbody><tr><td class="t">String</td><td>+153,220</td></tr><tr><td class="t">Array</td><td>+5,690</td></tr><tr><td class="t">Proc</td><td>+3,268</td></tr><tr><td class="t">Hash</td><td>+3,090</td></tr><tr><td class="t">Regexp</td><td>+1,978</td></tr><tr><td class="t">Class</td><td>+1,362</td></tr><tr><td class="t">ActionController::Routing::DividerSegment</td><td>+946</td></tr><tr><td class="t">ActiveSupport::Callbacks::Callback</td><td>+754</td></tr><tr><td class="t">ActionController::Routing::StaticSegment</td><td>+604</td></tr><tr><td class="t">Module</td><td>+563</td></tr><tr><td class="t">Gem::Version</td><td>+425</td></tr><tr><td class="t">Gem::Requirement</td><td>+389</td></tr><tr><td class="t">ActionController::Routing::Route</td><td>+305</td></tr><tr><td class="t">ActionController::Routing::DynamicSegment</td><td>+258</td></tr><tr><td class="t">ActionController::Routing::OptionalFormatSegment</td><td>+223</td></tr><tr><td class="t">ActiveSupport::Callbacks::CallbackChain</td><td>+218</td></tr><tr><td class="t">Range</td><td>+190</td></tr><tr><td class="t">ActiveRecord::Reflection::AssociationReflection</td><td>+142</td></tr><tr><td class="t">Time</td><td>+132</td></tr><tr><td class="t">Gem::Specification</td><td>+128</td></tr><tr><td class="t">Gem::Dependency</td><td>+120</td></tr><tr><td class="t">Float</td><td>+103</td></tr><tr><td class="t">UnboundMethod</td><td>+98</td></tr><tr><td class="t">Magick::CompositeOperator</td><td>+56</td></tr><tr><td class="t">Set</td><td>+34</td></tr><tr><td class="t">Magick::PreviewType</td><td>+30</td></tr><tr><td class="t">HashWithIndifferentAccess</td><td>+24</td></tr><tr><td class="t">Magick::ColorspaceType</td><td>+23</td></tr><tr><td class="t">Magick::FilterTypes</td><td>+22</td></tr><tr><td class="t">Mime::Type</td><td>+21</td></tr><tr><td class="t">ActiveRecord::Reflection::ThroughReflection</td><td>+19</td></tr><tr><td class="t">Magick::ChannelType</td><td>+18</td></tr><tr><td class="t">Magick::ImageLayerMethod</td><td>+16</td></tr><tr><td class="t">Rails::Plugin</td><td>+15</td></tr><tr><td class="t">Rails::OrderedOptions</td><td>+14</td></tr><tr><td class="t">Magick::VirtualPixelMethod</td><td>+13</td></tr><tr><td class="t">Magick::GravityType</td><td>+12</td></tr><tr><td class="t">Magick::QuantumExpressionOperator</td><td>+12</td></tr><tr><td class="t">ActionController::MiddlewareStack::Middleware</td><td>+12</td></tr><tr><td class="t">Magick::ImageType</td><td>+12</td></tr><tr><td class="t">Mutex</td><td>+12</td></tr><tr><td class="t">Rails::GemDependency</td><td>+11</td></tr><tr><td class="t">Rails::GemPlugin</td><td>+11</td></tr><tr><td class="t">Magick::CompressionType</td><td>+11</td></tr><tr><td class="t">Magick::StretchType</td><td>+10</td></tr><tr><td class="t">Bignum</td><td>+9</td></tr><tr><td class="t">Magick::OrientationType</td><td>+9</td></tr><tr><td class="t">ThinkingSphinx::Index::FauxColumn</td><td>+9</td></tr><tr><td class="t">Magick::InterlaceType</td><td>+8</td></tr><tr><td class="t">Magick::DistortImageMethod</td><td>+8</td></tr></tbody></table><h3>Top objects</h3><table border="0"><tbody><tr><td class="t">String</td><td>111,167</td></tr><tr><td class="t">Array</td><td>5,860</td></tr><tr><td class="t">Proc</td><td>3,268</td></tr><tr><td class="t">Hash</td><td>3,091</td></tr><tr><td class="t">Regexp</td><td>1,978</td></tr><tr><td class="t">Class</td><td>1,362</td></tr><tr><td class="t">ActionController::Routing::DividerSegment</td><td>946</td></tr><tr><td class="t">ActiveSupport::Callbacks::Callback</td><td>754</td></tr><tr><td class="t">ActionController::Routing::StaticSegment</td><td>604</td></tr><tr><td class="t">Module</td><td>563</td></tr><tr><td class="t">Gem::Version</td><td>425</td></tr><tr><td class="t">Gem::Requirement</td><td>389</td></tr><tr><td class="t">ActionController::Routing::Route</td><td>305</td></tr><tr><td class="t">ActionController::Routing::DynamicSegment</td><td>258</td></tr><tr><td class="t">ActionController::Routing::OptionalFormatSegment</td><td>223</td></tr><tr><td class="t">ActiveSupport::Callbacks::CallbackChain</td><td>218</td></tr><tr><td class="t">Range</td><td>190</td></tr><tr><td class="t">ActiveRecord::Reflection::AssociationReflection</td><td>142</td></tr><tr><td class="t">Time</td><td>130</td></tr><tr><td class="t">Gem::Specification</td><td>128</td></tr><tr><td class="t">Gem::Dependency</td><td>120</td></tr><tr><td class="t">UnboundMethod</td><td>98</td></tr><tr><td class="t">Float</td><td>94</td></tr><tr><td class="t">Magick::CompositeOperator</td><td>56</td></tr><tr><td class="t">Set</td><td>34</td></tr><tr><td class="t">Magick::PreviewType</td><td>30</td></tr><tr><td class="t">HashWithIndifferentAccess</td><td>24</td></tr><tr><td class="t">Magick::ColorspaceType</td><td>23</td></tr><tr><td class="t">Magick::FilterTypes</td><td>22</td></tr><tr><td class="t">Mime::Type</td><td>21</td></tr><tr><td class="t">ActiveRecord::Reflection::ThroughReflection</td><td>19</td></tr><tr><td class="t">Magick::ChannelType</td><td>18</td></tr><tr><td class="t">Magick::ImageLayerMethod</td><td>16</td></tr><tr><td class="t">Rails::Plugin</td><td>15</td></tr><tr><td class="t">Rails::OrderedOptions</td><td>14</td></tr><tr><td class="t">Magick::VirtualPixelMethod</td><td>13</td></tr><tr><td class="t">Magick::QuantumExpressionOperator</td><td>12</td></tr><tr><td class="t">Mutex</td><td>12</td></tr><tr><td class="t">ActionController::MiddlewareStack::Middleware</td><td>12</td></tr><tr><td class="t">Magick::ImageType</td><td>12</td></tr><tr><td class="t">Magick::GravityType</td><td>12</td></tr><tr><td class="t">Rails::GemDependency</td><td>11</td></tr><tr><td class="t">Magick::CompressionType</td><td>11</td></tr><tr><td class="t">Rails::GemPlugin</td><td>11</td></tr><tr><td class="t">Magick::StretchType</td><td>10</td></tr><tr><td class="t">Magick::OrientationType</td><td>9</td></tr><tr><td class="t">Bignum</td><td>9</td></tr><tr><td class="t">ThinkingSphinx::Index::FauxColumn</td><td>9</td></tr><tr><td class="t">Magick::MetricType</td><td>8</td></tr><tr><td class="t">Magick::StorageType</td><td>8</td></tr></tbody></table><h3>Request history</h3>[59.01 MB] GET /stats/scrap<pre>Number of objects : 824154 (612748 AST nodes, 74.35%)
+Heap slot size : 20
+GC cycles so far : 26
+Number of heaps : 7
+Total size of objects: 16096.76 KB
+Total size of heaps : 18036.66 KB (1939.91 KB = 10.76% unused)
+Leading free slots : 13649 (266.58 KB = 1.48%)
+Trailing free slots : 0 (0.00 KB = 0.00%)
+Number of contiguous groups of 16 slots: 2113 (3.66%)
+Number of terminal objects: 3890 (0.42%)
+</pre><br></body></html>
9 scrap.yml.example
@@ -0,0 +1,9 @@
+max_requests: 150
+max_objects: 10
+classes:
+ "ActionController::Base": true
+ "ActionController::Session::AbstractStore::SessionHash": true
+ "ActiveRecord::Base":
+ print_objects: true
+ foo: bar
+ show_fields: [id, updated_at]

0 comments on commit ee99515

Please sign in to comment.