Skip to content

Commit

Permalink
Initial commit. See readme for info.
Browse files Browse the repository at this point in the history
  • Loading branch information
Chris Heald committed Mar 24, 2009
0 parents commit ee99515
Show file tree
Hide file tree
Showing 6 changed files with 248 additions and 0 deletions.
19 changes: 19 additions & 0 deletions LICENSE
@@ -0,0 +1,19 @@
Copyright (c) 2008 Chris Heald

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
19 changes: 19 additions & 0 deletions README
@@ -0,0 +1,19 @@
Scrap is a Rails Metal endpoint designed to expose various garbage and memory-related metrics about your app. It may be particularly useful in tracking down memory leaks.

To use it, simply install the plugin. This will provide a new url, <code>/stats/scrap</code>, which will report a number of metrics about your app.

h2. Config

If present, Scrap will use a config/scrap.yml file. See the provided example file for a list of the configuration options accepted.

* max requests: How many requests to keep a record of. Older requests will be pushed out of the queue when the limit has been reached. Default is 150.
* max_objects: How many objects/deltas to show. Default is 50.
* classes: A hash of class names to do object counting on. Values may be "true" which prints the object count with a default set of options, or it may be a hash consisting of the following:
** print_objects: boolean - toggles the output of a representation of each instance of the type.
** show_fields: array - list of fields to show per instance. This actually invokes the "attributes" method of the object, so it's really only useful for ActiveRecord objects.
** small: boolean - if false, will not print counts in h3 tags. Default is true.
** min: integer - minimum count, if set, that an object must have to appear in the delta or top objects list. Default is nil.

h2. Other considerations

Scrap will take advantage of many of the wonderful metrics provided by recent versions of Ruby Enterprise Edition. It will work with other versions of Ruby, but you'll get a lot more info out of it if you're running REE 1.8.6-20090201 or later.
184 changes: 184 additions & 0 deletions app/metal/scrap.rb
@@ -0,0 +1,184 @@
class Scrap < Rails::Rack::Metal
PATH = "/stats/scrap".freeze
COMMIFY_REGEX = /(\d)(?=(\d\d\d)+(?!\d))/

@@gc_stats = {}
@@last_gc_run = nil
@@last_gc_mem = nil
@@requests_processed = 0
@@request_list = []
@@alive_at = nil
@@gc_stats_enabled = nil
@@config = nil

def self.config
@@config ||= YAML::load open(File.join(Rails.root, "config", "scrap.yml")).read
rescue Errno::ENOENT
@@config = {}
rescue
puts "[scrap] scrap.yml: #{$!.message}"
@@config = {}
end

def self.call(env)
if !@@gc_stats_enabled then
GC.enable_stats if GC.respond_to? :enable_stats
@@gc_stats_enabled = true
end
@@requests_processed += 1
@@last_gc_run ||= @@alive_at ||= Time.now.to_f
@@last_gc_mem ||= get_usage

req = sprintf("[%-10.2fMB] %s %s", get_usage, env["REQUEST_METHOD"], env["PATH_INFO"])
req << "<pre>#{ObjectSpace.statistics}</pre>" if ObjectSpace.respond_to? :statistics
@@request_list.unshift req
@@request_list.pop if @@request_list.length > (config["max_requests"] || 150)

if env["PATH_INFO"] == PATH
gc_stats
else
NotFoundResponse
end
end

def self.gc_stats
collected = nil
puts "Respond to? #{ObjectSpace.respond_to? :live_objects}"
if ObjectSpace.respond_to? :live_objects then
live = ObjectSpace.live_objects
GC.start
collected = live - ObjectSpace.live_objects
else
GC.start
end
GC.start
usage = get_usage

mem_delta = usage - @@last_gc_mem
time_delta = Time.now.to_f - @@last_gc_run

s = "<title>[#{$$}] Garbage Report</title>"
s << "<style> body { font-family: monospace; color: #222; } td { border-bottom: 1px solid #eee; padding: 1px 9px; } td.t { background: #fafafa; } tr:hover td { background: #fafaf0; border-color: #e0e0dd; } h1,h2,h3 { border-bottom: 1px solid #ddd; font-family: sans-serif; } </style>"

s << "<h1>Scrap - PID #{$$}</h1>"

s << "<table>"
s << sprintf("<tr><td class='t'>Memory usage:</td><td>%2.2fMB</td></tr>", usage)
s << sprintf("<tr><td class='t'>Delta:</td><td>%2.2fMB</td></tr>", mem_delta)
s << sprintf("<tr><td class='t'>Last Scrap req:</td><td>%2.2f seconds ago</td></tr>", time_delta)
s << sprintf("<tr><td class='t'>Requests processed:</td><td>%s</td></tr>", @@requests_processed)
s << sprintf("<tr><td class='t'>Alive for:</td><td>%2.2f seconds</td></tr>", Time.now.to_f - @@alive_at)
if GC.respond_to? :time then
s << sprintf("<tr><td class='t'>Time spent in GC:</td><td>%2.2f seconds</td></tr>", GC.time / 1000000.0)
end
if collected
s << sprintf("<tr><td class='t'>Collected objects:</td><td>%2d</td></tr>", collected)
s << sprintf("<tr><td class='t'>Live objects:</td><td>%2d</td></tr>", ObjectSpace.live_objects)
end
s << "</table>"

s << "<h3>Top #{config["max_objects"]} deltas since last request</h3>"
s << "<table border='0'>"
memcheck(config["max_objects"], Object, :deltas).each do |v|
next if v.last == 0
s << "<tr><td class='t'>#{v.first}</td><td>#{sprintf("%s%s", v.last >= 0 ? "+" : "-", commify(v.last))}</td></tr>"
end
s << "</table>"

s << "<h3>Top #{config["max_objects"]} objects</h3>"
s << "<table border='0'>"
memcheck(config["max_objects"]).each do |v|
s << "<tr><td class='t'>#{v.first}</td><td>#{commify v.last}</td></tr>"
end
s << "</table>"

(config["classes"] || {}).each do |klass, val|
puts val.inspect
opts = val === true ? {"print_objects" => true} : val
add_os(klass.constantize, s, opts)
end

s << "<h3>Request history</h3>"
@@request_list.each do |req|
s << req
s << "<br />"
end

@@last_gc_run = Time.now.to_f
@@last_gc_mem = usage
@@requests_processed = 0
[200, {"Content-Type" => "text/html"}, s]
end

def self.get_usage
usage = 0
begin
usage = `cat /proc/#{$$}/stat`.split(" ")[22].to_i / (1024 * 1024).to_f
rescue
# pass
end
return usage
end

def self.add_os(c, s, options = {})
print_objects = options["print_objects"]
small = options["small"]
min = options["min"]
show_fields = options["show_fields"]

ct = ObjectSpace.each_object(c) {}
return if min and ct < min

if small
s << "#{c} (#{ct})<br />"
else
s << "<h3>#{c} (#{ct})</h3>"
end

return if !print_objects
s << "<table>"
val = ObjectSpace.each_object(c) do |m|
s << "<tr><td class='t'>" << "&lt;#{m.class.to_s}:#{sprintf("0x%.8x", m.object_id)}&gt;</td>"
if show_fields then
show_fields.each do |field|
v = m.attributes[field.to_s]
if v.blank?
s << "<td>&nbsp;</td>"
else
s << "<td>#{field}: #{v}</td>"
end
end
end
s << "</tr>"
end
s << "</table>"
end

def self.memcheck(top, klass = Object, mode = :normal)
top ||= 50
os = Hash.new(0)
ObjectSpace.each_object(klass) do |o|
begin;
# If this is true, it's an association proxy, and we don't want to invoke method_missing on it,
# as it will result in a load of the association proxy from the DB, doing extra DB work and
# potentially creating a lot of AR objects. Hackalicious.
next if o.respond_to? :proxy_respond_to?
os[o.class.to_s] += 1 if o.respond_to? :class
rescue; end
end
if mode == :deltas then
os2 = Hash.new(0)
os.each do |k, v|
os2[k] = v - (@@gc_stats[k] || 0)
@@gc_stats[k] = v
end
sorted = os2.sort_by{|k,v| -v }.first(top)
else
sorted = os.sort_by{|k,v| -v }.first(top)
end
end

def self.commify(i)
i.to_s.gsub(COMMIFY_REGEX, "\\1,")
end
end
5 changes: 5 additions & 0 deletions init.rb
@@ -0,0 +1,5 @@
begin
Rails::Rack::Metal::metal_paths << File.join(File.dirname(__FILE__), "app", "metal")
rescue NameError
puts "[WARNING] Scrap requires Rails 2.3 or better. Not booting."
end
12 changes: 12 additions & 0 deletions sample.html
@@ -0,0 +1,12 @@
<html><head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8"><title>[9752] Garbage Report</title><style> body { font-family: monospace; color: #222; } td { border-bottom: 1px solid #eee; padding: 1px 9px; } td.t { background: #fafafa; } tr:hover td { background: #fafaf0; border-color: #e0e0dd; } h1,h2,h3 { border-bottom: 1px solid #ddd; font-family: sans-serif; } </style></head><body><h1>Scrap - PID 9752</h1><table><tbody><tr><td class="t">Memory usage:</td><td>59.01MB</td></tr><tr><td class="t">Delta:</td><td>0.00MB</td></tr><tr><td class="t">Last Scrap req:</td><td>0.39 seconds ago</td></tr><tr><td class="t">Requests processed:</td><td>1</td></tr><tr><td class="t">Alive for:</td><td>0.39 seconds</td></tr><tr><td class="t">Time spent in GC:</td><td>0.18 seconds</td></tr><tr><td class="t">Collected objects:</td><td>309826</td></tr><tr><td class="t">Live objects:</td><td>696773</td></tr></tbody></table><h3>Top deltas since last request</h3><table border="0"><tbody><tr><td class="t">String</td><td>+153,220</td></tr><tr><td class="t">Array</td><td>+5,690</td></tr><tr><td class="t">Proc</td><td>+3,268</td></tr><tr><td class="t">Hash</td><td>+3,090</td></tr><tr><td class="t">Regexp</td><td>+1,978</td></tr><tr><td class="t">Class</td><td>+1,362</td></tr><tr><td class="t">ActionController::Routing::DividerSegment</td><td>+946</td></tr><tr><td class="t">ActiveSupport::Callbacks::Callback</td><td>+754</td></tr><tr><td class="t">ActionController::Routing::StaticSegment</td><td>+604</td></tr><tr><td class="t">Module</td><td>+563</td></tr><tr><td class="t">Gem::Version</td><td>+425</td></tr><tr><td class="t">Gem::Requirement</td><td>+389</td></tr><tr><td class="t">ActionController::Routing::Route</td><td>+305</td></tr><tr><td class="t">ActionController::Routing::DynamicSegment</td><td>+258</td></tr><tr><td class="t">ActionController::Routing::OptionalFormatSegment</td><td>+223</td></tr><tr><td class="t">ActiveSupport::Callbacks::CallbackChain</td><td>+218</td></tr><tr><td class="t">Range</td><td>+190</td></tr><tr><td class="t">ActiveRecord::Reflection::AssociationReflection</td><td>+142</td></tr><tr><td class="t">Time</td><td>+132</td></tr><tr><td class="t">Gem::Specification</td><td>+128</td></tr><tr><td class="t">Gem::Dependency</td><td>+120</td></tr><tr><td class="t">Float</td><td>+103</td></tr><tr><td class="t">UnboundMethod</td><td>+98</td></tr><tr><td class="t">Magick::CompositeOperator</td><td>+56</td></tr><tr><td class="t">Set</td><td>+34</td></tr><tr><td class="t">Magick::PreviewType</td><td>+30</td></tr><tr><td class="t">HashWithIndifferentAccess</td><td>+24</td></tr><tr><td class="t">Magick::ColorspaceType</td><td>+23</td></tr><tr><td class="t">Magick::FilterTypes</td><td>+22</td></tr><tr><td class="t">Mime::Type</td><td>+21</td></tr><tr><td class="t">ActiveRecord::Reflection::ThroughReflection</td><td>+19</td></tr><tr><td class="t">Magick::ChannelType</td><td>+18</td></tr><tr><td class="t">Magick::ImageLayerMethod</td><td>+16</td></tr><tr><td class="t">Rails::Plugin</td><td>+15</td></tr><tr><td class="t">Rails::OrderedOptions</td><td>+14</td></tr><tr><td class="t">Magick::VirtualPixelMethod</td><td>+13</td></tr><tr><td class="t">Magick::GravityType</td><td>+12</td></tr><tr><td class="t">Magick::QuantumExpressionOperator</td><td>+12</td></tr><tr><td class="t">ActionController::MiddlewareStack::Middleware</td><td>+12</td></tr><tr><td class="t">Magick::ImageType</td><td>+12</td></tr><tr><td class="t">Mutex</td><td>+12</td></tr><tr><td class="t">Rails::GemDependency</td><td>+11</td></tr><tr><td class="t">Rails::GemPlugin</td><td>+11</td></tr><tr><td class="t">Magick::CompressionType</td><td>+11</td></tr><tr><td class="t">Magick::StretchType</td><td>+10</td></tr><tr><td class="t">Bignum</td><td>+9</td></tr><tr><td class="t">Magick::OrientationType</td><td>+9</td></tr><tr><td class="t">ThinkingSphinx::Index::FauxColumn</td><td>+9</td></tr><tr><td class="t">Magick::InterlaceType</td><td>+8</td></tr><tr><td class="t">Magick::DistortImageMethod</td><td>+8</td></tr></tbody></table><h3>Top objects</h3><table border="0"><tbody><tr><td class="t">String</td><td>111,167</td></tr><tr><td class="t">Array</td><td>5,860</td></tr><tr><td class="t">Proc</td><td>3,268</td></tr><tr><td class="t">Hash</td><td>3,091</td></tr><tr><td class="t">Regexp</td><td>1,978</td></tr><tr><td class="t">Class</td><td>1,362</td></tr><tr><td class="t">ActionController::Routing::DividerSegment</td><td>946</td></tr><tr><td class="t">ActiveSupport::Callbacks::Callback</td><td>754</td></tr><tr><td class="t">ActionController::Routing::StaticSegment</td><td>604</td></tr><tr><td class="t">Module</td><td>563</td></tr><tr><td class="t">Gem::Version</td><td>425</td></tr><tr><td class="t">Gem::Requirement</td><td>389</td></tr><tr><td class="t">ActionController::Routing::Route</td><td>305</td></tr><tr><td class="t">ActionController::Routing::DynamicSegment</td><td>258</td></tr><tr><td class="t">ActionController::Routing::OptionalFormatSegment</td><td>223</td></tr><tr><td class="t">ActiveSupport::Callbacks::CallbackChain</td><td>218</td></tr><tr><td class="t">Range</td><td>190</td></tr><tr><td class="t">ActiveRecord::Reflection::AssociationReflection</td><td>142</td></tr><tr><td class="t">Time</td><td>130</td></tr><tr><td class="t">Gem::Specification</td><td>128</td></tr><tr><td class="t">Gem::Dependency</td><td>120</td></tr><tr><td class="t">UnboundMethod</td><td>98</td></tr><tr><td class="t">Float</td><td>94</td></tr><tr><td class="t">Magick::CompositeOperator</td><td>56</td></tr><tr><td class="t">Set</td><td>34</td></tr><tr><td class="t">Magick::PreviewType</td><td>30</td></tr><tr><td class="t">HashWithIndifferentAccess</td><td>24</td></tr><tr><td class="t">Magick::ColorspaceType</td><td>23</td></tr><tr><td class="t">Magick::FilterTypes</td><td>22</td></tr><tr><td class="t">Mime::Type</td><td>21</td></tr><tr><td class="t">ActiveRecord::Reflection::ThroughReflection</td><td>19</td></tr><tr><td class="t">Magick::ChannelType</td><td>18</td></tr><tr><td class="t">Magick::ImageLayerMethod</td><td>16</td></tr><tr><td class="t">Rails::Plugin</td><td>15</td></tr><tr><td class="t">Rails::OrderedOptions</td><td>14</td></tr><tr><td class="t">Magick::VirtualPixelMethod</td><td>13</td></tr><tr><td class="t">Magick::QuantumExpressionOperator</td><td>12</td></tr><tr><td class="t">Mutex</td><td>12</td></tr><tr><td class="t">ActionController::MiddlewareStack::Middleware</td><td>12</td></tr><tr><td class="t">Magick::ImageType</td><td>12</td></tr><tr><td class="t">Magick::GravityType</td><td>12</td></tr><tr><td class="t">Rails::GemDependency</td><td>11</td></tr><tr><td class="t">Magick::CompressionType</td><td>11</td></tr><tr><td class="t">Rails::GemPlugin</td><td>11</td></tr><tr><td class="t">Magick::StretchType</td><td>10</td></tr><tr><td class="t">Magick::OrientationType</td><td>9</td></tr><tr><td class="t">Bignum</td><td>9</td></tr><tr><td class="t">ThinkingSphinx::Index::FauxColumn</td><td>9</td></tr><tr><td class="t">Magick::MetricType</td><td>8</td></tr><tr><td class="t">Magick::StorageType</td><td>8</td></tr></tbody></table><h3>Request history</h3>[59.01 MB] GET /stats/scrap<pre>Number of objects : 824154 (612748 AST nodes, 74.35%)
Heap slot size : 20
GC cycles so far : 26
Number of heaps : 7
Total size of objects: 16096.76 KB
Total size of heaps : 18036.66 KB (1939.91 KB = 10.76% unused)
Leading free slots : 13649 (266.58 KB = 1.48%)
Trailing free slots : 0 (0.00 KB = 0.00%)
Number of contiguous groups of 16 slots: 2113 (3.66%)
Number of terminal objects: 3890 (0.42%)
</pre><br></body></html>
9 changes: 9 additions & 0 deletions scrap.yml.example
@@ -0,0 +1,9 @@
max_requests: 150
max_objects: 10
classes:
"ActionController::Base": true
"ActionController::Session::AbstractStore::SessionHash": true
"ActiveRecord::Base":
print_objects: true
foo: bar
show_fields: [id, updated_at]

0 comments on commit ee99515

Please sign in to comment.