Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Initial commit. See readme for info.
- Loading branch information
Chris Heald
committed
Mar 24, 2009
0 parents
commit ee99515
Showing
6 changed files
with
248 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Original file line | Diff line number | Diff line change |
---|---|---|---|
@@ -0,0 +1,19 @@ | |||
Copyright (c) 2008 Chris Heald | |||
|
|||
Permission is hereby granted, free of charge, to any person obtaining a copy | |||
of this software and associated documentation files (the "Software"), to deal | |||
in the Software without restriction, including without limitation the rights | |||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | |||
copies of the Software, and to permit persons to whom the Software is | |||
furnished to do so, subject to the following conditions: | |||
|
|||
The above copyright notice and this permission notice shall be included in | |||
all copies or substantial portions of the Software. | |||
|
|||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | |||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | |||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | |||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN | |||
THE SOFTWARE. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Original file line | Diff line number | Diff line change |
---|---|---|---|
@@ -0,0 +1,19 @@ | |||
Scrap is a Rails Metal endpoint designed to expose various garbage and memory-related metrics about your app. It may be particularly useful in tracking down memory leaks. | |||
|
|||
To use it, simply install the plugin. This will provide a new url, <code>/stats/scrap</code>, which will report a number of metrics about your app. | |||
|
|||
h2. Config | |||
|
|||
If present, Scrap will use a config/scrap.yml file. See the provided example file for a list of the configuration options accepted. | |||
|
|||
* max requests: How many requests to keep a record of. Older requests will be pushed out of the queue when the limit has been reached. Default is 150. | |||
* max_objects: How many objects/deltas to show. Default is 50. | |||
* classes: A hash of class names to do object counting on. Values may be "true" which prints the object count with a default set of options, or it may be a hash consisting of the following: | |||
** print_objects: boolean - toggles the output of a representation of each instance of the type. | |||
** show_fields: array - list of fields to show per instance. This actually invokes the "attributes" method of the object, so it's really only useful for ActiveRecord objects. | |||
** small: boolean - if false, will not print counts in h3 tags. Default is true. | |||
** min: integer - minimum count, if set, that an object must have to appear in the delta or top objects list. Default is nil. | |||
|
|||
h2. Other considerations | |||
|
|||
Scrap will take advantage of many of the wonderful metrics provided by recent versions of Ruby Enterprise Edition. It will work with other versions of Ruby, but you'll get a lot more info out of it if you're running REE 1.8.6-20090201 or later. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Original file line | Diff line number | Diff line change |
---|---|---|---|
@@ -0,0 +1,184 @@ | |||
class Scrap < Rails::Rack::Metal | |||
PATH = "/stats/scrap".freeze | |||
COMMIFY_REGEX = /(\d)(?=(\d\d\d)+(?!\d))/ | |||
|
|||
@@gc_stats = {} | |||
@@last_gc_run = nil | |||
@@last_gc_mem = nil | |||
@@requests_processed = 0 | |||
@@request_list = [] | |||
@@alive_at = nil | |||
@@gc_stats_enabled = nil | |||
@@config = nil | |||
|
|||
def self.config | |||
@@config ||= YAML::load open(File.join(Rails.root, "config", "scrap.yml")).read | |||
rescue Errno::ENOENT | |||
@@config = {} | |||
rescue | |||
puts "[scrap] scrap.yml: #{$!.message}" | |||
@@config = {} | |||
end | |||
|
|||
def self.call(env) | |||
if !@@gc_stats_enabled then | |||
GC.enable_stats if GC.respond_to? :enable_stats | |||
@@gc_stats_enabled = true | |||
end | |||
@@requests_processed += 1 | |||
@@last_gc_run ||= @@alive_at ||= Time.now.to_f | |||
@@last_gc_mem ||= get_usage | |||
|
|||
req = sprintf("[%-10.2fMB] %s %s", get_usage, env["REQUEST_METHOD"], env["PATH_INFO"]) | |||
req << "<pre>#{ObjectSpace.statistics}</pre>" if ObjectSpace.respond_to? :statistics | |||
@@request_list.unshift req | |||
@@request_list.pop if @@request_list.length > (config["max_requests"] || 150) | |||
|
|||
if env["PATH_INFO"] == PATH | |||
gc_stats | |||
else | |||
NotFoundResponse | |||
end | |||
end | |||
|
|||
def self.gc_stats | |||
collected = nil | |||
puts "Respond to? #{ObjectSpace.respond_to? :live_objects}" | |||
if ObjectSpace.respond_to? :live_objects then | |||
live = ObjectSpace.live_objects | |||
GC.start | |||
collected = live - ObjectSpace.live_objects | |||
else | |||
GC.start | |||
end | |||
GC.start | |||
usage = get_usage | |||
|
|||
mem_delta = usage - @@last_gc_mem | |||
time_delta = Time.now.to_f - @@last_gc_run | |||
|
|||
s = "<title>[#{$$}] Garbage Report</title>" | |||
s << "<style> body { font-family: monospace; color: #222; } td { border-bottom: 1px solid #eee; padding: 1px 9px; } td.t { background: #fafafa; } tr:hover td { background: #fafaf0; border-color: #e0e0dd; } h1,h2,h3 { border-bottom: 1px solid #ddd; font-family: sans-serif; } </style>" | |||
|
|||
s << "<h1>Scrap - PID #{$$}</h1>" | |||
|
|||
s << "<table>" | |||
s << sprintf("<tr><td class='t'>Memory usage:</td><td>%2.2fMB</td></tr>", usage) | |||
s << sprintf("<tr><td class='t'>Delta:</td><td>%2.2fMB</td></tr>", mem_delta) | |||
s << sprintf("<tr><td class='t'>Last Scrap req:</td><td>%2.2f seconds ago</td></tr>", time_delta) | |||
s << sprintf("<tr><td class='t'>Requests processed:</td><td>%s</td></tr>", @@requests_processed) | |||
s << sprintf("<tr><td class='t'>Alive for:</td><td>%2.2f seconds</td></tr>", Time.now.to_f - @@alive_at) | |||
if GC.respond_to? :time then | |||
s << sprintf("<tr><td class='t'>Time spent in GC:</td><td>%2.2f seconds</td></tr>", GC.time / 1000000.0) | |||
end | |||
if collected | |||
s << sprintf("<tr><td class='t'>Collected objects:</td><td>%2d</td></tr>", collected) | |||
s << sprintf("<tr><td class='t'>Live objects:</td><td>%2d</td></tr>", ObjectSpace.live_objects) | |||
end | |||
s << "</table>" | |||
|
|||
s << "<h3>Top #{config["max_objects"]} deltas since last request</h3>" | |||
s << "<table border='0'>" | |||
memcheck(config["max_objects"], Object, :deltas).each do |v| | |||
next if v.last == 0 | |||
s << "<tr><td class='t'>#{v.first}</td><td>#{sprintf("%s%s", v.last >= 0 ? "+" : "-", commify(v.last))}</td></tr>" | |||
end | |||
s << "</table>" | |||
|
|||
s << "<h3>Top #{config["max_objects"]} objects</h3>" | |||
s << "<table border='0'>" | |||
memcheck(config["max_objects"]).each do |v| | |||
s << "<tr><td class='t'>#{v.first}</td><td>#{commify v.last}</td></tr>" | |||
end | |||
s << "</table>" | |||
|
|||
(config["classes"] || {}).each do |klass, val| | |||
puts val.inspect | |||
opts = val === true ? {"print_objects" => true} : val | |||
add_os(klass.constantize, s, opts) | |||
end | |||
|
|||
s << "<h3>Request history</h3>" | |||
@@request_list.each do |req| | |||
s << req | |||
s << "<br />" | |||
end | |||
|
|||
@@last_gc_run = Time.now.to_f | |||
@@last_gc_mem = usage | |||
@@requests_processed = 0 | |||
[200, {"Content-Type" => "text/html"}, s] | |||
end | |||
|
|||
def self.get_usage | |||
usage = 0 | |||
begin | |||
usage = `cat /proc/#{$$}/stat`.split(" ")[22].to_i / (1024 * 1024).to_f | |||
rescue | |||
# pass | |||
end | |||
return usage | |||
end | |||
|
|||
def self.add_os(c, s, options = {}) | |||
print_objects = options["print_objects"] | |||
small = options["small"] | |||
min = options["min"] | |||
show_fields = options["show_fields"] | |||
|
|||
ct = ObjectSpace.each_object(c) {} | |||
return if min and ct < min | |||
|
|||
if small | |||
s << "#{c} (#{ct})<br />" | |||
else | |||
s << "<h3>#{c} (#{ct})</h3>" | |||
end | |||
|
|||
return if !print_objects | |||
s << "<table>" | |||
val = ObjectSpace.each_object(c) do |m| | |||
s << "<tr><td class='t'>" << "<#{m.class.to_s}:#{sprintf("0x%.8x", m.object_id)}></td>" | |||
if show_fields then | |||
show_fields.each do |field| | |||
v = m.attributes[field.to_s] | |||
if v.blank? | |||
s << "<td> </td>" | |||
else | |||
s << "<td>#{field}: #{v}</td>" | |||
end | |||
end | |||
end | |||
s << "</tr>" | |||
end | |||
s << "</table>" | |||
end | |||
|
|||
def self.memcheck(top, klass = Object, mode = :normal) | |||
top ||= 50 | |||
os = Hash.new(0) | |||
ObjectSpace.each_object(klass) do |o| | |||
begin; | |||
# If this is true, it's an association proxy, and we don't want to invoke method_missing on it, | |||
# as it will result in a load of the association proxy from the DB, doing extra DB work and | |||
# potentially creating a lot of AR objects. Hackalicious. | |||
next if o.respond_to? :proxy_respond_to? | |||
os[o.class.to_s] += 1 if o.respond_to? :class | |||
rescue; end | |||
end | |||
if mode == :deltas then | |||
os2 = Hash.new(0) | |||
os.each do |k, v| | |||
os2[k] = v - (@@gc_stats[k] || 0) | |||
@@gc_stats[k] = v | |||
end | |||
sorted = os2.sort_by{|k,v| -v }.first(top) | |||
else | |||
sorted = os.sort_by{|k,v| -v }.first(top) | |||
end | |||
end | |||
|
|||
def self.commify(i) | |||
i.to_s.gsub(COMMIFY_REGEX, "\\1,") | |||
end | |||
end |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Original file line | Diff line number | Diff line change |
---|---|---|---|
@@ -0,0 +1,5 @@ | |||
begin | |||
Rails::Rack::Metal::metal_paths << File.join(File.dirname(__FILE__), "app", "metal") | |||
rescue NameError | |||
puts "[WARNING] Scrap requires Rails 2.3 or better. Not booting." | |||
end |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Original file line | Diff line number | Diff line change |
---|---|---|---|
@@ -0,0 +1,12 @@ | |||
<html><head> | |||
<meta http-equiv="content-type" content="text/html; charset=UTF-8"><title>[9752] Garbage Report</title><style> body { font-family: monospace; color: #222; } td { border-bottom: 1px solid #eee; padding: 1px 9px; } td.t { background: #fafafa; } tr:hover td { background: #fafaf0; border-color: #e0e0dd; } h1,h2,h3 { border-bottom: 1px solid #ddd; font-family: sans-serif; } </style></head><body><h1>Scrap - PID 9752</h1><table><tbody><tr><td class="t">Memory usage:</td><td>59.01MB</td></tr><tr><td class="t">Delta:</td><td>0.00MB</td></tr><tr><td class="t">Last Scrap req:</td><td>0.39 seconds ago</td></tr><tr><td class="t">Requests processed:</td><td>1</td></tr><tr><td class="t">Alive for:</td><td>0.39 seconds</td></tr><tr><td class="t">Time spent in GC:</td><td>0.18 seconds</td></tr><tr><td class="t">Collected objects:</td><td>309826</td></tr><tr><td class="t">Live objects:</td><td>696773</td></tr></tbody></table><h3>Top deltas since last request</h3><table border="0"><tbody><tr><td class="t">String</td><td>+153,220</td></tr><tr><td class="t">Array</td><td>+5,690</td></tr><tr><td class="t">Proc</td><td>+3,268</td></tr><tr><td class="t">Hash</td><td>+3,090</td></tr><tr><td class="t">Regexp</td><td>+1,978</td></tr><tr><td class="t">Class</td><td>+1,362</td></tr><tr><td class="t">ActionController::Routing::DividerSegment</td><td>+946</td></tr><tr><td class="t">ActiveSupport::Callbacks::Callback</td><td>+754</td></tr><tr><td class="t">ActionController::Routing::StaticSegment</td><td>+604</td></tr><tr><td class="t">Module</td><td>+563</td></tr><tr><td class="t">Gem::Version</td><td>+425</td></tr><tr><td class="t">Gem::Requirement</td><td>+389</td></tr><tr><td class="t">ActionController::Routing::Route</td><td>+305</td></tr><tr><td class="t">ActionController::Routing::DynamicSegment</td><td>+258</td></tr><tr><td class="t">ActionController::Routing::OptionalFormatSegment</td><td>+223</td></tr><tr><td class="t">ActiveSupport::Callbacks::CallbackChain</td><td>+218</td></tr><tr><td class="t">Range</td><td>+190</td></tr><tr><td class="t">ActiveRecord::Reflection::AssociationReflection</td><td>+142</td></tr><tr><td class="t">Time</td><td>+132</td></tr><tr><td class="t">Gem::Specification</td><td>+128</td></tr><tr><td class="t">Gem::Dependency</td><td>+120</td></tr><tr><td class="t">Float</td><td>+103</td></tr><tr><td class="t">UnboundMethod</td><td>+98</td></tr><tr><td class="t">Magick::CompositeOperator</td><td>+56</td></tr><tr><td class="t">Set</td><td>+34</td></tr><tr><td class="t">Magick::PreviewType</td><td>+30</td></tr><tr><td class="t">HashWithIndifferentAccess</td><td>+24</td></tr><tr><td class="t">Magick::ColorspaceType</td><td>+23</td></tr><tr><td class="t">Magick::FilterTypes</td><td>+22</td></tr><tr><td class="t">Mime::Type</td><td>+21</td></tr><tr><td class="t">ActiveRecord::Reflection::ThroughReflection</td><td>+19</td></tr><tr><td class="t">Magick::ChannelType</td><td>+18</td></tr><tr><td class="t">Magick::ImageLayerMethod</td><td>+16</td></tr><tr><td class="t">Rails::Plugin</td><td>+15</td></tr><tr><td class="t">Rails::OrderedOptions</td><td>+14</td></tr><tr><td class="t">Magick::VirtualPixelMethod</td><td>+13</td></tr><tr><td class="t">Magick::GravityType</td><td>+12</td></tr><tr><td class="t">Magick::QuantumExpressionOperator</td><td>+12</td></tr><tr><td class="t">ActionController::MiddlewareStack::Middleware</td><td>+12</td></tr><tr><td class="t">Magick::ImageType</td><td>+12</td></tr><tr><td class="t">Mutex</td><td>+12</td></tr><tr><td class="t">Rails::GemDependency</td><td>+11</td></tr><tr><td class="t">Rails::GemPlugin</td><td>+11</td></tr><tr><td class="t">Magick::CompressionType</td><td>+11</td></tr><tr><td class="t">Magick::StretchType</td><td>+10</td></tr><tr><td class="t">Bignum</td><td>+9</td></tr><tr><td class="t">Magick::OrientationType</td><td>+9</td></tr><tr><td class="t">ThinkingSphinx::Index::FauxColumn</td><td>+9</td></tr><tr><td class="t">Magick::InterlaceType</td><td>+8</td></tr><tr><td class="t">Magick::DistortImageMethod</td><td>+8</td></tr></tbody></table><h3>Top objects</h3><table border="0"><tbody><tr><td class="t">String</td><td>111,167</td></tr><tr><td class="t">Array</td><td>5,860</td></tr><tr><td class="t">Proc</td><td>3,268</td></tr><tr><td class="t">Hash</td><td>3,091</td></tr><tr><td class="t">Regexp</td><td>1,978</td></tr><tr><td class="t">Class</td><td>1,362</td></tr><tr><td class="t">ActionController::Routing::DividerSegment</td><td>946</td></tr><tr><td class="t">ActiveSupport::Callbacks::Callback</td><td>754</td></tr><tr><td class="t">ActionController::Routing::StaticSegment</td><td>604</td></tr><tr><td class="t">Module</td><td>563</td></tr><tr><td class="t">Gem::Version</td><td>425</td></tr><tr><td class="t">Gem::Requirement</td><td>389</td></tr><tr><td class="t">ActionController::Routing::Route</td><td>305</td></tr><tr><td class="t">ActionController::Routing::DynamicSegment</td><td>258</td></tr><tr><td class="t">ActionController::Routing::OptionalFormatSegment</td><td>223</td></tr><tr><td class="t">ActiveSupport::Callbacks::CallbackChain</td><td>218</td></tr><tr><td class="t">Range</td><td>190</td></tr><tr><td class="t">ActiveRecord::Reflection::AssociationReflection</td><td>142</td></tr><tr><td class="t">Time</td><td>130</td></tr><tr><td class="t">Gem::Specification</td><td>128</td></tr><tr><td class="t">Gem::Dependency</td><td>120</td></tr><tr><td class="t">UnboundMethod</td><td>98</td></tr><tr><td class="t">Float</td><td>94</td></tr><tr><td class="t">Magick::CompositeOperator</td><td>56</td></tr><tr><td class="t">Set</td><td>34</td></tr><tr><td class="t">Magick::PreviewType</td><td>30</td></tr><tr><td class="t">HashWithIndifferentAccess</td><td>24</td></tr><tr><td class="t">Magick::ColorspaceType</td><td>23</td></tr><tr><td class="t">Magick::FilterTypes</td><td>22</td></tr><tr><td class="t">Mime::Type</td><td>21</td></tr><tr><td class="t">ActiveRecord::Reflection::ThroughReflection</td><td>19</td></tr><tr><td class="t">Magick::ChannelType</td><td>18</td></tr><tr><td class="t">Magick::ImageLayerMethod</td><td>16</td></tr><tr><td class="t">Rails::Plugin</td><td>15</td></tr><tr><td class="t">Rails::OrderedOptions</td><td>14</td></tr><tr><td class="t">Magick::VirtualPixelMethod</td><td>13</td></tr><tr><td class="t">Magick::QuantumExpressionOperator</td><td>12</td></tr><tr><td class="t">Mutex</td><td>12</td></tr><tr><td class="t">ActionController::MiddlewareStack::Middleware</td><td>12</td></tr><tr><td class="t">Magick::ImageType</td><td>12</td></tr><tr><td class="t">Magick::GravityType</td><td>12</td></tr><tr><td class="t">Rails::GemDependency</td><td>11</td></tr><tr><td class="t">Magick::CompressionType</td><td>11</td></tr><tr><td class="t">Rails::GemPlugin</td><td>11</td></tr><tr><td class="t">Magick::StretchType</td><td>10</td></tr><tr><td class="t">Magick::OrientationType</td><td>9</td></tr><tr><td class="t">Bignum</td><td>9</td></tr><tr><td class="t">ThinkingSphinx::Index::FauxColumn</td><td>9</td></tr><tr><td class="t">Magick::MetricType</td><td>8</td></tr><tr><td class="t">Magick::StorageType</td><td>8</td></tr></tbody></table><h3>Request history</h3>[59.01 MB] GET /stats/scrap<pre>Number of objects : 824154 (612748 AST nodes, 74.35%) | |||
Heap slot size : 20 | |||
GC cycles so far : 26 | |||
Number of heaps : 7 | |||
Total size of objects: 16096.76 KB | |||
Total size of heaps : 18036.66 KB (1939.91 KB = 10.76% unused) | |||
Leading free slots : 13649 (266.58 KB = 1.48%) | |||
Trailing free slots : 0 (0.00 KB = 0.00%) | |||
Number of contiguous groups of 16 slots: 2113 (3.66%) | |||
Number of terminal objects: 3890 (0.42%) | |||
</pre><br></body></html> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Original file line | Diff line number | Diff line change |
---|---|---|---|
@@ -0,0 +1,9 @@ | |||
max_requests: 150 | |||
max_objects: 10 | |||
classes: | |||
"ActionController::Base": true | |||
"ActionController::Session::AbstractStore::SessionHash": true | |||
"ActiveRecord::Base": | |||
print_objects: true | |||
foo: bar | |||
show_fields: [id, updated_at] |