Skip to content
Newer
Older
100644 211 lines (181 sloc) 8.18 KB
c7bf1c3 @jaggederest queue time, initial header work
jaggederest authored Dec 22, 2010
1 module NewRelic
2 module Agent
3 module Instrumentation
4 module QueueTime
b142a50 @jaggederest queue time in-flight - add middleware related methods
jaggederest authored Feb 17, 2011
5 unless defined?(MAIN_HEADER)
6 MAIN_HEADER = 'HTTP_X_REQUEST_START'
7 MIDDLEWARE_HEADER = 'HTTP_X_MIDDLEWARE_START'
aca6854 @jaggederest queue time finally actually measuring queue time, wip
jaggederest authored Feb 25, 2011
8 QUEUE_HEADER = 'HTTP_X_QUEUE_START'
9 ALT_QUEUE_HEADER = 'HTTP_X_QUEUE_TIME'
ab9157b @jaggederest add an alternate path for heroku to queue time detection
jaggederest authored Mar 2, 2011
10 HEROKU_QUEUE_HEADER = 'HTTP_X_HEROKU_QUEUE_WAIT_TIME'
b142a50 @jaggederest queue time in-flight - add middleware related methods
jaggederest authored Feb 18, 2011
11 APP_HEADER = 'HTTP_X_APPLICATION_START'
e3c53bb @jaggederest remove trailing whitespace
jaggederest authored Apr 28, 2011
12
b142a50 @jaggederest queue time in-flight - add middleware related methods
jaggederest authored Feb 18, 2011
13 HEADER_REGEX = /([^\s\/,(t=)]+)? ?t=([0-9]+)/
14 SERVER_METRIC = 'WebFrontend/WebServer/'
32cdce8 @jaggederest queue time now includes middleware header parsing
jaggederest authored Feb 17, 2011
15 MIDDLEWARE_METRIC = 'Middleware/'
aca6854 @jaggederest queue time finally actually measuring queue time, wip
jaggederest authored Feb 25, 2011
16 # no individual queue metric - more than one queue?!
32cdce8 @jaggederest queue time now includes middleware header parsing
jaggederest authored Feb 18, 2011
17 ALL_SERVER_METRIC = 'WebFrontend/WebServer/all'
18 ALL_MIDDLEWARE_METRIC = 'Middleware/all'
aca6854 @jaggederest queue time finally actually measuring queue time, wip
jaggederest authored Feb 25, 2011
19 ALL_QUEUE_METRIC = 'WebFrontend/QueueTime'
20 end
21
3b5cff8 @jaggederest new fun with integration testing, including it in the controller inst…
jaggederest authored Feb 28, 2011
22 def parse_frontend_headers(headers)
e752302 @jaggederest update controller instrumentation to catch errors in queue time stuff…
jaggederest authored Mar 1, 2011
23 # these methods add internal state, so we dup so other parts
24 # of the app don't have to worry about it.
25 # May have performance implications with very large env hashes
3b5cff8 @jaggederest new fun with integration testing, including it in the controller inst…
jaggederest authored Feb 28, 2011
26 env = headers.dup
0b78241 @jaggederest remove a method name conflict with our website
jaggederest authored Apr 27, 2011
27 add_end_time_header(Time.now, env)
3d06e4f @jaggederest fixing broken tests, improving queue time so that it is reflected in …
jaggederest authored Apr 22, 2011
28 middleware_start = parse_middleware_time_from(env)
29 queue_start = parse_queue_time_from(env)
30 server_start = parse_server_time_from(env)
907f4e4 @jaggederest queue time for browser monitoring
jaggederest authored Apr 21, 2011
31 # returned for the controller instrumentation
3d06e4f @jaggederest fixing broken tests, improving queue time so that it is reflected in …
jaggederest authored Apr 22, 2011
32 [middleware_start, queue_start, server_start].min
b142a50 @jaggederest queue time in-flight - add middleware related methods
jaggederest authored Feb 18, 2011
33 end
e3c53bb @jaggederest remove trailing whitespace
jaggederest authored Apr 28, 2011
34
aca6854 @jaggederest queue time finally actually measuring queue time, wip
jaggederest authored Feb 25, 2011
35 private
e3c53bb @jaggederest remove trailing whitespace
jaggederest authored Apr 28, 2011
36
94e449d @jaggederest inflight queue time - add test for overlap
jaggederest authored Feb 17, 2011
37 # main method to extract server time info from env hash,
c7bf1c3 @jaggederest queue time, initial header work
jaggederest authored Dec 23, 2010
38 # records individual server metrics and one roll-up for all servers
94e449d @jaggederest inflight queue time - add test for overlap
jaggederest authored Feb 18, 2011
39 def parse_server_time_from(env)
b142a50 @jaggederest queue time in-flight - add middleware related methods
jaggederest authored Feb 18, 2011
40 end_time = parse_end_time(env)
41 matches = get_matches_from_header(MAIN_HEADER, env)
e3c53bb @jaggederest remove trailing whitespace
jaggederest authored Apr 28, 2011
42
b142a50 @jaggederest queue time in-flight - add middleware related methods
jaggederest authored Feb 18, 2011
43 record_individual_server_stats(end_time, matches)
44 record_rollup_server_stat(end_time, matches)
45 end
46
47 def parse_middleware_time_from(env)
48 end_time = parse_end_time(env)
49 matches = get_matches_from_header(MIDDLEWARE_HEADER, env)
50
51 record_individual_middleware_stats(end_time, matches)
3d06e4f @jaggederest fixing broken tests, improving queue time so that it is reflected in …
jaggederest authored Apr 22, 2011
52 oldest_time = record_rollup_middleware_stat(end_time, matches)
aca6854 @jaggederest queue time finally actually measuring queue time, wip
jaggederest authored Feb 25, 2011
53 # notice this bit: we reset the end time to the earliest
54 # middleware tag so that other frontend metrics don't
55 # include this time.
3d06e4f @jaggederest fixing broken tests, improving queue time so that it is reflected in …
jaggederest authored Apr 22, 2011
56 add_end_time_header(oldest_time, env)
57 oldest_time
aca6854 @jaggederest queue time finally actually measuring queue time, wip
jaggederest authored Feb 25, 2011
58 end
59
60 def parse_queue_time_from(env)
e3c53bb @jaggederest remove trailing whitespace
jaggederest authored Apr 28, 2011
61 oldest_time = nil
aca6854 @jaggederest queue time finally actually measuring queue time, wip
jaggederest authored Feb 25, 2011
62 end_time = parse_end_time(env)
63 alternate_length = check_for_alternate_queue_length(env)
64 if alternate_length
65 # skip all that fancy-dan stuff
66 NewRelic::Agent.get_stats(ALL_QUEUE_METRIC).trace_call(alternate_length)
3d06e4f @jaggederest fixing broken tests, improving queue time so that it is reflected in …
jaggederest authored Apr 22, 2011
67 oldest_time = (end_time - alternate_length) # should be a time
aca6854 @jaggederest queue time finally actually measuring queue time, wip
jaggederest authored Feb 25, 2011
68 else
69 matches = get_matches_from_header(QUEUE_HEADER, env)
3d06e4f @jaggederest fixing broken tests, improving queue time so that it is reflected in …
jaggederest authored Apr 22, 2011
70 oldest_time = record_rollup_queue_stat(end_time, matches)
aca6854 @jaggederest queue time finally actually measuring queue time, wip
jaggederest authored Feb 25, 2011
71 end
72 # notice this bit: we reset the end time to the earliest
73 # queue tag or the start time minus the queue time so that
74 # other frontend metrics don't include this time.
3d06e4f @jaggederest fixing broken tests, improving queue time so that it is reflected in …
jaggederest authored Apr 22, 2011
75 add_end_time_header(oldest_time, env)
76 oldest_time
aca6854 @jaggederest queue time finally actually measuring queue time, wip
jaggederest authored Feb 25, 2011
77 end
78
79 def check_for_alternate_queue_length(env)
ab9157b @jaggederest add an alternate path for heroku to queue time detection
jaggederest authored Mar 2, 2011
80 heroku_length = check_for_heroku_queue_length(env)
81 return heroku_length if heroku_length
aca6854 @jaggederest queue time finally actually measuring queue time, wip
jaggederest authored Feb 25, 2011
82 header = env[ALT_QUEUE_HEADER]
83 return nil unless header
84 (header.gsub('t=', '').to_i / 1_000_000.0)
c7bf1c3 @jaggederest queue time, initial header work
jaggederest authored Dec 23, 2010
85 end
86
ab9157b @jaggederest add an alternate path for heroku to queue time detection
jaggederest authored Mar 2, 2011
87 def check_for_heroku_queue_length(env)
88 header = env[HEROKU_QUEUE_HEADER]
89 return nil unless header
90 (header.gsub(/[^0-9]/, '').to_i / 1_000.0)
91 end
92
b142a50 @jaggederest queue time in-flight - add middleware related methods
jaggederest authored Feb 18, 2011
93 def get_matches_from_header(header, env)
94 return [] if env.nil?
95 get_matches(env[header]).map do |name, time|
96 convert_to_name_time_pair(name, time)
97 end
98 end
99
100 def get_matches(string)
101 string.to_s.scan(HEADER_REGEX)
102 end
103
104 def convert_to_name_time_pair(name, time)
105 [name, convert_from_microseconds(time.to_i)]
106 end
32cdce8 @jaggederest queue time now includes middleware header parsing
jaggederest authored Feb 18, 2011
107
108 def record_individual_stat_of_type(type, end_time, matches)
109 matches = matches.sort_by {|name, time| time }
110 matches.reverse!
111 matches.inject(end_time) {|end_time, pair|
112 name, time = pair
aca6854 @jaggederest queue time finally actually measuring queue time, wip
jaggederest authored Feb 25, 2011
113 self.send(type, name, time, end_time) if name
32cdce8 @jaggederest queue time now includes middleware header parsing
jaggederest authored Feb 18, 2011
114 time
115 }
116 end
e3c53bb @jaggederest remove trailing whitespace
jaggederest authored Apr 28, 2011
117
c7bf1c3 @jaggederest queue time, initial header work
jaggederest authored Dec 23, 2010
118 # goes through the list of servers and records each one in
119 # reverse order, subtracting the time for each successive
120 # server from the earlier ones in the list.
121 # an example because it's complicated:
122 # start data:
123 # [['a', Time.at(1000)], ['b', Time.at(1001)]], start time: Time.at(1002)
124 # initial run: Time.at(1002), ['b', Time.at(1001)]
125 # next: Time.at(1001), ['a', Time.at(1000)]
126 # see tests for more
127 def record_individual_server_stats(end_time, matches) # (Time, [[String, Time]]) -> nil
aca6854 @jaggederest queue time finally actually measuring queue time, wip
jaggederest authored Feb 25, 2011
128 record_individual_stat_of_type(:record_server_time_for, end_time, matches)
c7bf1c3 @jaggederest queue time, initial header work
jaggederest authored Dec 23, 2010
129 end
130
32cdce8 @jaggederest queue time now includes middleware header parsing
jaggederest authored Feb 18, 2011
131 def record_individual_middleware_stats(end_time, matches)
aca6854 @jaggederest queue time finally actually measuring queue time, wip
jaggederest authored Feb 25, 2011
132 record_individual_stat_of_type(:record_middleware_time_for, end_time, matches)
32cdce8 @jaggederest queue time now includes middleware header parsing
jaggederest authored Feb 18, 2011
133 end
e3c53bb @jaggederest remove trailing whitespace
jaggederest authored Apr 28, 2011
134
c7bf1c3 @jaggederest queue time, initial header work
jaggederest authored Dec 23, 2010
135 # records the total time for all servers in a rollup metric
b142a50 @jaggederest queue time in-flight - add middleware related methods
jaggederest authored Feb 18, 2011
136 def record_rollup_server_stat(end_time, matches) # (Time, [String, Time]) -> nil
030cc35 @jaggederest queue time testing, implementation of alt queue time check
jaggederest authored Feb 25, 2011
137 record_rollup_stat_of_type(ALL_SERVER_METRIC, end_time, matches)
c7bf1c3 @jaggederest queue time, initial header work
jaggederest authored Dec 23, 2010
138 end
32cdce8 @jaggederest queue time now includes middleware header parsing
jaggederest authored Feb 18, 2011
139
140 def record_rollup_middleware_stat(end_time, matches)
030cc35 @jaggederest queue time testing, implementation of alt queue time check
jaggederest authored Feb 25, 2011
141 record_rollup_stat_of_type(ALL_MIDDLEWARE_METRIC, end_time, matches)
142 end
143
144 def record_rollup_queue_stat(end_time, matches)
145 record_rollup_stat_of_type(ALL_QUEUE_METRIC, end_time, matches)
146 end
147
148 def record_rollup_stat_of_type(metric, end_time, matches)
3d06e4f @jaggederest fixing broken tests, improving queue time so that it is reflected in …
jaggederest authored Apr 22, 2011
149 oldest_time = find_oldest_time(matches) || end_time
150 record_time_stat(metric, oldest_time, end_time)
151 oldest_time
32cdce8 @jaggederest queue time now includes middleware header parsing
jaggederest authored Feb 18, 2011
152 end
e3c53bb @jaggederest remove trailing whitespace
jaggederest authored Apr 28, 2011
153
c7bf1c3 @jaggederest queue time, initial header work
jaggederest authored Dec 23, 2010
154 # searches for the first server to touch a request
155 def find_oldest_time(matches) # [[String, Time]] -> Time
156 matches.map do |name, time|
157 time
158 end.min
159 end
e3c53bb @jaggederest remove trailing whitespace
jaggederest authored Apr 28, 2011
160
c7bf1c3 @jaggederest queue time, initial header work
jaggederest authored Dec 23, 2010
161 # basically just assembles the metric name
94e449d @jaggederest inflight queue time - add test for overlap
jaggederest authored Feb 18, 2011
162 def record_server_time_for(name, start_time, end_time) # (Maybe String, Time, Time) -> nil
b142a50 @jaggederest queue time in-flight - add middleware related methods
jaggederest authored Feb 18, 2011
163 record_time_stat(SERVER_METRIC + name, start_time, end_time) if name
c7bf1c3 @jaggederest queue time, initial header work
jaggederest authored Dec 23, 2010
164 end
165
32cdce8 @jaggederest queue time now includes middleware header parsing
jaggederest authored Feb 18, 2011
166 def record_middleware_time_for(name, start_time, end_time)
167 record_time_stat(MIDDLEWARE_METRIC + name, start_time, end_time)
168 end
169
c7bf1c3 @jaggederest queue time, initial header work
jaggederest authored Dec 23, 2010
170 # Checks that the time is not negative, and does the actual
171 # data recording
172 def record_time_stat(name, start_time, end_time) # (String, Time, Time) -> nil
173 total_time = end_time - start_time
174 if total_time < 0
9bdc398 @samg Improve log message for queue time fail
samg authored Dec 19, 2011
175 raise "should not provide an end time less than start time: #{end_time} #{end_time.strftime('%N')} is less than #{start_time} #{start_time.strftime('%N')} (total_time=#{total_time}) "
c7bf1c3 @jaggederest queue time, initial header work
jaggederest authored Dec 23, 2010
176 else
177 NewRelic::Agent.get_stats(name).trace_call(total_time)
178 end
179 end
180
b142a50 @jaggederest queue time in-flight - add middleware related methods
jaggederest authored Feb 18, 2011
181 def add_end_time_header(end_time, env) # (Time, Env) -> nil
3b5cff8 @jaggederest new fun with integration testing, including it in the controller inst…
jaggederest authored Feb 28, 2011
182 return unless end_time
b142a50 @jaggederest queue time in-flight - add middleware related methods
jaggederest authored Feb 18, 2011
183 env[APP_HEADER] = "t=#{convert_to_microseconds(end_time)}"
184 end
185
186 def parse_end_time(env)
187 header = env[APP_HEADER]
188 return Time.now unless header
189 convert_from_microseconds(header.gsub('t=', '').to_i)
190 end
191
c7bf1c3 @jaggederest queue time, initial header work
jaggederest authored Dec 23, 2010
192 # convert a time to the value provided by the header, for convenience
193 def convert_to_microseconds(time) # Time -> Int
194 raise TypeError.new('Cannot convert a non-time into microseconds') unless time.is_a?(Time) || time.is_a?(Numeric)
195 return time if time.is_a?(Numeric)
aca6854 @jaggederest queue time finally actually measuring queue time, wip
jaggederest authored Feb 25, 2011
196 (time.to_f * 1_000_000).to_i
c7bf1c3 @jaggederest queue time, initial header work
jaggederest authored Dec 23, 2010
197 end
e3c53bb @jaggederest remove trailing whitespace
jaggederest authored Apr 28, 2011
198
c7bf1c3 @jaggederest queue time, initial header work
jaggederest authored Dec 23, 2010
199 # convert a time from the header value (time in microseconds)
200 # into a ruby time object
201 def convert_from_microseconds(int) # Int -> Time
202 raise TypeError.new('Cannot convert a non-number into a time') unless int.is_a?(Time) || int.is_a?(Numeric)
b142a50 @jaggederest queue time in-flight - add middleware related methods
jaggederest authored Feb 18, 2011
203 return int if int.is_a?(Time)
aca6854 @jaggederest queue time finally actually measuring queue time, wip
jaggederest authored Feb 25, 2011
204 Time.at((int / 1_000_000.0))
c7bf1c3 @jaggederest queue time, initial header work
jaggederest authored Dec 23, 2010
205 end
206 end
207 end
208 end
209 end
210
Something went wrong with that request. Please try again.