Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Newer
Older
100644 211 lines (181 sloc) 8.293 kb
c7bf1c3 Justin George queue time, initial header work
jaggederest authored
1 module NewRelic
2 module Agent
3 module Instrumentation
4 module QueueTime
b142a50 Justin George queue time in-flight - add middleware related methods
jaggederest authored
5 unless defined?(MAIN_HEADER)
6 MAIN_HEADER = 'HTTP_X_REQUEST_START'
7 MIDDLEWARE_HEADER = 'HTTP_X_MIDDLEWARE_START'
aca6854 Justin George queue time finally actually measuring queue time, wip
jaggederest authored
8 QUEUE_HEADER = 'HTTP_X_QUEUE_START'
9 ALT_QUEUE_HEADER = 'HTTP_X_QUEUE_TIME'
ab9157b Justin George add an alternate path for heroku to queue time detection
jaggederest authored
10 HEROKU_QUEUE_HEADER = 'HTTP_X_HEROKU_QUEUE_WAIT_TIME'
b142a50 Justin George queue time in-flight - add middleware related methods
jaggederest authored
11 APP_HEADER = 'HTTP_X_APPLICATION_START'
e3c53bb Justin George remove trailing whitespace
jaggederest authored
12
b142a50 Justin George queue time in-flight - add middleware related methods
jaggederest authored
13 HEADER_REGEX = /([^\s\/,(t=)]+)? ?t=([0-9]+)/
14 SERVER_METRIC = 'WebFrontend/WebServer/'
32cdce8 Justin George queue time now includes middleware header parsing
jaggederest authored
15 MIDDLEWARE_METRIC = 'Middleware/'
aca6854 Justin George queue time finally actually measuring queue time, wip
jaggederest authored
16 # no individual queue metric - more than one queue?!
32cdce8 Justin George queue time now includes middleware header parsing
jaggederest authored
17 ALL_SERVER_METRIC = 'WebFrontend/WebServer/all'
18 ALL_MIDDLEWARE_METRIC = 'Middleware/all'
aca6854 Justin George queue time finally actually measuring queue time, wip
jaggederest authored
19 ALL_QUEUE_METRIC = 'WebFrontend/QueueTime'
20 end
21
3b5cff8 Justin George new fun with integration testing, including it in the controller instrum...
jaggederest authored
22 def parse_frontend_headers(headers)
e752302 Justin George update controller instrumentation to catch errors in queue time stuff, m...
jaggederest authored
23 # these methods add internal state, so we dup so other parts
24 # of the app don't have to worry about it.
25 # May have performance implications with very large env hashes
3b5cff8 Justin George new fun with integration testing, including it in the controller instrum...
jaggederest authored
26 env = headers.dup
0b78241 Justin George remove a method name conflict with our website
jaggederest authored
27 add_end_time_header(Time.now, env)
3d06e4f Justin George fixing broken tests, improving queue time so that it is reflected in apd...
jaggederest authored
28 middleware_start = parse_middleware_time_from(env)
29 queue_start = parse_queue_time_from(env)
30 server_start = parse_server_time_from(env)
907f4e4 Justin George queue time for browser monitoring
jaggederest authored
31 # returned for the controller instrumentation
3d06e4f Justin George fixing broken tests, improving queue time so that it is reflected in apd...
jaggederest authored
32 [middleware_start, queue_start, server_start].min
b142a50 Justin George queue time in-flight - add middleware related methods
jaggederest authored
33 end
e3c53bb Justin George remove trailing whitespace
jaggederest authored
34
aca6854 Justin George queue time finally actually measuring queue time, wip
jaggederest authored
35 private
e3c53bb Justin George remove trailing whitespace
jaggederest authored
36
94e449d Justin George inflight queue time - add test for overlap
jaggederest authored
37 # main method to extract server time info from env hash,
c7bf1c3 Justin George queue time, initial header work
jaggederest authored
38 # records individual server metrics and one roll-up for all servers
94e449d Justin George inflight queue time - add test for overlap
jaggederest authored
39 def parse_server_time_from(env)
b142a50 Justin George queue time in-flight - add middleware related methods
jaggederest authored
40 end_time = parse_end_time(env)
41 matches = get_matches_from_header(MAIN_HEADER, env)
e3c53bb Justin George remove trailing whitespace
jaggederest authored
42
b142a50 Justin George queue time in-flight - add middleware related methods
jaggederest authored
43 record_individual_server_stats(end_time, matches)
44 record_rollup_server_stat(end_time, matches)
45 end
46
47 def parse_middleware_time_from(env)
48 end_time = parse_end_time(env)
49 matches = get_matches_from_header(MIDDLEWARE_HEADER, env)
50
51 record_individual_middleware_stats(end_time, matches)
3d06e4f Justin George fixing broken tests, improving queue time so that it is reflected in apd...
jaggederest authored
52 oldest_time = record_rollup_middleware_stat(end_time, matches)
aca6854 Justin George queue time finally actually measuring queue time, wip
jaggederest authored
53 # notice this bit: we reset the end time to the earliest
54 # middleware tag so that other frontend metrics don't
55 # include this time.
3d06e4f Justin George fixing broken tests, improving queue time so that it is reflected in apd...
jaggederest authored
56 add_end_time_header(oldest_time, env)
57 oldest_time
aca6854 Justin George queue time finally actually measuring queue time, wip
jaggederest authored
58 end
59
60 def parse_queue_time_from(env)
e3c53bb Justin George remove trailing whitespace
jaggederest authored
61 oldest_time = nil
aca6854 Justin George queue time finally actually measuring queue time, wip
jaggederest authored
62 end_time = parse_end_time(env)
63 alternate_length = check_for_alternate_queue_length(env)
64 if alternate_length
65 # skip all that fancy-dan stuff
66 NewRelic::Agent.get_stats(ALL_QUEUE_METRIC).trace_call(alternate_length)
3d06e4f Justin George fixing broken tests, improving queue time so that it is reflected in apd...
jaggederest authored
67 oldest_time = (end_time - alternate_length) # should be a time
aca6854 Justin George queue time finally actually measuring queue time, wip
jaggederest authored
68 else
69 matches = get_matches_from_header(QUEUE_HEADER, env)
3d06e4f Justin George fixing broken tests, improving queue time so that it is reflected in apd...
jaggederest authored
70 oldest_time = record_rollup_queue_stat(end_time, matches)
aca6854 Justin George queue time finally actually measuring queue time, wip
jaggederest authored
71 end
72 # notice this bit: we reset the end time to the earliest
73 # queue tag or the start time minus the queue time so that
74 # other frontend metrics don't include this time.
3d06e4f Justin George fixing broken tests, improving queue time so that it is reflected in apd...
jaggederest authored
75 add_end_time_header(oldest_time, env)
76 oldest_time
aca6854 Justin George queue time finally actually measuring queue time, wip
jaggederest authored
77 end
78
79 def check_for_alternate_queue_length(env)
ab9157b Justin George add an alternate path for heroku to queue time detection
jaggederest authored
80 heroku_length = check_for_heroku_queue_length(env)
81 return heroku_length if heroku_length
aca6854 Justin George queue time finally actually measuring queue time, wip
jaggederest authored
82 header = env[ALT_QUEUE_HEADER]
83 return nil unless header
84 (header.gsub('t=', '').to_i / 1_000_000.0)
c7bf1c3 Justin George queue time, initial header work
jaggederest authored
85 end
86
ab9157b Justin George add an alternate path for heroku to queue time detection
jaggederest authored
87 def check_for_heroku_queue_length(env)
88 header = env[HEROKU_QUEUE_HEADER]
89 return nil unless header
90 (header.gsub(/[^0-9]/, '').to_i / 1_000.0)
91 end
92
b142a50 Justin George queue time in-flight - add middleware related methods
jaggederest authored
93 def get_matches_from_header(header, env)
94 return [] if env.nil?
95 get_matches(env[header]).map do |name, time|
96 convert_to_name_time_pair(name, time)
97 end
98 end
99
100 def get_matches(string)
101 string.to_s.scan(HEADER_REGEX)
102 end
103
104 def convert_to_name_time_pair(name, time)
105 [name, convert_from_microseconds(time.to_i)]
106 end
32cdce8 Justin George queue time now includes middleware header parsing
jaggederest authored
107
108 def record_individual_stat_of_type(type, end_time, matches)
109 matches = matches.sort_by {|name, time| time }
110 matches.reverse!
111 matches.inject(end_time) {|end_time, pair|
112 name, time = pair
aca6854 Justin George queue time finally actually measuring queue time, wip
jaggederest authored
113 self.send(type, name, time, end_time) if name
32cdce8 Justin George queue time now includes middleware header parsing
jaggederest authored
114 time
115 }
116 end
e3c53bb Justin George remove trailing whitespace
jaggederest authored
117
c7bf1c3 Justin George queue time, initial header work
jaggederest authored
118 # goes through the list of servers and records each one in
119 # reverse order, subtracting the time for each successive
120 # server from the earlier ones in the list.
121 # an example because it's complicated:
122 # start data:
123 # [['a', Time.at(1000)], ['b', Time.at(1001)]], start time: Time.at(1002)
124 # initial run: Time.at(1002), ['b', Time.at(1001)]
125 # next: Time.at(1001), ['a', Time.at(1000)]
126 # see tests for more
127 def record_individual_server_stats(end_time, matches) # (Time, [[String, Time]]) -> nil
aca6854 Justin George queue time finally actually measuring queue time, wip
jaggederest authored
128 record_individual_stat_of_type(:record_server_time_for, end_time, matches)
c7bf1c3 Justin George queue time, initial header work
jaggederest authored
129 end
130
32cdce8 Justin George queue time now includes middleware header parsing
jaggederest authored
131 def record_individual_middleware_stats(end_time, matches)
aca6854 Justin George queue time finally actually measuring queue time, wip
jaggederest authored
132 record_individual_stat_of_type(:record_middleware_time_for, end_time, matches)
32cdce8 Justin George queue time now includes middleware header parsing
jaggederest authored
133 end
e3c53bb Justin George remove trailing whitespace
jaggederest authored
134
c7bf1c3 Justin George queue time, initial header work
jaggederest authored
135 # records the total time for all servers in a rollup metric
b142a50 Justin George queue time in-flight - add middleware related methods
jaggederest authored
136 def record_rollup_server_stat(end_time, matches) # (Time, [String, Time]) -> nil
030cc35 Justin George queue time testing, implementation of alt queue time check
jaggederest authored
137 record_rollup_stat_of_type(ALL_SERVER_METRIC, end_time, matches)
c7bf1c3 Justin George queue time, initial header work
jaggederest authored
138 end
32cdce8 Justin George queue time now includes middleware header parsing
jaggederest authored
139
140 def record_rollup_middleware_stat(end_time, matches)
030cc35 Justin George queue time testing, implementation of alt queue time check
jaggederest authored
141 record_rollup_stat_of_type(ALL_MIDDLEWARE_METRIC, end_time, matches)
142 end
143
144 def record_rollup_queue_stat(end_time, matches)
145 record_rollup_stat_of_type(ALL_QUEUE_METRIC, end_time, matches)
146 end
147
148 def record_rollup_stat_of_type(metric, end_time, matches)
3d06e4f Justin George fixing broken tests, improving queue time so that it is reflected in apd...
jaggederest authored
149 oldest_time = find_oldest_time(matches) || end_time
150 record_time_stat(metric, oldest_time, end_time)
151 oldest_time
32cdce8 Justin George queue time now includes middleware header parsing
jaggederest authored
152 end
e3c53bb Justin George remove trailing whitespace
jaggederest authored
153
c7bf1c3 Justin George queue time, initial header work
jaggederest authored
154 # searches for the first server to touch a request
155 def find_oldest_time(matches) # [[String, Time]] -> Time
156 matches.map do |name, time|
157 time
158 end.min
159 end
e3c53bb Justin George remove trailing whitespace
jaggederest authored
160
c7bf1c3 Justin George queue time, initial header work
jaggederest authored
161 # basically just assembles the metric name
94e449d Justin George inflight queue time - add test for overlap
jaggederest authored
162 def record_server_time_for(name, start_time, end_time) # (Maybe String, Time, Time) -> nil
b142a50 Justin George queue time in-flight - add middleware related methods
jaggederest authored
163 record_time_stat(SERVER_METRIC + name, start_time, end_time) if name
c7bf1c3 Justin George queue time, initial header work
jaggederest authored
164 end
165
32cdce8 Justin George queue time now includes middleware header parsing
jaggederest authored
166 def record_middleware_time_for(name, start_time, end_time)
167 record_time_stat(MIDDLEWARE_METRIC + name, start_time, end_time)
168 end
169
c7bf1c3 Justin George queue time, initial header work
jaggederest authored
170 # Checks that the time is not negative, and does the actual
171 # data recording
172 def record_time_stat(name, start_time, end_time) # (String, Time, Time) -> nil
173 total_time = end_time - start_time
174 if total_time < 0
b142a50 Justin George queue time in-flight - add middleware related methods
jaggederest authored
175 raise "should not provide an end time less than start time: #{end_time} is less than #{start_time}"
c7bf1c3 Justin George queue time, initial header work
jaggederest authored
176 else
177 NewRelic::Agent.get_stats(name).trace_call(total_time)
178 end
179 end
180
b142a50 Justin George queue time in-flight - add middleware related methods
jaggederest authored
181 def add_end_time_header(end_time, env) # (Time, Env) -> nil
3b5cff8 Justin George new fun with integration testing, including it in the controller instrum...
jaggederest authored
182 return unless end_time
b142a50 Justin George queue time in-flight - add middleware related methods
jaggederest authored
183 env[APP_HEADER] = "t=#{convert_to_microseconds(end_time)}"
184 end
185
186 def parse_end_time(env)
187 header = env[APP_HEADER]
188 return Time.now unless header
189 convert_from_microseconds(header.gsub('t=', '').to_i)
190 end
191
c7bf1c3 Justin George queue time, initial header work
jaggederest authored
192 # convert a time to the value provided by the header, for convenience
193 def convert_to_microseconds(time) # Time -> Int
194 raise TypeError.new('Cannot convert a non-time into microseconds') unless time.is_a?(Time) || time.is_a?(Numeric)
195 return time if time.is_a?(Numeric)
aca6854 Justin George queue time finally actually measuring queue time, wip
jaggederest authored
196 (time.to_f * 1_000_000).to_i
c7bf1c3 Justin George queue time, initial header work
jaggederest authored
197 end
e3c53bb Justin George remove trailing whitespace
jaggederest authored
198
c7bf1c3 Justin George queue time, initial header work
jaggederest authored
199 # convert a time from the header value (time in microseconds)
200 # into a ruby time object
201 def convert_from_microseconds(int) # Int -> Time
202 raise TypeError.new('Cannot convert a non-number into a time') unless int.is_a?(Time) || int.is_a?(Numeric)
b142a50 Justin George queue time in-flight - add middleware related methods
jaggederest authored
203 return int if int.is_a?(Time)
aca6854 Justin George queue time finally actually measuring queue time, wip
jaggederest authored
204 Time.at((int / 1_000_000.0))
c7bf1c3 Justin George queue time, initial header work
jaggederest authored
205 end
206 end
207 end
208 end
209 end
210
Something went wrong with that request. Please try again.