Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Newer
Older
100644 259 lines (191 sloc) 6.128 kb
1592e82 TJ Holowaychuk Initial commit
authored
1
2 # jog
3
4 JSON logging & reporting inspired by Loggly.
5
2e59ea3 TJ Holowaychuk docs
authored
6 ## Installation
7
8 $ npm install jog
9
9757709 TJ Holowaychuk features
authored
10 ## Features
11
12 - namespace support
13 - rich json documents
14 - log levels
15 - file store
16 - redis store
17 - document streaming
18 - tail -f like streaming
19 - CLI to tail and map / reduce logs
20
b781b11 TJ Holowaychuk move api docs up
authored
21 ## API
22
240d053 TJ Holowaychuk Changed .msg -> .type
authored
23 ### log.write(level, type[, obj])
b781b11 TJ Holowaychuk move api docs up
authored
24
25 Write to the logs:
26
27 ```js
240d053 TJ Holowaychuk Changed .msg -> .type
authored
28 log.write(level, type[, obj])
29 log.debug(type[, obj])
30 log.info(type[, obj])
31 log.warn(type[, obj])
32 log.error(type[, obj])
b781b11 TJ Holowaychuk move api docs up
authored
33 ```
34
f57c23c TJ Holowaychuk ns docs
authored
35 ### log.ns(obj)
36
37 Namespace with the given `obj`, returning a new `Jog` instance
38 inheriting previous properties. You may call this several times
39 to produce more and more specific loggers.
40
41 ```js
0547aa8 TJ Holowaychuk typo
authored
42 var log = jog(new jog.FileStore('/tmp/log'));
f57c23c TJ Holowaychuk ns docs
authored
43
44 // log a user 5
45 log = log.ns({ uid: 5 });
46
47 // log video id 99 for user 5
48 log = log.ns({ vid: 99 });
49
50 // or both at once
51 log = log.ns({ uid: 5, vid: 99 });
52 ```
53
b781b11 TJ Holowaychuk move api docs up
authored
54 ### log.stream(options)
55
56 Return an `EventEmitter` emitting "data" and "end" events.
57
58 - `end` when __false__ streaming will not end
59 - `interval` the interval at which to poll (store-specific)
60
61 ### log.clear(callback)
62
63 Clear the logs and invoke the callback.
64
2e59ea3 TJ Holowaychuk docs
authored
65 ## Example
66
67 Log random data using the `FileStore` and tail the file
b84fd89 TJ Holowaychuk docs
authored
68 for changes (typically in different processes). Jog will add
240d053 TJ Holowaychuk Changed .msg -> .type
authored
69 the `.level` and `.type` properties for you.
2e59ea3 TJ Holowaychuk docs
authored
70
71 ```js
72 var jog = require('jog')
73 , log = jog(new jog.FileStore('/tmp/tail'))
74 , id = 0;
75
76 // generate random log data
77 function again() {
78 log.info('something happened', { id: ++id, user: 'Tobi' });
79 setTimeout(again, Math.random() * 100 | 0);
80 }
81
82 again();
83
84 // tail the json "documents"
85 log.stream({ end: false, interval: 500 })
86 .on('data', function(obj){
87 console.log(obj);
88 });
89 ```
90
91 yields:
92
93 ```js
94 { id: 1,
95 level: 'info',
240d053 TJ Holowaychuk Changed .msg -> .type
authored
96 type: 'something happened',
2e59ea3 TJ Holowaychuk docs
authored
97 timestamp: 1332907641734 }
98 { id: 2,
99 level: 'info',
240d053 TJ Holowaychuk Changed .msg -> .type
authored
100 type: 'something happened',
2e59ea3 TJ Holowaychuk docs
authored
101 timestamp: 1332907641771 }
102 ...
103 ```
104
3ffc760 TJ Holowaychuk docs
authored
105 ## jog(1)
106
107 ```
76fc39e TJ Holowaychuk docs: updated usage docs
authored
108
3ffc760 TJ Holowaychuk docs
authored
109 Usage: jog [options]
c838910 TJ Holowaychuk add docs for --reduce
authored
110
3ffc760 TJ Holowaychuk docs
authored
111 Options:
c838910 TJ Holowaychuk add docs for --reduce
authored
112
76fc39e TJ Holowaychuk docs: updated usage docs
authored
113 -h, --help output usage information
114 -V, --version output the version number
115 -w, --within <ms> filter events to within the given <ms>
116 -t, --type <name> filter using the given type <name>
117 -l, --level <name> filter using the given level <name>
118 -f, --ignore-eof do not stop on EOF
119 -F, --file <path> load from the given <path>
120 -R, --redis load from redis store
121 -s, --select <fn> use the given <fn> for filtering
122 -m, --map <fn> use the given <fn> for mapping
c838910 TJ Holowaychuk add docs for --reduce
authored
123 -r, --reduce <fn> use the given <fn> for reducing
a352784 TJ Holowaychuk docs
authored
124 -c, --color color the output
c838910 TJ Holowaychuk add docs for --reduce
authored
125 -j, --json output JSON (--color will not work)
126
3ffc760 TJ Holowaychuk docs
authored
127 ```
128
0b42c9c TJ Holowaychuk docs
authored
129 ### Examples
130
a4327c1 TJ Holowaychuk allow --map several times
authored
131 View all logs from tobi. The `_` object for the function
132 bodies of `--select` and `--map` represents the current
133 document, it's all just javascript.
0b42c9c TJ Holowaychuk docs
authored
134
135 ```
136 jog --file /tmp/jog --select "_.user == 'tobi'"
137 [ { user: 'tobi',
138 duration: 1000,
139 level: 'info',
240d053 TJ Holowaychuk Changed .msg -> .type
authored
140 type: 'rendering video',
0b42c9c TJ Holowaychuk docs
authored
141 timestamp: 1332861272100 },
142 { user: 'tobi',
143 duration: 2000,
144 level: 'info',
240d053 TJ Holowaychuk Changed .msg -> .type
authored
145 type: 'compiling video',
0b42c9c TJ Holowaychuk docs
authored
146 timestamp: 1332861272100 },
147 ...
148 ```
149
150 Filter video compilation durations from "tobi" only:
151
152 ```
153 $ jog --file /var/log/videos.log --select "_.user == 'tobi'" --map _.duration
154 [ 1000, 2000, 1200, 1000, 2000, 1200 ]
155 ```
156
b2a2348 TJ Holowaychuk docs
authored
157 Flags can be used several times:
a4327c1 TJ Holowaychuk allow --map several times
authored
158
159 ```
240d053 TJ Holowaychuk Changed .msg -> .type
authored
160 jog --file /var/log/videos.log --select "_.vid < 5" --map _.type --map "_.split(' ')"
a4327c1 TJ Holowaychuk allow --map several times
authored
161 [ [ 'compiling', 'video' ],
162 [ 'compiling', 'video' ],
163 [ 'compiling', 'video' ],
164 [ 'compiling', 'video' ] ]
165 ```
166
3bbb42c TJ Holowaychuk Added `-f, --ignore-eof` support to jog(1)
authored
167 Tail errors only, with color:
168
169 ```
170 $ jog --file my.log -f -c --select '_.level == "error"'
171 { level: 'error',
240d053 TJ Holowaychuk Changed .msg -> .type
authored
172 type: 'something broke',
3bbb42c TJ Holowaychuk Added `-f, --ignore-eof` support to jog(1)
authored
173 timestamp: 1333943982669 }
174 ```
175
b2a2348 TJ Holowaychuk docs
authored
176 Display error messages within the last 10 seconds:
177
178 ```
240d053 TJ Holowaychuk Changed .msg -> .type
authored
179 $ jog -F my.log --level error --select "Date.now() - _.timestamp < 10000" --map _.type
b2a2348 TJ Holowaychuk docs
authored
180 [ 'something broke', 'something broke', 'something broke' ]
181 ```
182
bd7d06c TJ Holowaychuk docs: --within
authored
183 Events within the last 10 minutes, 5 seconds, and 200ms:
184
185 ```
186 $ jog -F my.log --within 10m
187 $ jog -F my.log --within 5s
188 $ jog -F my.log --within 200
189 ```
190
28d2f41 TJ Holowaychuk docs
authored
191 ## Stores
192
193 By default Jog ships with the `FileStore` and `RedisStore`, however anything
194 with the following methods implemented will work:
195
196 - `add(obj)` to add a log object
197 - `stream() => EventEmitter` to stream data
256a69c TJ Holowaychuk docs
authored
198 - `stream({ end: false }) => EventEmitter` to stream data indefinitely
28d2f41 TJ Holowaychuk docs
authored
199 - `clear(fn)` to clear the logs
200
201 ### FileStore(path)
202
203 Store logs on disk.
204
205 ```js
206 var jog = require('jog');
207 var log = jog(new jog.FileStore('/var/log/videos.log'));
208 ```
209
210 ### RedisStore([client])
211
212 Store logs in redis.
213
214 ```js
215 var jog = require('jog');
216 var log = jog(new jog.RedisStore);
217 ```
218
9e24e76 TJ Holowaychuk perf docs
authored
219 ## Performance
220
221 No profiling or optimizations yet but the `FileStore` can
a456946 TJ Holowaychuk redis perf
authored
222 stream back 250,000 documents (~21MB) in 1.2 seconds on my
9e24e76 TJ Holowaychuk perf docs
authored
223 macbook air.
224
a456946 TJ Holowaychuk redis perf
authored
225 The `RedisStore` with 250,000 documents streamed back
226 in 2.8 seconds on my air.
227
9e55252 TJ Holowaychuk docs
authored
228 ## Running tests
229
230 ```
231 $ npm install
62d3ec3 TJ Holowaychuk redis-server for testing
authored
232 $ redis-server &
9e55252 TJ Holowaychuk docs
authored
233 $ make test
234 ```
235
1592e82 TJ Holowaychuk Initial commit
authored
236 ## License
237
238 (The MIT License)
239
240 Copyright (c) 2012 LearnBoost &lt;tj@learnboost.com&gt;
241
242 Permission is hereby granted, free of charge, to any person obtaining
243 a copy of this software and associated documentation files (the
244 'Software'), to deal in the Software without restriction, including
245 without limitation the rights to use, copy, modify, merge, publish,
246 distribute, sublicense, and/or sell copies of the Software, and to
247 permit persons to whom the Software is furnished to do so, subject to
248 the following conditions:
249
250 The above copyright notice and this permission notice shall be
251 included in all copies or substantial portions of the Software.
252
253 THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
254 EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
255 MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
256 IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
257 CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
258 TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
259 SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Something went wrong with that request. Please try again.