overlog (node.js module)
An easy to use solution to store and fetch the logs.
Library and Command-Line Interface
push(entry)adds an entry.
fetch(time_interval)returns all entries from certain time interval.
fetch_and_follow()fetches entries from certain time range and goes into following mode afterwards.
Both storer and fetcher can be used via command line or via node.js bindings.
Shell tests provide a good idea on basic usage and functionality.
Directory to store the files has to be provided. Technically, two directories are required: one for intermediate files, append-only, and one for thefinalized files. They will be created as
$DIR/destination/ as necessary.
In PubSub mode, an HTTP status endpoint is being exposed as well. All it takes to use PubSub is to specify PubSub channel name, default values for other parameters would work.
If not using PubSub for storer's locking mechanism, path to a lock file (via
pidlock, directory and file) is required as well.
Log entries are JSON objects. They should have the
"ms" field set to
Date.now() at the moment log entry was created.
- Append-only intermediate files.
- Atomically rename them into final format.
- If necessary, replay older files at startup.
- Keep first and last timestamps in the filenames for easier access.
In PubSub mode, an HTTP status page is being exposed, check out
The tool was designed for the
fetch_and_follow() usecase, that is easiest to access with fetcher's command line flags
node.js code performing these actions can be used directly as well.
Bells and Whistles
- With HTTP status endpoint, PubSub and a command-line tool.
- Process-level locked.
- Unit-tested through.