Skip to content

Latest commit

 

History

History
396 lines (292 loc) · 17.6 KB

transports.md

File metadata and controls

396 lines (292 loc) · 17.6 KB

Wide Transports

In wide a transport is essentially a storage device for your logs. Each instance of a wide logger can have multiple transports configured at different levels. For example, one may want error logs to be stored in a persistent remote location (like a database), but all logs output to the console or a local file.

There are several core transports included in wide, which leverage the built-in networking and file I/O offered by node.js core. In addition, there are third-party transports which are supported by the wide core team. And last (but not least) there are additional transports written by members of the community.

Wide Core

There are several core transports included in wide, which leverage the built-in networking and file I/O offered by node.js core.

Console Transport

  wide.add(wide.transports.Console, options)

The Console transport takes four simple options:

  • level: Level of messages that this transport should log (default 'debug').
  • silent: Boolean flag indicating whether to suppress output (default false).
  • colorize: Boolean flag indicating if we should colorize output (default false).
  • timestamp: Boolean flag indicating if we should prepend output with timestamps (default false). If function is specified, its return value will be used instead of timestamps.

Metadata: Logged via util.inspect(data);

File Transport

  wide.add(wide.transports.File, options)

The File transport should really be the 'Stream' transport since it will accept any WritableStream. It is named such because it will also accept filenames via the 'filename' option:

  • level: Level of messages that this transport should log.
  • silent: Boolean flag indicating whether to suppress output.
  • colorize: Boolean flag indicating if we should colorize output.
  • timestamp: Boolean flag indicating if we should prepend output with timestamps (default false). If function is specified, its return value will be used instead of timestamps.
  • filename: The filename of the logfile to write output to.
  • maxsize: Max size in bytes of the logfile, if the size is exceeded then a new file is created.
  • maxFiles: Limit the number of files created when the size of the logfile is exceeded.
  • stream: The WriteableStream to write output to.
  • json: If true, messages will be logged as JSON (default true).

Metadata: Logged via util.inspect(data);

DailyRotateFile Transport

  wide.add(wide.transports.DailyRotateFile, options)

The DailyRotateFile transport can rotate files by minute, hour, day, month or year. Its options are identical to the File transport with the lone addition of the 'datePattern' option:

  • datePattern: A string representing the pattern to be used when appending the date to the filename (default '.yyyy-MM-dd'). The data characters used in this string will dictate the frequency of the file rotation. For example if your datePattern is simply '.HH' you will end up with 24 log files that are picked up and appended to every day.

Valid data characters in the datePattern are:

  • yy: Last two digits of the year.
  • yyyy: Full year.
  • M: The month.
  • MM: The zero padded month.
  • d: The day.
  • dd: The zero padded day.
  • H: The hour.
  • HH: The zero padded hour.
  • m: The minute.
  • mm: The zero padded minute.

Metadata: Logged via util.inspect(data);

Http Transport

  wide.add(wide.transports.Http, options)

The Http transport is a generic way to log, query, and stream logs from an arbitrary Http endpoint, preferably winstond. It takes options that are passed to the node.js http or https request:

  • host: (Default: localhost) Remote host of the HTTP logging endpoint
  • port: (Default: 80 or 443) Remote port of the HTTP logging endpoint
  • path: (Default: /) Remote URI of the HTTP logging endpoint
  • auth: (Default: None) An object representing the username and password for HTTP Basic Auth
  • ssl: (Default: false) Value indicating if we should us HTTPS

Wide More

Starting with wide@0.3.0 an effort was made to remove any transport which added additional dependencies to wide. At the time there were several transports already in wide which will always be supported by the wide core team.

CouchDB Transport

As of wide@0.6.0 the CouchDB transport has been broken out into a new module: wide-couchdb.

  wide.add(wide.transports.Couchdb, options)

The Couchdb will place your logs in a remote CouchDB database. It will also create a Design Document, _design/Logs for later querying and streaming your logs from CouchDB. The transport takes the following options:

  • host: (Default: localhost) Remote host of the HTTP logging endpoint
  • port: (Default: 5984) Remote port of the HTTP logging endpoint
  • db: (Default: wide) Remote URI of the HTTP logging endpoint
  • auth: (Default: None) An object representing the username and password for HTTP Basic Auth
  • ssl: (Default: false) Value indicating if we should us HTTPS

Redis Transport

  wide.add(wide.transports.Redis, options)

This transport accepts the options accepted by the node-redis client:

  • host: (Default localhost) Remote host of the Redis server
  • port: (Default 6379) Port the Redis server is running on.
  • auth: (Default None) Password set on the Redis server

In addition to these, the Redis transport also accepts the following options.

  • length: (Default 200) Number of log messages to store.
  • container: (Default wide) Name of the Redis container you wish your logs to be in.
  • channel: (Default None) Name of the Redis channel to stream logs from.

Metadata: Logged as JSON literal in Redis

Loggly Transport

As of wide@0.6.0 the Loggly transport has been broken out into a new module: wide-loggly.

  wide.add(wide.transports.Loggly, options);

The Loggly transport is based on Nodejitsu's node-loggly implementation of the Loggly API. If you haven't heard of Loggly before, you should probably read their value proposition. The Loggly transport takes the following options. Either 'inputToken' or 'inputName' is required:

  • level: Level of messages that this transport should log.
  • subdomain: The subdomain of your Loggly account. [required]
  • auth: The authentication information for your Loggly account. [required with inputName]
  • inputName: The name of the input this instance should log to.
  • inputToken: The input token of the input this instance should log to.
  • json: If true, messages will be sent to Loggly as JSON.

Metadata: Logged in suggested Loggly format

Riak Transport

As of wide@0.3.0 the Riak transport has been broken out into a new module: wide-riak. Using it is just as easy:

  var Riak = require('wide-riak').Riak;
  wide.add(Riak, options);

In addition to the options accepted by the riak-js client, the Riak transport also accepts the following options. It is worth noting that the riak-js debug option is set to false by default:

  • level: Level of messages that this transport should log.
  • bucket: The name of the Riak bucket you wish your logs to be in or a function to generate bucket names dynamically.
  // Use a single bucket for all your logs
  var singleBucketTransport = new (Riak)({ bucket: 'some-logs-go-here' });

  // Generate a dynamic bucket based on the date and level
  var dynamicBucketTransport = new (Riak)({
    bucket: function (level, msg, data, now) {
      var d = new Date(now);
      return level + [d.getDate(), d.getMonth(), d.getFullYear()].join('-');
    }
  });

Metadata: Logged as JSON literal in Riak

MongoDB Transport

As of wide@0.3.0 the MongoDB transport has been broken out into a new module: wide-mongodb. Using it is just as easy:

  var MongoDB = require('wide-mongodb').MongoDB;
  wide.add(MongoDB, options);

The MongoDB transport takes the following options. 'db' is required:

  • level: Level of messages that this transport should log.
  • silent: Boolean flag indicating whether to suppress output.
  • db: The name of the database you want to log to. [required]
  • collection: The name of the collection you want to store log messages in, defaults to 'log'.
  • safe: Boolean indicating if you want eventual consistency on your log messages, if set to true it requires an extra round trip to the server to ensure the write was committed, defaults to true.
  • host: The host running MongoDB, defaults to localhost.
  • port: The port on the host that MongoDB is running on, defaults to MongoDB's default port.

Metadata: Logged as a native JSON object.

Additional Transports

The community has truly embraced wide; there are over 23 wide transports and over half of them are maintained by authors external to the wide core team. If you want to check them all out, just search npm:

  $ npm search wide

If you have an issue using one of these modules you should contact the module author directly

SimpleDB Transport

The wide-simpledb transport is just as easy:

  var SimpleDB = require('wide-simpledb').SimpleDB;
  wide.add(SimpleDB, options);

The SimpleDB transport takes the following options. All items marked with an asterisk are required:

  • awsAccessKey:* your AWS Access Key
  • secretAccessKey:* your AWS Secret Access Key
  • awsAccountId:* your AWS Account Id
  • domainName:* a string or function that returns the domain name to log to
  • region:* the region your domain resides in
  • itemName: a string ('uuid', 'epoch', 'timestamp') or function that returns the item name to log

Metadata: Logged as a native JSON object to the 'data' attribute of the item.

Mail Transport

The wide-mail is an email transport:

  var Mail = require('wide-mail').Mail;
  wide.add(Mail, options);

The Mail transport uses node-mail behind the scenes. Options are the following, to and host are required:

  • to: The address(es) you want to send to. [required]
  • from: The address you want to send from. (default: wide@[server-host-name])
  • host: SMTP server hostname
  • port: SMTP port (default: 587 or 25)
  • secure: Use secure
  • username User for server auth
  • password Password for server auth
  • level: Level of messages that this transport should log.
  • silent: Boolean flag indicating whether to suppress output.

Metadata: Stringified as JSON in email.

Amazon SNS (Simple Notification System) Transport

The wide-sns transport uses amazon SNS to send emails, texts, or a bunch of other notifications.

  require('wide-sns').SNS;
  wide.add(wide.transports.SNS, options);

Options:

  • aws_key: Your Amazon Web Services Key. [required]
  • aws_secret: Your Amazon Web Services Secret. [required]
  • subscriber: Subscriber number - found in your SNS AWS Console, after clicking on a topic. Same as AWS Account ID. [required]
  • topic_arn: Also found in SNS AWS Console - listed under a topic as Topic ARN. [required]
  • region: AWS Region to use. Can be one of: us-east-1,us-west-1,eu-west-1,ap-southeast-1,ap-northeast-1,us-gov-west-1,sa-east-1. (default: us-east-1)
  • subject: Subject for notifications. (default: "Wide Error Report")
  • message: Message of notifications. Uses placeholders for level (%l), error message (%e), and metadata (%m). (default: "Level '%l' Error:\n%e\n\nMetadata:\n%m")
  • level: lowest level this transport will log. (default: info)

Graylog2 Transport

wide-graylog2 is a Graylog2 transport:

  var Graylog2 = require('wide-graylog2').Graylog2;
  wide.add(Graylog2, options);

The Graylog2 transport connects to a Graylog2 server over UDP using the following options:

  • level: Level of messages this transport should log. (default: info)

  • silent: Boolean flag indicating whether to suppress output. (default: false)

  • graylogHost: IP address or hostname of the graylog2 server. (default: localhost)

  • graylogPort: Port to send messages to on the graylog2 server. (default: 12201)

  • graylogHostname: The hostname associated with graylog2 messages. (default: require('os').hostname())

  • graylogFacility: The graylog2 facility to send log messages.. (default: nodejs)

Metadata: Stringified as JSON in the full message GELF field.

Cassandra Transport

wide-cassandra is a Cassandra transport:

  var Cassandra = require('wide-cassandra').Cassandra;
  wide.add(Cassandra, options);

The Cassandra transport connects to a cluster using the native protocol with the following options:

  • level: Level of messages that this transport should log (default: 'info').
  • table: The name of the Cassandra column family you want to store log messages in (default: 'logs').
  • partitionBy: How you want the logs to be partitioned. Possible values 'hour' and 'day'(Default).
  • consistency: The consistency of the insert query (default: quorum).

In addition to the options accepted by the Node.js Cassandra driver Client.

  • hosts: Cluster nodes that will handle the write requests: Array of strings containing the hosts, for example ['host1', 'host2'] (required).
  • keyspace: The name of the keyspace that will contain the logs table (required). The keyspace should be already created in the cluster.

Find more Transports

  $ npm search wide
  (...)
  wide-amon         Wide transport for Amon logging                            =zoramite
  wide-amqp         An AMQP transport for wide                                 =kr1sp1n
  wide-cassandra    A Cassandra transport for wide                             =jorgebay
  wide-couchdb      a couchdb transport for wide                               =alz
  wide-express      Express middleware to let you use wide from the browser.   =regality
  wide-graylog2     A graylog2 transport for wide                              =smithclay
  wide-hbase        A HBase transport for wide                                 =ddude
  wide-loggly       A Loggly transport for wide                                =taoyuan
  wide-mail         A mail transport for wide                                  =wavded
  wide-mail2        A mail transport for wide                                  =ivolo
  wide-mongodb      A MongoDB transport for wide                               =taoyuan
  wide-nodemail     A mail transport for wide                                  =reinpk
  wide-nssocket     nssocket transport for wide                                =mmalecki
  wide-papertrail   A Papertrail transport for wide                            =kenperkins
  wide-redis        A fixed-length Redis transport for wide                    =taoyuan
  wide-riak         A Riak transport for wide                                  =taoyuan
  wide-scribe       A scribe transport for wide                                =wnoronha
  wide-simpledb     A Wide transport for Amazon SimpleDB                       =chilts
  wide-skywriter    A Windows Azure table storage transport for wide           =pofallon
  wide-sns          A Simple Notification System Transport for wide            =jesseditson
  wide-syslog       A syslog transport for wide                                =taoyuan
  wide-syslog-ain2  An ain2 based syslog transport for wide                    =lamtha
  wide-winlog       Windows Event Log logger for Wide                          =jfromaniello
  wide-zmq          A 0MQ transport for wide                                   =dhendo
  wide-growl        A growl transport for wide                                 =pgherveou