rotating-file-stream

Usage

var rfs    = require('rotating-file-stream');
var stream = rfs('file.log', {
    size:     '10M', // rotate every 10 MegaBytes written
    interval: '1d',  // rotate daily
    compress: 'gzip' // compress rotated files
});

under development

This package is currently under development.

Please check the TODO list to be aware of what is missing.

Installation

With npm:

npm install rotating-file-stream

API

rfs(filename, options)

Returns a new stream.Writable to filename as fs.createWriteStream does. The file is rotated following options rules.

filename {String|Function}

The most complex problem about file name is: "how to call the rotated file name?"

The answer to this question may vary in many forms depending on application requirements and/or specifications. If there are no requirements, a String can be used and default rotated file name generator will be used; otherwise a Function which returns the rotated file name can be used.

function filename(time, index)

• time: {Date} If rotation by interval is enabled, the start time of rotation period, otherwise the time when rotation job started. If null, the not rotated file name must be returned.
• index {Number} The progressive index of rotation by size in the same rotation period.

An example of a complex rotated file name generator function could be:

function pad(num) {
    return (num + "").length == 1 ? "0" + num : num;
}

function generator(time, index) {
    if(! time)
        return "file.log";

    var month  = time.getFullYear() + "" + pad(time.getMonth() + 1);
    var day    = pad(time.getDate());
    var hour   = pad(time.getHours());
    var minute = pad(time.getMinutes());

    return "/storage/" + month + "/" +
        month + day + "-" + hour + minute + "-" + index + "-" + filename;
}

var rfs    = require('rotating-file-stream');
var stream = rfs(generator, {
    size:     '10M',
    interval: '1d'
});

Note: If part of returned destination path does not exists, the rotation job will try to create it.

options {Object}

• compress: {String} (default: null) Specifies compression method of rotated files.
• interval: {String} (default: null) Specifies the time interval to rotate the file.
• size: {String} (default: null) Specifies the file size to rotate the file.
• highWaterMark: {Number} (default: 16K) Proxied to new stream.Writable
• mode: {Integer} (default: 0o666) Proxied to fs.open

size

Accepts a positive integer followed by one of these possible letters:

• B: Bites
• K: KiloBites
• M: MegaBytes
• G: GigaBytes

  size: '300K', // rotates the file when its size exceeds 300 KiloBytes

  size: '100M', // rotates the file when its size exceeds 100 MegaBytes

  size: '1G', // rotates the file when its size exceeds a GigaBytes

interval

Accepts a positive integer followed by one of these possible letters:

• s: seconds. Accepts integer divider of 60.
• m: minutes. Accepts integer divider of 60.
• h: hours. Accepts integer divider of 24.
• d: days

  interval: '5m', // rotates the file at minutes 0, 5, 10, 15 and so on

  interval: '2h', // rotates the file at midnight, 02:00, 04:00 and so on

  interval: '1d', // rotates the file at every midnight

compress

Due the nature of Node.js compression may be done with an external command (to use other CPUs than the one used by Node.js to not subtract CPU power to our application) or with internal code (to use the CPU used by Node.js to not subtract more CPU power than expected to the system). This decision is left to you.

Following fixed strings are allowed to compress the files with internal libraries:

• bzip
• gzip

To enable external compression, a function can be used or simple the boolean true value to use default external compression. The function should accept source and dest file names and must return the shell command to be executed to archive the file. The two following code snippets have exactly the same effect:

var rfs    = require('rotating-file-stream');
var stream = rfs('file.log', {
    size:     '10M',
    compress: true
});

var rfs    = require('rotating-file-stream');
var stream = rfs('file.log', {
    size:     '10M',
    compress: function(source, dest) {
        return "cat " + source + " | gzip -t9 > " + dest;
    }
});

Note: The shell command to archive the rotated file should not remove the source file, it will be removed by the package if archive job complete with success.

Events

Custom Events are emitted by the stream.

var rfs    = require('rotating-file-stream');
var stream = rfs(...);

stream.on('error', function(err) {
    // here are reported errors occurred while rotating as well write errors
});

stream.on('rotation', function() {
    // rotation job started
});

stream.on('rotated', function(filename) {
    // rotation job completed with success and produced given filename
});

Under the hood

Logs should be handled so carefully, so this package tries to never overwrite files.

At stream creation, if the not rotated log file already exists and its size exceeds the rotation size, an initial rotation attempt is done.

At each rotation attempt a check is done to verify that destination rotated file does not exists yet; if this is not the case a new destination rotated file name is generated and the same check is performed before going on. This is repeated until a not existing destination file name is found or the package is exhausted. For this reason the rotated file name generator function may be called several times for each rotation job.

Unexpected

If I understood correctly, there are some case which should never happen.
Anyway  I want to be sure,  so I  decided to throw an  Error if code runs
through one of these cases.
If it happen that you catch one of these, please make me aware of that as
soon as possible in order to handle the case.

                                                            The author

Compatibility

This package is written following Node.js 4.0 specifications always taking care about backward compatibility. The package it tested under following versions:

• 4.0
• 0.12
• 0.11
• 0.10

Licence

MIT Licence

Bugs

Do not hesitate to report any bug or inconsistency @github.

TODO

• Rotate by interval
• Create missing directories in paths
• External compression
• Internal compression gzip
• Internal compression bzip
• Test all error case handling

Changelog

• 2015-09-17 - v0.0.2
  • Rotation by size
• 2015-09-14 - v0.0.1
  • README.md
• 2015-09-10 - v0.0.0
  • Embryonal stage