Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clustered appender for log4js. #149

Merged
merged 1 commit into from
Aug 9, 2013
Merged

Conversation

mkielar
Copy link

@mkielar mkielar commented Aug 8, 2013

Hi,

I've created a multiprocess appender that uses process.send / worker.on(message, callback) mechanisms instead of sockets in order to synchronize logging.

It also allows use of multiple actual loggers in the master logger, and does not require user/programmer do point explicitely which mode to use.

See an example below:

if (cluster.isMaster) {

    // Init master logger
    log4js.configure({
        appenders: [
            { 
                // Adding only one, "clustered" appender. 
                // Note, there is no "mode" setting - the mode (master / worker) is automatucally determined from cluster.isMaster / cluster.isWorker
                type: "clustered",

                // Add as many "normal" appenders as you like. They wil all be used once the "clustered" master appender receives a loggingEvent
                appenders: [
                    { type: 'console' },
                    { type: 'file', filename: "log.log", maxLogSize: 10000000, backups: 10 },
                ]
            }
        ]
    });

    // Init logger like you used to
    var logger = log4js.getLogger("master");

    // Fork workers.
    for (var i = 0; i < 4; i++) {
        var worker = cluster.fork();
        logger.info("forked: " + i);
    }

} else {

    // Init worker loggers
    log4js.configure({
        appenders: [
            { 
                // Note, there again is no "mode" setting as the worker mode is automatically determined from cluster.isWorker
                type: "clustered",

                // Note, we do not add any other appenders here. Every message is sent to master logger.
            }
        ]
    });


    // Init logger like you used to
    var logger = log4js.getLogger("worker_" + cluster.worker.id);

    logger.info('message: ' + i + ' from worker ' + cluster.worker.id);

}

+ lib/appenders/clustered.js
+ test/clusteredAppender-test.js

Instead os using sockets (like multiprocess) or dead and unmaintained hook.io, Clustered appender
uses process.send(message) / worker.on('message', callback) mechanisms for transporting data
between worker processes and master logger.

Master logger takes an "appenders" array of actual appenders that are triggered when worker appenders send some data.
This guarantees sequential writes to appenders, so the log messages are not mixed in single lines of log.
nomiddlename pushed a commit that referenced this pull request Aug 9, 2013
Clustered appender for log4js.
@nomiddlename nomiddlename merged commit 49892f3 into log4js-node:master Aug 9, 2013
@nomiddlename
Copy link
Collaborator

Thanks for this, nice work. The multiprocess appender was written in the days before you could pass messages to workers (and indeed, before cluster became part of node core). This will be a good replacement for it, when you don't need to log to different hosts.

@SameerSiddiqui
Copy link

can you extend it to where i can have different loggers write to different files from Master comming from child.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants