Fluentd sender doesn't send logs in case of end() call.

[nikhilkakade2108]

I am creating fluentd sender by using createFluentSender() function.
Below is my code.

var fluentdSender=fluentdLogger.createFluentSender(lambdaName,fluentdConfig);

var logger = new (winston.Logger)({
  transports: [new (winston.transports.Console)()]
});

exports.setRequestId=function(reqId){
  requestId=reqId;
}

logger.on('logging', (transport, level, message, meta) => {
  var data={"message":message,"requestId":"2108191222","environment":environment,"level":level};
    fluentdSender.emit('',data);
    if (meta.end) {
      fluentdSender.end();
    }
 });

It is not logging anything in backend. If I remove fluentdSender.end() it works properly.
Removing fluentdSender.end() hangs the process and in case of lambda, function timeout occurs.
Using setTimeout() cannot be the solution in my case, since we have multiple lambdas, which eventually increase cost for us.

I cannot use winston support transport because I need to manipulate message before emit.

[okkez]

fluentSender.end() will close socket immediately.
If you want to wait processing the last message, you can use requireAckResponse option and fluentSender.end("label", {"message": "some message"}).

[okkez]

@nikhilkakade2108 Please read winston section in README.md.

[nikhilkakade2108]

I added requireAckResponse: true in config, but it is not working for me. The problem here is, I cannot use support.winstonTransport() due to some limitations (I need to add lambdaname by default in every log statement). Is there any other way to end the fluentdSender with all logs flushed at back-end and without using setTimeout()?

The question I have is, what is difference between using support.winstonTransport() and createFluentSender(), because former works properly without any setTimeout()?

calling emit() manually for fluentdSender and ending the connection doesn't send a single log in backend.

[okkez]

Hmm, your code in first comment is not same as sample code in README.md.
It seems that you don't use winston transport for fluent-logger in your code above.

Which do you want to use fluent-logger's winston support or raw FluentSender?

Could you show me minimal reproducible code?
I could not imagine your situation enough.

[nikhilkakade2108]

yes I am using raw FluentSender. Please refer this code.

var winston=require('winston');
var logger = new (winston.Logger)({
  transports: [new (winston.transports.Console)()]
});

var fluentdSender = require('fluent-logger').createFluentSender('myTag', {
   host: 'localhost',
   port: 24224,
   timeout: 3.0,
   reconnectInterval: 600000
});

logger.on('logging', (transport, level, message, meta) => {
  var data={"message":message,"requestId":"21081","environment":environment,"level":level};
    fluentdSender.emit('',data);
    if (meta.end) {
      fluentdSender.end();
    }
 });

 logger.info('start logging');
 logger.info('something something');
 logger.info('end logging',{end:true});

I cannot see any logs at Fluentd end with above code.
If I add setTimeout(()=>{fluentdSender.end()},1000) ; it works.

[okkez]

fluendSender.end() has same interface with fluentSender.emit().
fluentSender.end() with no arguments will shutdown fluentSender instance immediately.

We can wait ack response from server using following snippet:

fluentSender.end("label", {"message": "Good bye!"});

[nikhilkakade2108]

writting fluentSender.end("label", {"message": "Good bye!"}); worked for me.
Thanks okkez. Closing the issue.