Not showing resource.labels.function_name nor labels."execution_id"

[opyate]

Environment details

• OS: Firebase
• Node.js version: v6.14.0
• npm version:
• @google-cloud/logging-winston version: 0.9.0

Steps to reproduce

1. Set up logging-winston as per the Google docs:

const winston = require('winston');
const Logger = winston.Logger;
const Console = winston.transports.Console;

const LoggingWinston = require('@google-cloud/logging-winston').LoggingWinston;

const loggingWinston = new LoggingWinston();

const logger = new Logger({
    level: 'info',
    transports: [
        new Console(),
        // for Stackdriver
        loggingWinston,
    ],
});

module.exports = logger

2. Log a payload

logger.info('Audit', {foo: 'bar'})

3. StackDriver console shows an Audit entry, but without resource.labels.function_name and labels."execution_id"

I'd like to see my Winston log entry have these attributes as if I used regular console.log:

[opyate]

Actually, it's just occurred to me that console.log on Firebase might be patched to add these values, which means I have to do it myself, e.g.:

const logMetadata = {
  resource: {
    type: 'cloud_function',
    labels: {
      function_name: process.env.FUNCTION_NAME ,
      project: process.env.GCLOUD_PROJECT,
      region: process.env.FUNCTION_REGION
    },
  },
};

...and I'll have to get the execution_id from somewhere too.

[DominicKramer]

Thanks for opening this issue. I agree with your assessment of the issue that you would need to supply the labels in question.

I have re-opened this issue as a feature request since it would be good to determine if these labels should be applied automatically if/when available.

However, it is good that, even if the labels are not applied automatically, they can still be applied manually to get the desired functionality.

[opyate]

Thanks @DominicKramer , it'll be especially nice to see execution_id on there.

[arturowczarek]

However, it is good that, even if the labels are not applied automatically, they can still be applied manually to get the desired functionality.

@DominicKramer : Is there any workaround to manually add execution_id?

[DominicKramer]

Talking with the GCF team and after doing some experimentation, I have found that the following can be used to obtain the metadata requested (this code uses the preferred environment variables first):

const logMetadata = {
    resource: {
      type: 'cloud_function',
      labels: {
        function_name: process.env.K_SERVICE || process.env.FUNCTION_NAME,
        project: process.env.GOOGLE_CLOUD_PROJECT || process.env.GCLOUD_PROJECT,
        region: process.env.GOOGLE_CLOUD_REGION || process.env.FUNCTION_REGION,
        executionId: req.header('Function-Execution-Id')
      },
    },
  };

That is, with experimentation, I found that the request header has the execution ID. However, I didn't see the use of the header to get the execution ID documented, and so it is not official supported.

This is a viable solution, if you have access to the request object at the point where you are creating logs.

[DominicKramer]

PR #276 is open that addresses this issue. However, the PR does not contain code to auto-populate the execution id since the logger does not have access to the current request object. Further, this would be hard to get because a log statement need not be invoked in an execution path that involves a request object.

[ofrobots]

Further, this would be hard to get because a log statement need not be invoked in an execution path that involves a request object.

We do have this capability through the request.log function added by the express middleware.

[DominicKramer]

@ofrobots That's a great point. For express users we can automatically include the execution_id. 😃

[DominicKramer]

@opyate I have an update in @google-cloud/logging in PR #436 address an issue where the wrong environment variable could be used to get the function name or region.

It looks, however, that for Node 6 and Node 8 uses of Google Cloud Functions, the function name metadata should be available in the logs via PR 400.

Try the newest version of @google-cloud/logging-winston. It should now show the metadata. Let me know if it doesn't.

Also, in the GCP logs the execution_id is under labels and not resources.labels. Does having execution_id under labels work for you, or does it need to be under resources.labels?

[opyate]

@DominicKramer thanks, I'll wait for googleapis/nodejs-logging#400 to get released.
Just a note: I'm using cloud functions triggered from Firestore, not HTTP, so the execution_id still evades me. I can probably use the eventId instead.

[DominicKramer]

@opyate thanks for your patience with us getting this out. I've looked at the cloud functions docs for triggering from Firestore, and I don't see a way to get execution_id either. Hopefully the eventId can work for you.

[DominicKramer]

We were looking at this more, and thought we could use the express middleware (PR #278) to add the execution_id, but it looks like that is not possible.

Fortunately we thought of an easier approach for you. The console.log function on GCF is special in that anything printed to the console has metadata attached to it in the logs. Thus, you could just use console.log for logging and get the execution_id available in the Logs Viewer.

[DominicKramer]

I'm going to close this issue since @google-cloud/logging PR #436 and the use of console.log should resolve the issues. If you are still having issues, leave a comment and we can re-open this.

[opyate]

Thus, you could just use console.log for logging and get the execution_id available in the Logs Viewer.

Thanks, but the reason I want to use this package was to get structured logging.
Thanks for looking into it.

[techieshark]

Still struggling to get the execution_id.

I'm going to close this issue since … the use of console.log should resolve the issues.

https://cloud.google.com/functions/docs/monitoring/logging#writing_logs

Cloud Functions includes simple logging by default. Logs written to stdout or stderr will appear automatically in the Cloud Console. For more advanced logging, use the Stackdriver Logging Client Library.

Given that the official documentation actually suggests using something other than console.log, can this issue get re-opened?

[ljcremer]

I would also like to use structured logging and not getting the execution_id with Winston.

Would be great to have this re-opened.

[mrguiman]

@opyate were you ever able to find a solution to this ? I've resorted to using winston to be able to set severity, but have now lost the labels that were pushed by CF by default

[njho]

+1