New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Any template ready for GCP? #54
Comments
Processing payloads was sort of cryptic to me given my lack of knowledge about its workings. I had a better visual with the if (this.includeMDC) {
event.getMDCPropertyMap().forEach((key, value) -> {
if (!FILTERED_MDC_FIELDS.contains(key)) {
map.put(key, value);
}
});
}
if (this.includeTimestamp) {
map.put(StackdriverTraceConstants.TIMESTAMP_SECONDS_ATTRIBUTE,
TimeUnit.MILLISECONDS.toSeconds(event.getTimeStamp()));
map.put(StackdriverTraceConstants.TIMESTAMP_NANOS_ATTRIBUTE,
TimeUnit.MILLISECONDS.toNanos(event.getTimeStamp() % 1_000));
} Here I'd rather go with add(StackdriverTraceConstants.SEVERITY_ATTRIBUTE, this.includeLevel,
String.valueOf(event.getLevel()), map);
add(JsonLayout.THREAD_ATTR_NAME, this.includeThreadName, event.getThreadName(), map);
add(JsonLayout.LOGGER_ATTR_NAME, this.includeLoggerName, event.getLoggerName(), map);
if (this.includeFormattedMessage) {
String message = event.getFormattedMessage();
if (this.includeExceptionInMessage) {
IThrowableProxy throwableProxy = event.getThrowableProxy();
if (throwableProxy != null) {
String stackTrace = getThrowableProxyConverter().convert(event);
if (stackTrace != null && !stackTrace.equals("")) {
message += "\n" + stackTrace;
}
}
}
map.put(JsonLayout.FORMATTED_MESSAGE_ATTR_NAME, message);
} This is interesting. When add(JsonLayout.MESSAGE_ATTR_NAME, this.includeMessage, event.getMessage(), map);
add(JsonLayout.CONTEXT_ATTR_NAME, this.includeContextName, event.getLoggerContextVO().getName(), map); We don't have an equivalent for this in addThrowableInfo(JsonLayout.EXCEPTION_ATTR_NAME, this.includeException, event, map);
addTraceId(event, map); which translates to protected String formatTraceId(final String traceId) {
// Trace IDs are either 64-bit or 128-bit, which is 16-digit hex, or 32-digit hex.
// If traceId is 64-bit (16-digit hex), then we need to prepend 0's to make a 32-digit hex.
if (traceId != null && traceId.length() == 16) {
return "0000000000000000" + traceId;
}
return traceId;
}
private void addTraceId(ILoggingEvent event, Map<String, Object> map) {
if (!this.includeTraceId) {
return;
}
String traceId =
event.getMDCPropertyMap().get(StackdriverTraceConstants.MDC_FIELD_TRACE_ID);
if (traceId == null) {
traceId = TraceIdLoggingEnhancer.getCurrentTraceId();
}
if (!StringUtils.isEmpty(traceId)
&& !StringUtils.isEmpty(this.projectId)
&& !this.projectId.endsWith("_IS_UNDEFINED")) {
traceId = StackdriverTraceConstants.composeFullTraceName(
this.projectId, formatTraceId(traceId));
}
add(StackdriverTraceConstants.TRACE_ID_ATTRIBUTE, this.includeTraceId, traceId, map);
} That's quite some business logic going here. An innocent attempt can be add(StackdriverTraceConstants.SPAN_ID_ATTRIBUTE, this.includeSpanId,
event.getMDCPropertyMap().get(StackdriverTraceConstants.MDC_FIELD_SPAN_ID), map);
if (this.serviceContext != null) {
map.put(StackdriverTraceConstants.SERVICE_CONTEXT_ATTRIBUTE, this.serviceContext);
} I don't have a solution here except dragging if (this.customJson != null && !this.customJson.isEmpty()) {
for (Map.Entry<String, Object> entry : this.customJson.entrySet()) {
map.putIfAbsent(entry.getKey(), entry.getValue());
}
} This can be achieved by either modifying the template or providing In the light of these findings, at a first glance, the following template might do the trick: {
"mdc1": "${json:mdc:key1}",
"mdc2": "${json:mdc:key2}",
"mdc3": "${json:mdc:key3}",
"timestamp": "${json:timestamp}",
"severity": "${json:level}",
"thread": "${json:thread:name}",
"logger": "${json:logger:name}",
"message": "${json:message}",
"exception": "${json:exception:stackTrace:text}",
"logging.googleapis.com/trace": "${json:mdc:X-B3-TraceId}",
"serviceContext": "${json:mdc:serviceContext}"
} @ilgrosso, would you mind giving this a try, please? Any other feedback and/or comments are also welcome. |
The above looks great, thanks! |
Thanks for you help @vy , worked like a charm. |
@ilgrosso, before closing the issue, it might be good that we add this as a FAQ to README. Would you mind sharing how did you exactly solve your problem, please? The template JSON, Java code, prepared MDC fields, etc. The more details you provide, the more helpful it will be. |
I have no MDC fields (yet?), no Java code to add. Just added the Maven dependency, grabbed your JSON template as above and defined appenders like as follows: <Console name="mainFile" target="SYSTEM_OUT">
<LogstashLayout dateTimeFormatPattern="yyyy-MM-dd'T'HH:mm:ss.SSSZZZ"
timeZoneId="UTC"
eventTemplateUri="classpath:logstashGCP.json"
prettyPrintEnabled="false"
stackTraceEnabled="true"/>
</Console> |
How did you deal with |
Ah sorry for not reporting: at the moment I don't need neither |
How can I get custom field say version and it should read the value from other properties file or json file . We don't want to hard code it and wanted to dynamically populate and don't want to put it in mdc. |
First, I would strongly advise you to switch from You can define the values either as JVM properties or environment variables. Then you can access these using lookups. This is explained (and demonstrated) in detail in the documentation of both projects. |
I was wondering if there is any template available for usage with GCP, according to https://cloud.google.com/logging/docs/agent/configuration#process-payload
Thanks!
The text was updated successfully, but these errors were encountered: