Allow customizing pattern of "message" field for AccessEvents

[nickavv]

The message field is useful for a number of reasons in applications that consume and display logs. However, the properties that appear in the message field are not currently customizable. As per the documentation, the pattern is fixed as: ${remoteHost} - ${remoteUser} [${timestamp}] "${requestUrl}" ${statusCode} ${contentLength}

In my particular case, the remoteUser property is not allowed to be logged, for security reasons. I've ignored the separate remote_user field, but I cannot remove that property from the message field without ignoring the message field entirely.

It would be nice if the pattern that the message field logged was configurable.

[philsttr]

Hi @nickavv

I definitely agree that logstash-logback-encoder could make customizing the message logged for access events easier. Great idea.

I just want to let you know that it is possible to do this today by using the AccessEventCompositeJsonEncoder instead of the LogstashAccessEncoder.

See https://github.com/logstash/logstash-logback-encoder#composite-encoderlayout for details on the AccessEventCompositeJsonEncoder, and https://github.com/logstash/logstash-logback-encoder#providers-for-accessevents for all of the providers you can add to it.

In your case, you would not add the standard message provider. But would instead either add a custom JsonProvider with an implementation similar to AccessMessageJsonProvider, or use the pattern provider to define a custom pattern for the message field using the conversion words defined by logback for access events

[nickavv]

Thanks for the comment @philsttr, I just wanted to follow up for anybody having a similar issue that I was able to get things working the way I wanted with your workaround. This is what my encoder looks like in my logback-access.xml file now:

<encoder class="net.logstash.logback.encoder.AccessEventCompositeJsonEncoder">
   	<providers>
    		<timestamp />
    		<pattern>
    			<fieldName>message</fieldName>
    			<pattern>
    			{ 
    				"message": "%i{X-Forwarded-For} - [%t] %requestURL %statusCode %bytesSent"		
    			}
    			</pattern>
    		</pattern>
    		<method />
    		<statusCode />
    		<stackTrace />
    		<requestedUri />
    		<remoteHost />
    		<contentLength />
    		<elapsedTime />
    	</providers>
    </encoder>

[kobozo]

I'm also searching for a good way to customize the message field, but only the message field and keep all the other fields output as it is without the need of repeating them in the pattern. This because we can dynamically set mdc fields form the program and I want the logback to be untouched even when adding a mdc field.

Something like

<encoder class="net.logstash.logback.encoder.LogstashEncoder">
    <customFields>{"client":"${CLIENT_NAME}","json": "#asJson{%message}"}</customFields>
    <lowerCaseHeaderNames>true</lowerCaseHeaderNames>
</encoder>

[nickavv]

Yes even though I was able to use the workaround I still think my original ask in the ticket was valuable, being able to customize message while still using the LogstashEncoder