Lex Parser

You can use Lex syntax to define a log parser.

Let's assume we have a log file with the following content:

2013-03-06 10:17:01,375 [main] DEBUG acme.mr.TestJob - run(): setting basic properties...
System.Net.WebException: The operation has timed-out.
   at System.Net.HttpWebRequest.GetResponse()
   at System.Windows.Forms.Control.OnClick(EventArgs e)
   at System.Windows.Forms.Button.OnClick(EventArgs e)
   at System.Windows.Forms.Button.OnMouseUp(MouseEventArgs mevent)
   at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks)
   at System.Windows.Forms.Control.WndProc(Message& m)
   at System.Windows.Forms.ButtonBase.WndProc(Message& m)
   at System.Windows.Forms.Button.WndProc(Message& m)
   at System.Windows.Forms.ControlNativeWindow.OnMessage(Message& m)
   at System.Windows.Forms.ControlNativeWindow.WndProc(Message& m)
   at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)
2013-03-06 10:17:01,380 [main] DEBUG acme.mr.TestJob - run(): customize mapper and reducer...
2013-03-06 10:17:01,380 [main] DEBUG acme.mr.TestJob - run(): +++ customizing mr-job mapper...
2013-03-06 10:17:01,382 [main] DEBUG acme.mr.TestJob - run(): +-- customizing mr-job mapper done
2013-03-06 10:17:01,382 [main] DEBUG acme.mr.TestJob - run(): +++ customizing mr-job reducer...
2013-03-06 10:17:01,383 [main] DEBUG acme.mr.TestJob - run(): +-- customizing mr-job reducer done

To define the parser for the log file we first need to create a new preset (or edit an existing one).

Lex parser consists of segments parser and record parser.

Segment parser determines begin offset and end offset of a log entry. Record parser transforms raw log entry into LogWatch entry.

We could share Lex code between these two parsers using "Common" field.

In our example we don't need any shared code, because we treat a text line as a segment.

Let's define segment parser:

timestamp [0-9]{4}\-[0-9]{2}\-[0-9]{2}[ ][0-9]{2}:[0-9]{2}:[0-9]{2}[,][0-9]{3}
end \r\n{timestamp}
%x IN_SEGMENT
%%
<INITIAL>{timestamp} BeginSegment(); BEGIN(IN_SEGMENT);
<IN_SEGMENT>{end} EndSegment(); BeginSegment();
<INITIAL,IN_SEGMENT><<EOF>> EndSegment();

And a code for the record parser:

timestamp [0-9]{4}\-[0-9]{2}\-[0-9]{2}[ ][0-9]{2}:[0-9]{2}:[0-9]{2}[,][0-9]{3}
thread [^\]]+
level INFO|DEBUG|TRACE|WARN|ERROR|FATAL
logger [a-zA-Z0-9.]+[ ]-[ ][a-zA-Z0-9]+[(][^)]*[)]
message .+
exception (.+\r\n)+

%x MATCHED_TIMESTAMP
%x MATCHED_LEVEL
%x MATCHED_LOGGER
%x MATCHED_MESSAGE
%%

<INITIAL>{timestamp} Timestamp = TextAsTimestamp("yyyy-MM-dd HH:mm:ss,fff"); BEGIN(MATCHED_TIMESTAMP);
<MATCHED_TIMESTAMP>{level} Level = yytext; BEGIN(MATCHED_LEVEL);
<MATCHED_LEVEL>{logger} Logger = yytext; BEGIN(MATCHED_LOGGER);
<MATCHED_LOGGER>{message} Message = yytext.TrimStart(':', ' '); BEGIN(MATCHED_MESSAGE);
<MATCHED_MESSAGE>{exception} Exception = yytext;

Method TextAsTimestamp parses the current matched text into DateTime using the provided format.

Then click "compile & run" button and check the output.

If there is no error you can save the preset and use it.