Identify columns and levels

[retroflex]

Just found your app. Very nice work!!

I tried it with my log file and the time is parsed perfectly but the rest is not. Example of a log line:

[2021-04-09T10:14:12.012Z] [On-prem] [schema_2] [PID=24700] [w3wp] [TID=32888] [service] [critical] [ChildProcess.cpp:312] Process could not be started

My first problem is the levels. I'm using C++ library "spdlog" which is popular. It has these levels:
"trace", "debug", "info", "warning", "error", "critical"
Alternatively these short names:
"T", "D", "I", "W", "E", "C"

Is there any way to map these to the levels in Tailviewer?

My second problem is regarding columns. I think Tailviewer should columnize my file, right? But maybe it needs another format than mine with brackets?

[Kittyfisto]

Hey @retroflex, sorry for the long delay, I was taking some time off :)

Is there any way to map these to the levels in Tailviewer?

Only programmatically as of right now. Tailviewer natively supports log levels from log4net / log4cpp. Other log levels will have to be extended via plugins. I did one such extension last year where someone had a similar request for another popular .NET logging framework (Serilog).
Unfortunately I don't have a lot of time at the moment to work on additional features as I'm currently preparing TV for a 1.0 release, so I can't work on adding this particular format myself right now, but I always accept contributions when there are any :)

Alternatively, if you're willing to wait for a few months, I should be able to come around to it eventually.

My second problem is regarding columns. I think Tailviewer should columnize my file, right?

This feature is planned, but not done just yet. This feature is planned for v1.2: https://github.com/Kittyfisto/Tailviewer/projects/6

At the moment, I've helped myself to write a plugins for the log files I work with at work which tabularize the text, before being displayed. If you're interested, I can share the relevant code with you, if you'd like to try the same for you in the meantime.

[retroflex]

@Kittyfisto Thanks for the reply! Regarding levels, I could have a look at a plugin for spdlog. Can you point me in the right direction, like what interface I need to implement?

Columns: Sure, I'd like to have a look the code. I guess one can live without columns, but it gives a bit better overview.

[Kittyfisto]

@retroflex Sure, I can help you with that.

In order to achieve what you want, you have to do the following two steps:

1. Offer a way for Tailviewer to detect that a log file is an "spdlog" (as opposed to another supported format such as Serilog)
2. Offer a parser which understands the format and parses a log entry into its constituents so that tailviewer may start interpreting the columns: If the parser is able to parse the timestamp then merging multipel log files by timestamp becomes possible, if the parser is able to parse the level then coloring / filtering by log level becomes possible, etc...)

You can find an implementing for both of these jobs for Serilog here: https://github.com/Kittyfisto/Tailviewer/tree/master/src/Plugins/Tailviewer.Formats.Serilog

The workflow of this plugin requires that the user register a particular Serilog format with Tailviewer, which is why this example requires the implementation of three plugin interfaces:

ICustomLogFileFormatCreatorPlugin: https://github.com/Kittyfisto/Tailviewer/blob/master/src/Plugins/Tailviewer.Formats.Serilog/SerilogFileFormatCreatorPlugin.cs
By implementing this plugin, you unlock adding new custom formats to tailviewer. A custom format is defined by the logging library used and the custom format string which expresses how the log file is structured. Most logging frameworks don't force a particular format but instead allow the user to specify their desired format in the form of a format string. For Serilog, it might look like the following: {Timestamp:yyyy-MM-dd HH:mm:ss.fff K} [{Level:u3}] {Message}
The purpose of this plugin is to create a descriptor for one particular custom format.

From the looks of it, spdlog also allows the user to define their pattern, such as [%H:%M:%S %z] [%n] [%^---%L---%$] [thread %t] %v as taken from their documentation.

ILogFileFormatMatcherPlugin:
https://github.com/Kittyfisto/Tailviewer/blob/master/src/Plugins/Tailviewer.Formats.Serilog/SerilogFileFormatMatcherPlugin.cs
Implementing this plugin allows you to create a matcher which is given parts of a log file to try to identify which log file it is. In this example, the matcher should try to find if the log file given matches any of the registered custom spdlog formats

ILogEntryParserPlugin: https://github.com/Kittyfisto/Tailviewer/blob/master/src/Plugins/Tailviewer.Formats.Serilog/SerilogEntryParserPlugin.cs
Tailviewer will, once a log file was matched to a particular format, try to create an ILogEntryParserPlugin which is actually responsible for parsing the log file according to the format.

A good starting point would be to simply copy the Serilog plugin and make modifications based on that. It even comes with a suite of unit tests which should allow you to quickly find out if you're on the right track.

You can find an additional article on how to develop plugins here: https://github.com/Kittyfisto/Tailviewer/blob/master/docs/DevelopingPlugins.md

Regarding columnization, I'll have to go and dig it up as this plugin isn't part of tailviewer itself. I will contact you with more details.