Problem extracting data (Fields related)

[marceloyb]

Hi, i downloaded the app and started using it. I had to make some adaptations on the search of the dashboards, since in my Splunk the fields you were searching are not default extracted. The thing is, in some dashboards i managed to do fine in the extracting, but in some others i can't get trustable information. Could you please share how you did your field extraction?

[diogofgm]

Hi Marcelo
Have you installed the add on? Also what format are you using? CEF or LEEF?

[marceloyb]

Yes, addon is installed. I believe CEF, since it's been configured directly in KSC Panel.

[diogofgm]

You should be using LEEF. If I recall correctly you need to choose QRadar in the format to have it output LEEF. Splunk option wasn’t available when I started building these add nos and app.

[marceloyb]

Tried changing to LEEF (QRadar) and Splunk stopped receiving the logs if i use sourcetype kaspersky:leef. Any ideas why? If i use the "syslog" sourcetype it works.

Edit: Actually it is receiving, but the kaspersky:leef sourcetype is somehow throwing events into my indexer with wrong time stamp (4 hours in the future)

[diogofgm]

can you post a sanitised sample of you data? the only reason to have data generated with a timestamp in the future would be if you have problems with timezone or misconfigured server times since the add on is not doing anything regarding time. Check the time of you splunk server and kaspersky server. Are you using a common ntp server?

[diogofgm]

After looking at you profile I saw you might be in GMT-3. I was thinking, if I'm correct 4h in the future would be something like your data is coming in as UTC if you are under DST.
Thats something it never occurred to me and I actually didn't configured in the TA since in Portugal we are GMT/UTC (Apart from when we are GMT+1 due to DST). Its possible that kaspersky is outputting data in UTC and since no TZ is defined you get data in the future.

[marceloyb]

Probably. The problem happens once the data get into the TA i made a change on the sourcetype kaspersky:leef so the timestamp will be defined on the Current time, but there are still some discrepancies. Since i'm kinda newbie on Splunk, i couldn't find where i could fix the cause, but with the changes the app is working fine.

[diogofgm]

Have you compared the timestamp and TZ of the server where you are hosting the Kaspersky SC console with the on in the logs?
What is the TZ you have setup in your splunk profile (Your account name > preferences)?
In the image you posted, what is the right timestamp? the one in splunk or the one in the logs?

The thing here is, when you index data you can either extract or assign a TZ to your data. This allows splunk to adjust it to TZ the one you have in your profile.

From splunk docs:
https://docs.splunk.com/Documentation/Splunk/7.3.2/Admin/Propsconf

TZ = <timezone identifier>
* The algorithm for determining the time zone for a particular event is as
  follows:
  * If the event has a timezone in its raw text (for example, UTC, -08:00),
  use that.
  * If TZ is set to a valid timezone string, use that.
  * If the event was forwarded, and the forwarder-indexer connection uses
  the version 6.0 and higher forwarding protocol, use the timezone provided
  by the forwarder.
  * Otherwise, use the timezone of the system that is running splunkd.
* Default: empty string

[marceloyb]

I checked some logs from yesterday, when i used the "syslog" Source type on the Data Input, and the results are the same that i got using the "kaspersky:leef" Source type, so i don't think the problem is actually the TA. On the Kaspersky SC console the time is correct. The TZ on my profile is GMT -3 (Brasilia).

When i used the CEF format for the logs the time wasn't showing on the log, but with the LEEF it shows. Could it be related smh?

[diogofgm]

From what you're telling me I'm starting to believe its safe to assume that regardless the time you have in Kaspersky server, LEEF will always output UTC. If that's the case, its just a matter of adding TZ to the "kaspersky:leef" source type. Can you test adding TZ = UTC to the source type in the TA and index some data?

[marceloyb]

add TZ = UTC to the /etc/apps/TA-kaspersky/default/props.conf file, right? Did and still keeps the same.

This is the config on the "kaspersky:leef" source type regarding Timestamp

[diogofgm]

have you restarted Splunk after that?

[marceloyb]

Yes. I was looking the logs and on some of them, the time was -1 instead of the +4. I don't remember what exactly i did after that, but it involved reinstalling TA and messing in the configurations

[diogofgm]

Ok ive been looking into my data and it seams that my thesis is correct. LEEF outputs in UTC/GMT. I have 1h diference in my data because we are in DST with means GMT+1. I’m going to fix this TZ issue and release an update.

[diogofgm]

The new version of the add on has been released in splunkbase. Please check if it solves your issue.