Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parsing rules fix filter 8 #28448

Merged
merged 16 commits into from Jul 27, 2023
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
31 changes: 18 additions & 13 deletions Packs/JuniperSRX/ParsingRules/JuniperSRX/JuniperSRX.xif
@@ -1,14 +1,19 @@
[INGEST:vendor="juniper", product="srx", target_dataset="juniper_srx_raw", no_hit = keep]
// event time 1
alter tmp_event_time1 = arrayindex(regextract(_raw_log , "^(\w+\s+\d+\s+\d+\:\d+\:\d+)"),0)
,tmp_year = format_timestamp("%Y",_insert_time )
| alter tmp_event_time1 = parse_timestamp("%b %d %H:%M:%S %Y",concat(tmp_event_time1, " ", tmp_year))
// event time 2
| alter tmp_split_event_time2 = split(arrayindex(regextract(_raw_log , "\>\d+\s+(\d+\-\d+\-\d+T\d+\:\d+\:\d+\.\d+)"),0),".")
| alter tmp_time2_epoch_s = to_epoch(parse_timestamp("%Y-%m-%dT%H:%M:%S", arrayindex(tmp_split_event_time2 ,0)), "millis")
,tmp_time2_epoch_ms = to_integer(arrayindex(tmp_split_event_time2 , 1))
| alter tmp_cc = add(tmp_time2_epoch_s , tmp_time2_epoch_ms )
| alter tmp_event_time2 = to_timestamp(tmp_cc, "millis")
//
| alter _time = coalesce(tmp_event_time1, tmp_event_time2)
| fields - tmp_event_time1 , tmp_year , tmp_event_time2 , tmp_time2_epoch_ms , tmp_time2_epoch_s , tmp_cc , tmp_split_event_time2;
// Support only date time of format: yyyy-MM-ddThh:mm:%E3S or format: MMM dd hh:mm:ss. For example: "2021-12-08T10:00:00.665" or "Nov 10 10:00:00".
// Parsing time format 1
alter tmp_time1 = arrayindex(regextract(_raw_log , "^(\w+\s+\d+\s+\d+\:\d+\:\d+)"),0),
tmp_Year = format_timestamp("%Y",_insert_time)
| alter tmp_time1_1 = concat(tmp_Year, " ", tmp_time1)
| alter tmp_time1_1 = parse_timestamp("%Y %b %e %H:%M:%S", tmp_time1_1)
| alter tmp_timeDiff = timestamp_diff(tmp_time1_1, current_time(), "DAY")
// Check if the date is a future date
| alter tmp_Year2 = if(tmp_timeDiff > 0, to_string(subtract(to_integer(tmp_Year),1)),null)
// Create timestamp minus 1 year if the timestamp is a future one
| alter tmp_time1_2 = if(tmp_Year2 != null, concat(tmp_Year2, " ", tmp_time1), null)
| alter tmp_time1_2 = if(tmp_time1_2 != null, parse_timestamp("%Y %b %e %H:%M:%S", tmp_time1_2), null)
| alter tmp_time1 = coalesce(tmp_time1_2, tmp_time1_1)
// Parsing time format 2
| alter tmp_time2 = arrayindex(regextract(_raw_log , "\>\d+\s+(\d+\-\d+\-\d+T\d+\:\d+\:\d+\.\d+)"),0)
| alter tmp_time2 = parse_timestamp("%Y-%m-%dT%H:%M:%E3S", tmp_time2)
| alter _time = coalesce(tmp_time1, tmp_time2)
| fields -tmp_time1, tmp_Year, tmp_time1_1, tmp_timeDiff, tmp_Year2, tmp_time1_2, tmp_time2;
27 changes: 27 additions & 0 deletions Packs/JuniperSRX/README.md
@@ -1,3 +1,29 @@
# Juniper SRX
This pack includes Cortex XSIAM content.

## Configuration on Server Side
You need to configure your Juniper SRX device to forward Syslog messages.

Perform the following in order to configure log forwarding:
1. Log in to the Juniper SRX device CLI console.
2. Run the below command:
```bash
set system syslog host <IP address of the remote Syslog server> any any
```
The instructions above set the logging with default configuration values.


Note: In order to parse the timestamp correctly, make sure that the SRX devices are configured with the default time zone (UTC).
The supported time formats are:
- yyyy-MM-ddThh:mm:%E3S (2021-12-08T10:00:00.665)
- MMM dd hh:mm:ss (Nov 10 10:00:00)

More information can be found [here](https://supportportal.juniper.net/s/article/SRX-Getting-Started-Configure-System-Logging?language=en_US) and [here](https://supportportal.juniper.net/s/article/SRX-Getting-Started-Configure-Logging?language=en_US).


## Collect Events from Vendor
In order to use the collector, use the [Broker VM](#broker-vm) option.

### Broker VM
You will need to use the information described [here](https://docs-cortex.paloaltonetworks.com/r/Cortex-XDR/Cortex-XDR-Pro-Administrator-Guide/Configure-the-Broker-VM).\
You can configure the specific vendor and product for this instance.
Expand All @@ -6,3 +32,4 @@ You can configure the specific vendor and product for this instance.
3. When configuring the Syslog Collector, set:
- vendor as vendor<- juniper
- product as product<- srx

3 changes: 3 additions & 0 deletions Packs/JuniperSRX/ReleaseNotes/1_0_9.md
@@ -0,0 +1,3 @@
#### Parsing Rules
##### JuniperSRX Parsing Rule
- Added a filter in the parsing rule to enhance its logic.
yasta5 marked this conversation as resolved.
Show resolved Hide resolved
2 changes: 1 addition & 1 deletion Packs/JuniperSRX/pack_metadata.json
Expand Up @@ -2,7 +2,7 @@
"name": "JuniperSRX",
"description": "Modeling Rules for the JuniperSRX logs collector",
"support": "xsoar",
"currentVersion": "1.0.8",
"currentVersion": "1.0.9",
"author": "Cortex XSOAR",
"url": "https://www.paloaltonetworks.com/cortex",
"email": "",
Expand Down
@@ -1,5 +1,6 @@
[INGEST:vendor="sonicwall", product="ns", target_dataset="sonicwall_ns_raw", no_hit=keep]
filter _raw_log contains "UTC"
|alter timestamp=arrayindex(regextract(_raw_log,"time\=\"([^=]+)\sUTC\""),0)
|alter _time=parse_timestamp("%Y-%m-%d %H:%M:%S", timestamp)
|fields -timestamp;
// Support only date time of format: yyyy-MM-dd hh:mm:ss. For example: "2021-12-08 10:00:00".
filter _raw_log ~= "time\=\"(\d{4,}-\d{2}-\d{2} \d{2}:\d{2}:\d{2})\sUTC\""
| alter timestamp = arrayindex(regextract(_raw_log,"time\=\"([^=]+)\sUTC\""),0)
| alter _time = parse_timestamp("%Y-%m-%d %H:%M:%S", timestamp)
| fields -timestamp;
5 changes: 4 additions & 1 deletion Packs/SonicWallNSv/README.md
Expand Up @@ -11,7 +11,10 @@ To configure the Sonicwall NSv to send Syslog to XSIAM server (Broker VM), use t
To configure Sonicwall to send logs in UTC formats please do the following:
1. Navigate to "Logs" -> "Syslog" -> "Syslog Settings"
2. Turn on "Display Syslog Timestamp in UTC"


Note: In order to parse the timestamp correctly, make sure that the timestamp is displayed in UTC.
The supported time format is yyyy-MM-dd hh:mm:ss (2021-12-08 10:00:00).

## Collect Events from Vendor

In order to use the collector, use the [Broker VM](#broker-vm) option.
Expand Down
3 changes: 3 additions & 0 deletions Packs/SonicWallNSv/ReleaseNotes/1_0_1.md
@@ -0,0 +1,3 @@
#### Parsing Rules
##### SonicwallNSv Parsing Rule
- Added a filter in the parsing rule to enhance its logic.
yasta5 marked this conversation as resolved.
Show resolved Hide resolved
2 changes: 1 addition & 1 deletion Packs/SonicWallNSv/pack_metadata.json
Expand Up @@ -2,7 +2,7 @@
"name": "SonicWall NSv",
"description": "The SonicWall NSv Series virtual firewall offers all the security advantages of a physical firewall with the operational and economic benefits of virtualization, including system scalability and agility, speed of system provisioning, simple management and cost reduction.",
"support": "xsoar",
"currentVersion": "1.0.0",
"currentVersion": "1.0.1",
"author": "Cortex XSOAR",
"url": "https://www.paloaltonetworks.com/cortex",
"email": "",
Expand Down
@@ -1,2 +1,3 @@
[INGEST:vendor="tenable", product="io", target_dataset="tenable_io_raw", no_hit=keep]
alter _time = coalesce(received, indexed);
filter received != null or indexed!= null
| alter _time = coalesce(received, indexed);
3 changes: 3 additions & 0 deletions Packs/Tenable_io/ReleaseNotes/2_1_10.md
@@ -0,0 +1,3 @@
#### Parsing Rules
##### Tenable.io Parsing Rule
- Added a filter in the parsing rule to enhance its logic.
yasta5 marked this conversation as resolved.
Show resolved Hide resolved
2 changes: 1 addition & 1 deletion Packs/Tenable_io/pack_metadata.json
Expand Up @@ -2,7 +2,7 @@
"name": "Tenable.io",
"description": "A comprehensive asset centric solution to accurately track resources while accommodating dynamic assets such as cloud, mobile devices, containers and web applications.",
"support": "xsoar",
"currentVersion": "2.1.9",
"currentVersion": "2.1.10",
"author": "Cortex XSOAR",
"url": "https://www.paloaltonetworks.com/cortex",
"email": "",
Expand Down