Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update cef to ECS 8.2 #2804

Merged
merged 7 commits into from
Apr 25, 2022
Merged

Update cef to ECS 8.2 #2804

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
cddc855
Update cef to ECS 8.2
taylor-swanson Feb 22, 2022
1720b32
Review comments
taylor-swanson Mar 15, 2022
a24065e
Remove event.ingested
taylor-swanson Mar 15, 2022
a5d846c
Merge branch 'main' into ecs8.2-cef
taylor-swanson Mar 15, 2022
c4a46cc
Make changes be minor
taylor-swanson Mar 22, 2022
8f33a2e
Merge branch 'main' into ecs8.2-cef
taylor-swanson Apr 5, 2022
94a269f
Regen files after merge
taylor-swanson Apr 5, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion packages/cef/_dev/build/build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
dependencies:
ecs:
reference: git@8.0
reference: git@8.2
8 changes: 4 additions & 4 deletions packages/cef/_dev/build/docs/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,8 +62,8 @@ Check Point CEF extensions are mapped as follows:
| requestCookies | - | - | checkpoint.cookie |
| sourceNtDomain | - | dns.question.name | - |
| Signature | - | vulnerability.id | - |
| Recipient | - | destination.user.email | - |
| Sender | - | source.user.email | - |
| Recipient | - | email.to.address | - |
| Sender | - | email.from.address | - |
| deviceCustomFloatingPoint1 | update version | observer.version | - |
| deviceCustomIPv6Address2 | source ipv6 address | source.ip | - |
| deviceCustomIPv6Address3 | destination ipv6 address | destination.ip | - |
Expand All @@ -80,7 +80,7 @@ Check Point CEF extensions are mapped as follows:
| deviceCustomString1 | dlp rule name | rule.name | - |
| deviceCustomString1 | email id | - | checkpoint.email_id |
| deviceCustomString2 | category | - | checkpoint.category |
| deviceCustomString2 | email subject | - | checkpoint.email_subject |
| deviceCustomString2 | email subject | email.subject | checkpoint.email_subject |
| deviceCustomString2 | sensor mode | - | checkpoint.sensor_mode |
| deviceCustomString2 | protection id | - | checkpoint.protection_id |
| deviceCustomString2 | scan invoke type | - | checkpoint.integrity_av_invoke_type |
Expand Down Expand Up @@ -108,7 +108,7 @@ Check Point CEF extensions are mapped as follows:
| deviceCustomString5 | matched category | rule.category | - |
| deviceCustomString5 | vlan id | network.vlan.id | - |
| deviceCustomString5 | authentication method | - | checkpoint.auth_method |
| deviceCustomString5 | email session id | - | checkpoint.email_session_id |
| deviceCustomString5 | email session id | email.message_id | checkpoint.email_session_id |
| deviceCustomDate2 | subscription expiration | - | checkpoint.subs_exp |
| deviceFlexNumber1 | confidence | - | checkpoint.confidence_level |
| deviceFlexNumber2 | performance impact | - | checkpoint.performance_impact |
Expand Down
5 changes: 5 additions & 0 deletions packages/cef/changelog.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
# newer versions go on top
- version: "2.0.0"
changes:
- description: Update to ECS 8.2 by modifying Check Point events to use the new email field set.
type: enhancement
link: https://github.com/elastic/integrations/pull/2804
- version: "1.4.2"
changes:
- description: Add field mappings for several `event.*` fields.
Expand Down
8 changes: 4 additions & 4 deletions packages/cef/data_stream/log/_dev/test/pipeline/test-cef.json-expected.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@
"port": 443
},
"ecs": {
"version": "8.0.0"
"version": "8.2.0"
},
"event": {
"code": "18",
Expand Down Expand Up @@ -146,7 +146,7 @@
}
},
"ecs": {
"version": "8.0.0"
"version": "8.2.0"
},
"event": {
"code": "18",
Expand Down Expand Up @@ -227,7 +227,7 @@
}
},
"ecs": {
"version": "8.0.0"
"version": "8.2.0"
},
"event": {
"code": "18",
Expand Down Expand Up @@ -272,7 +272,7 @@
"ip": "192.168.1.2"
},
"ecs": {
"version": "8.0.0"
"version": "8.2.0"
},
"event": {
"code": "18",
Expand Down
40 changes: 40 additions & 0 deletions packages/cef/data_stream/log/_dev/test/pipeline/test-cef.log-expected.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
{
"expected": [
{
"ecs": {
"version": "8.2.0"
},
"message": "CEF:0|Elastic|Vaporware|1.0.0-alpha|18|Web request|low|eventId=3457 requestMethod=POST slat=38.915 slong=-77.511 proto=TCP sourceServiceName=httpd requestContext=https://www.google.com src=89.160.20.156 spt=33876 dst=192.168.10.1 dpt=443 request=https://www.example.com/cart",
"tags": [
"preserve_original_event"
]
},
{
"ecs": {
"version": "8.2.0"
},
"message": "CEF:0|Elastic|Vaporware|1.0.0-alpha|18|Authentication|low|eventId=123 src=89.160.20.156 spt=33876 dst=89.160.20.156 dpt=443 duser=alice suser=bob destinationTranslatedAddress=10.10.10.10 fileHash=bc8bbe52f041fd17318f08a0f73762ce oldFileHash=a9796280592f86b74b27e370662d41eb",
"tags": [
"preserve_original_event"
]
},
{
"ecs": {
"version": "8.2.0"
},
"message": "CEF:0|Elastic|Vaporware|1.0.0-alpha|18|Authentication|low|spriv=user dpriv=root",
"tags": [
"preserve_original_event"
]
},
{
"ecs": {
"version": "8.2.0"
},
"message": "CEF:0|Elastic|Vaporware|1.0.0-alpha|18|Authentication|low|message=This event is padded with whitespace dst=192.168.1.2 src=192.168.3.4 ",
"tags": [
"preserve_original_event"
]
}
]
}
6 changes: 3 additions & 3 deletions packages/cef/data_stream/log/_dev/test/pipeline/test-checkpoint.json-expected.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@
"port": 443
},
"ecs": {
"version": "8.0.0"
"version": "8.2.0"
},
"event": {
"action": "Accept",
Expand Down Expand Up @@ -166,7 +166,7 @@
"port": 25
},
"ecs": {
"version": "8.0.0"
"version": "8.2.0"
},
"event": {
"action": "Bypass",
Expand Down Expand Up @@ -235,7 +235,7 @@
"ip": "::1"
},
"ecs": {
"version": "8.0.0"
"version": "8.2.0"
},
"event": {
"action": "Drop",
Expand Down
31 changes: 31 additions & 0 deletions packages/cef/data_stream/log/_dev/test/pipeline/test-checkpoint.log-expected.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
{
taylor-swanson marked this conversation as resolved.
Show resolved Hide resolved
"expected": [
{
"ecs": {
"version": "8.2.0"
},
"message": "CEF:0|Check Point|VPN-1 \u0026 FireWall-1|Check Point|Log|https|Unknown|act=Accept destinationTranslatedAddress=0.0.0.0 destinationTranslatedPort=0 deviceDirection=0 rt=1543270652000 sourceTranslatedAddress=192.168.103.254 sourceTranslatedPort=35398 spt=49363 dpt=443 cs2Label=Rule Name layer_name=Network layer_uuid=b406b732-2437-4848-9741-6eae1f5bf112 match_id=4 parent_rule=0 rule_action=Accept rule_uid=9e5e6e74-aa9a-4693-b9fe-53712dd27bea ifname=eth0 logid=0 loguid={0x5bfc70fc,0x1,0xfe65a8c0,0xc0000001} origin=192.168.101.254 originsicname=CN\\=R80,O\\=R80_M..6u6bdo sequencenum=1 version=5 dst=89.160.20.156 inzone=Internal nat_addtnl_rulenum=1 nat_rulenum=4 outzone=External product=VPN-1 \u0026 FireWall-1 proto=6 service_id=https src=192.168.101.100 cs5Label=Matched Category cs5=Business / Economy deviceCustomDate2=1508150533713 deviceCustomDate2Label=This field is made up",
"tags": [
"preserve_original_event"
]
},
{
"ecs": {
"version": "8.2.0"
},
"message": "CEF:0|Check Point|VPN-1 \u0026 FireWall-1|Check Point|Log|https|Unknown|act=Bypass cn1Label=Email Recipients Number cs1Label=Email ID cs4Label=Email Control cs4=SMTP Policy Restrictions cs5Label=Email Session ID deviceDirection=0 msg=Encrypted session rt=1545211330000 spt=4001 dpt=25 fileHash=55f4a511e6f630a6b1319505414f114e7bcaf13d deviceCustomDate2=Apr 11 2020 10:42:13 deviceCustomDate2Label=Subscription expiration",
"tags": [
"preserve_original_event"
]
},
{
"ecs": {
"version": "8.2.0"
},
"message": "CEF:0|Check Point|VPN-1 \u0026 FireWall-1|Check Point|Log|https|Unknown|act=Drop cp_app_risk=High cp_severity=Very-High baseEventCount=12 deviceFacility=4 c6a2=fd00::555 c6a2Label=Source IPv6 Address c6a3=::1 c6a3Label=Destination IPv6 Address fileHash=580a783c1cb2b20613323f715d231a69 cn2=5 cn2Label=Duration in Seconds",
"tags": [
"preserve_original_event"
]
}
]
}
20 changes: 10 additions & 10 deletions packages/cef/data_stream/log/_dev/test/pipeline/test-fp-ngfw-smc.json-expected.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
"version": "0"
},
"ecs": {
"version": "8.0.0"
"version": "8.2.0"
},
"event": {
"code": "0",
Expand Down Expand Up @@ -66,7 +66,7 @@
"version": "0"
},
"ecs": {
"version": "8.0.0"
"version": "8.2.0"
},
"event": {
"code": "9005",
Expand Down Expand Up @@ -122,7 +122,7 @@
"ip": "10.1.1.40"
},
"ecs": {
"version": "8.0.0"
"version": "8.2.0"
},
"event": {
"action": "Allow",
Expand Down Expand Up @@ -213,7 +213,7 @@
"port": 67
},
"ecs": {
"version": "8.0.0"
"version": "8.2.0"
},
"event": {
"code": "70019",
Expand Down Expand Up @@ -284,7 +284,7 @@
"ip": "192.168.1.1"
},
"ecs": {
"version": "8.0.0"
"version": "8.2.0"
},
"event": {
"action": "Refuse",
Expand Down Expand Up @@ -357,7 +357,7 @@
}
},
"ecs": {
"version": "8.0.0"
"version": "8.2.0"
},
"event": {
"code": "70021",
Expand Down Expand Up @@ -416,7 +416,7 @@
"version": "0"
},
"ecs": {
"version": "8.0.0"
"version": "8.2.0"
},
"event": {
"code": "72714",
Expand Down Expand Up @@ -474,7 +474,7 @@
"version": "0"
},
"ecs": {
"version": "8.0.0"
"version": "8.2.0"
},
"event": {
"code": "72715",
Expand Down Expand Up @@ -532,7 +532,7 @@
"version": "0"
},
"ecs": {
"version": "8.0.0"
"version": "8.2.0"
},
"event": {
"code": "72716",
Expand Down Expand Up @@ -589,7 +589,7 @@
"version": "0"
},
"ecs": {
"version": "8.0.0"
"version": "8.2.0"
},
"event": {
"code": "78002",
Expand Down
121 changes: 121 additions & 0 deletions packages/cef/data_stream/log/_dev/test/pipeline/test-fp-ngfw-smc.log-expected.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
{
"expected": [
{
"ecs": {
"version": "8.2.0"
},
"message": "CEF:0|FORCEPOINT|Firewall|6.6.1|0|Generic|0|deviceExternalId=Master FW node 1 dvc=10.1.1.40 dvchost=10.1.1.40 msg=log server connection established deviceFacility=Logging System rt=Jan 17 2020 08:52:10",
"tags": [
"preserve_original_event"
]
},
{
"ecs": {
"version": "8.2.0"
},
"message": "CEF:0|FORCEPOINT|Firewall|6.6.1|9005|FW_Communication-Communication-Error|0|deviceExternalId=Master FW node 1 dvc=10.1.1.40 dvchost=10.1.1.40 msg=Communication error: No route to host (-3, 5, 0) deviceFacility=Management rt=Jan 17 2020 08:52:09",
"tags": [
"preserve_original_event"
]
},
{
"ecs": {
"version": "8.2.0"
},
"message": "CEF:0|FORCEPOINT|Firewall|6.6.1|70018|Connection_Allowed|0|deviceExternalId=Master FW node 1 dvc=10.1.1.40 dvchost=10.1.1.40 src=10.37.205.252 dst=10.1.1.40 proto=1 deviceOutboundInterface=255 act=Allow msg=Referred connection: 10.1.1.40 -\u003e 10.37.133.35 frag\\=0x4000 TCP 47413-\u003e3020 deviceFacility=Packet Filtering rt=Jan 17 2020 08:52:09 app=Dest. Unreachable (Host Unreachable) cs1Label=RuleID cs1=2097157.1",
"tags": [
"preserve_original_event"
]
},
{
"ecs": {
"version": "8.2.0"
},
"message": "CEF:0|FORCEPOINT|Firewall|unknown|70019|Connection_Discarded|0|deviceExternalId=Firewall-10 node 1 dvc=10.1.1.10 dvchost=10.1.1.10 src=172.16.1.1 dst=89.160.20.156 spt=68 dpt=67 proto=17 deviceOutboundInterface=255 deviceFacility=Packet Filtering rt=Jan 17 2020 08:56:21 app=BOOTPS (UDP) cs1Label=RuleID cs1=605.0",
"tags": [
"preserve_original_event"
]
},
{
"ecs": {
"version": "8.2.0"
},
"message": "CEF:0|FORCEPOINT|Firewall|unknown|70020|Connection_Refused|0|deviceExternalId=Firewall-1 node 1 dvc=10.1.1.1 dvchost=10.1.1.1 src=172.16.1.1 dst=192.168.1.1 proto=1 deviceOutboundInterface=255 act=Refuse deviceFacility=Packet Filtering rt=Jan 17 2020 08:56:23 app=Echo Request (No Code) cs1Label=RuleID cs1=601.0",
"tags": [
"preserve_original_event"
]
},
{
"ecs": {
"version": "8.2.0"
},
"message": "CEF:0|FORCEPOINT|Firewall|unknown|70021|Connection_Closed|0|deviceExternalId=Firewall-6 node 1 dvc=10.1.1.6 dvchost=10.1.1.6 proto=6 deviceOutboundInterface=255 destinationServiceName=YouTube suser=alice deviceFacility=Packet Filtering rt=Jan 17 2020 08:56:20 app=TCP in=32526 out=27366",
"tags": [
"preserve_original_event"
]
},
{
"ecs": {
"version": "8.2.0"
},
"message": "CEF:0|FORCEPOINT|Firewall|unknown|72714|ECA_Metadata_login|0|deviceExternalId=Firewall-3 node 1 dvc=10.1.1.3 dvchost=10.1.1.3 src=192.168.1.1 suser=bob deviceFacility=Endpoint Context Agent rt=Jan 17 2020 08:56:33",
"tags": [
"preserve_original_event"
]
},
{
"ecs": {
"version": "8.2.0"
},
"message": "CEF:0|FORCEPOINT|Firewall|unknown|72715|ECA_Metadata_logout|0|deviceExternalId=Firewall-10 node 1 dvc=10.1.1.10 dvchost=10.1.1.10 src=192.168.1.1 suser=bob deviceFacility=Endpoint Context Agent rt=Jan 17 2020 08:56:31",
"tags": [
"preserve_original_event"
]
},
{
"ecs": {
"version": "8.2.0"
},
"message": "CEF:0|FORCEPOINT|Firewall|unknown|72716|ECA_Metadata_system_metadata_received|0|deviceExternalId=Firewall-8 node 1 dvc=10.1.1.8 dvchost=10.1.1.8 src=172.16.2.1 suser=alice deviceFacility=Endpoint Context Agent rt=Jan 17 2020 08:56:26",
"tags": [
"preserve_original_event"
]
},
{
"ecs": {
"version": "8.2.0"
},
"message": "CEF:0|FORCEPOINT|Firewall|6.6.1|78002|TLS connection state|0|deviceExternalId=Master FW node 1 dvc=10.1.1.40 dvchost=10.1.1.40 msg=TLS: Couldn't establish TLS connection (11, N/A) deviceFacility=Management rt=Jan 17 2020 08:52:09",
"tags": [
"preserve_original_event"
]
},
{
"ecs": {
"version": "8.2.0"
},
"message": "",
"tags": [
"preserve_original_event"
]
},
{
"ecs": {
"version": "8.2.0"
},
"message": "",
"tags": [
"preserve_original_event"
]
},
{
"ecs": {
"version": "8.2.0"
},
"message": "",
"tags": [
"preserve_original_event"
]
}
]
}
Loading