Skip to content

enhance: add base CEF and unifi CEF #1457

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 12 commits into
base: master
Choose a base branch
from
Open

enhance: add base CEF and unifi CEF #1457

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
e2dd90b
enhance: add base CEF and unifi CEF
LaurenceJJones Sep 4, 2025
f669a68
enhance: forgot assert
LaurenceJJones Sep 4, 2025
9f32635
enhance: move assert
LaurenceJJones Sep 4, 2025
084452d
enhance: add collection with all information about rsyslog
LaurenceJJones Sep 4, 2025
5d1b2dd
enhance: add more rsyslog config
LaurenceJJones Sep 4, 2025
1c76ce0
enhance: add more rsyslog config
LaurenceJJones Sep 4, 2025
46ba761
enhance: remove cef unifi and have a single unifi collection
LaurenceJJones Sep 4, 2025
1e55c85
Merge branch 'master' into unifi-cef-parser
LaurenceJJones Sep 4, 2025
db36da2
enhance: better rsyslog configuration
LaurenceJJones Sep 4, 2025
24eae22
enhance: Add utc timestamp to security and update other files
LaurenceJJones Sep 5, 2025
2cadc0b
enhance: Unifi puts a space between cef: and the version, add an opti…
LaurenceJJones Sep 8, 2025
4b73a7d
enhance: Add optional syslog headers to CEF incase of not stripping s…
LaurenceJJones Sep 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .tests/cef-logs/cef-logs.log
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
CEF:0|Ubiquiti|UniFi Network|9.4.19|544|Admin Accessed UniFi Network|1|UNIFIcategory=System UNIFIsubCategory=Admin UNIFIhost=Unifi Dream Machine UNIFIaccessMethod=web UNIFIadmin=Secure Admin src=10.72.1.222 UNIFIutcTime=2025-09-04T08:32:58.445Z msg=Secure Admin accessed UniFi Network using the web. Source IP: 10.72.1.222
CEF: 0|Ubiquiti|UniFi Network|9.4.19|544|Admin Accessed UniFi Network|1|UNIFIcategory=System UNIFIsubCategory=Admin UNIFIhost=Unifi Dream Machine UNIFIaccessMethod=web UNIFIadmin=Secure Admin src=10.72.1.222 UNIFIutcTime=2025-09-04T08:32:58.445Z msg=Secure Admin accessed UniFi Network using the web. Source IP: 10.72.1.222
0|Ubiquiti|UniFi Network|9.3.45|201|Threat Detected and Blocked|7|proto=TCP src=192.168.0.1 spt=54587 dst=192.168.0.233 dpt=80 UNIFIcategory=Security UNIFIsubCategory=Intrusion Prevention UNIFIhost=Express 7 UNIFIdeviceMac=84:78:48:80:0d:86 UNIFIdeviceName=Express 7 UNIFIdeviceModel=UX7 UNIFIdeviceIp=192.168.0.1 UNIFIdeviceVersion=4.2.15 UNIFIrisk=medium UNIFIipsSessionId=2138629792252828 UNIFIipsSignature=ET DROP Dshield Block Listed Source group 1 UNIFIipsSignatureId=2402000 msg=A network intrusion attempt from 192.168.0.1 to DS920+ macvlan has been detected and blocked.
Sep 8 08:32:20 UDM-Gent CEF: 0|Ubiquiti|UniFi Network|9.4.19|201|Threat Detected and Blocked|9|proto=TCP src=192.168.100.252 spt=65020 dst=192.168.1.100 dpt=80 UNIFIcategory=Security UNIFIsubCategory=Intrusion Prevention UNIFIhost=UDM-Gent UNIFIsite=UDM-Gent UNIFIdeviceMac=78:45:58:de:fc:e7 UNIFIdeviceName=UDM-Gent UNIFIdeviceModel=UDM UNIFIdeviceIp=192.168.100.1 UNIFIdeviceVersion=4.4.0 UNIFIrisk=high UNIFIipsSessionId=1328559940562927 UNIFIipsSignature=ET USER_AGENTS Suspicious User Agent (BlackSun) UNIFIipsSignatureId=2008983 UNIFIutcTime=2025-09-08T06:32:20.613Z msg=A network intrusion attempt from Desktop d5:5d to 192.168.1.100 has been detected and blocked.
10 changes: 10 additions & 0 deletions .tests/cef-logs/config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
parsers:
- ./parsers/s00-raw/crowdsecurity/cef-logs.yaml
scenarios:
- ""
postoverflows:
- ""
collections:
- ""
log_file: cef-logs.log
log_type: cef
61 changes: 61 additions & 0 deletions .tests/cef-logs/parser.assert
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
len(results) == 2
len(results["s00-raw"]["crowdsecurity/cef-logs"]) == 4
results["s00-raw"]["crowdsecurity/cef-logs"][0].Success == true
results["s00-raw"]["crowdsecurity/cef-logs"][0].Evt.Parsed["cef_device_product"] == "UniFi Network"
results["s00-raw"]["crowdsecurity/cef-logs"][0].Evt.Parsed["cef_device_vendor"] == "Ubiquiti"
results["s00-raw"]["crowdsecurity/cef-logs"][0].Evt.Parsed["cef_device_version"] == "9.4.19"
results["s00-raw"]["crowdsecurity/cef-logs"][0].Evt.Parsed["cef_event_name"] == "Admin Accessed UniFi Network"
results["s00-raw"]["crowdsecurity/cef-logs"][0].Evt.Parsed["cef_severity"] == "1"
results["s00-raw"]["crowdsecurity/cef-logs"][0].Evt.Parsed["cef_signature_id"] == "544"
results["s00-raw"]["crowdsecurity/cef-logs"][0].Evt.Parsed["cef_version"] == "0"
results["s00-raw"]["crowdsecurity/cef-logs"][0].Evt.Parsed["logsource"] == "cef"
results["s00-raw"]["crowdsecurity/cef-logs"][0].Evt.Parsed["message"] == "UNIFIcategory=System UNIFIsubCategory=Admin UNIFIhost=Unifi Dream Machine UNIFIaccessMethod=web UNIFIadmin=Secure Admin src=10.72.1.222 UNIFIutcTime=2025-09-04T08:32:58.445Z msg=Secure Admin accessed UniFi Network using the web. Source IP: 10.72.1.222"
results["s00-raw"]["crowdsecurity/cef-logs"][0].Evt.Parsed["program"] == "Ubiquiti"
basename(results["s00-raw"]["crowdsecurity/cef-logs"][0].Evt.Meta["datasource_path"]) == "cef-logs.log"
results["s00-raw"]["crowdsecurity/cef-logs"][0].Evt.Meta["datasource_type"] == "file"
results["s00-raw"]["crowdsecurity/cef-logs"][0].Evt.Whitelisted == false
results["s00-raw"]["crowdsecurity/cef-logs"][1].Success == true
results["s00-raw"]["crowdsecurity/cef-logs"][1].Evt.Parsed["cef_device_product"] == "UniFi Network"
results["s00-raw"]["crowdsecurity/cef-logs"][1].Evt.Parsed["cef_device_vendor"] == "Ubiquiti"
results["s00-raw"]["crowdsecurity/cef-logs"][1].Evt.Parsed["cef_device_version"] == "9.4.19"
results["s00-raw"]["crowdsecurity/cef-logs"][1].Evt.Parsed["cef_event_name"] == "Admin Accessed UniFi Network"
results["s00-raw"]["crowdsecurity/cef-logs"][1].Evt.Parsed["cef_severity"] == "1"
results["s00-raw"]["crowdsecurity/cef-logs"][1].Evt.Parsed["cef_signature_id"] == "544"
results["s00-raw"]["crowdsecurity/cef-logs"][1].Evt.Parsed["cef_version"] == "0"
results["s00-raw"]["crowdsecurity/cef-logs"][1].Evt.Parsed["logsource"] == "cef"
results["s00-raw"]["crowdsecurity/cef-logs"][1].Evt.Parsed["message"] == "UNIFIcategory=System UNIFIsubCategory=Admin UNIFIhost=Unifi Dream Machine UNIFIaccessMethod=web UNIFIadmin=Secure Admin src=10.72.1.222 UNIFIutcTime=2025-09-04T08:32:58.445Z msg=Secure Admin accessed UniFi Network using the web. Source IP: 10.72.1.222"
results["s00-raw"]["crowdsecurity/cef-logs"][1].Evt.Parsed["program"] == "Ubiquiti"
basename(results["s00-raw"]["crowdsecurity/cef-logs"][1].Evt.Meta["datasource_path"]) == "cef-logs.log"
results["s00-raw"]["crowdsecurity/cef-logs"][1].Evt.Meta["datasource_type"] == "file"
results["s00-raw"]["crowdsecurity/cef-logs"][1].Evt.Whitelisted == false
results["s00-raw"]["crowdsecurity/cef-logs"][2].Success == true
results["s00-raw"]["crowdsecurity/cef-logs"][2].Evt.Parsed["cef_device_product"] == "UniFi Network"
results["s00-raw"]["crowdsecurity/cef-logs"][2].Evt.Parsed["cef_device_vendor"] == "Ubiquiti"
results["s00-raw"]["crowdsecurity/cef-logs"][2].Evt.Parsed["cef_device_version"] == "9.3.45"
results["s00-raw"]["crowdsecurity/cef-logs"][2].Evt.Parsed["cef_event_name"] == "Threat Detected and Blocked"
results["s00-raw"]["crowdsecurity/cef-logs"][2].Evt.Parsed["cef_severity"] == "7"
results["s00-raw"]["crowdsecurity/cef-logs"][2].Evt.Parsed["cef_signature_id"] == "201"
results["s00-raw"]["crowdsecurity/cef-logs"][2].Evt.Parsed["cef_version"] == "0"
results["s00-raw"]["crowdsecurity/cef-logs"][2].Evt.Parsed["logsource"] == "cef"
results["s00-raw"]["crowdsecurity/cef-logs"][2].Evt.Parsed["message"] == "proto=TCP src=192.168.0.1 spt=54587 dst=192.168.0.233 dpt=80 UNIFIcategory=Security UNIFIsubCategory=Intrusion Prevention UNIFIhost=Express 7 UNIFIdeviceMac=84:78:48:80:0d:86 UNIFIdeviceName=Express 7 UNIFIdeviceModel=UX7 UNIFIdeviceIp=192.168.0.1 UNIFIdeviceVersion=4.2.15 UNIFIrisk=medium UNIFIipsSessionId=2138629792252828 UNIFIipsSignature=ET DROP Dshield Block Listed Source group 1 UNIFIipsSignatureId=2402000 msg=A network intrusion attempt from 192.168.0.1 to DS920+ macvlan has been detected and blocked."
results["s00-raw"]["crowdsecurity/cef-logs"][2].Evt.Parsed["program"] == "Ubiquiti"
basename(results["s00-raw"]["crowdsecurity/cef-logs"][2].Evt.Meta["datasource_path"]) == "cef-logs.log"
results["s00-raw"]["crowdsecurity/cef-logs"][2].Evt.Meta["datasource_type"] == "file"
results["s00-raw"]["crowdsecurity/cef-logs"][2].Evt.Whitelisted == false
results["s00-raw"]["crowdsecurity/cef-logs"][3].Success == true
results["s00-raw"]["crowdsecurity/cef-logs"][3].Evt.Parsed["cef_device_product"] == "UniFi Network"
results["s00-raw"]["crowdsecurity/cef-logs"][3].Evt.Parsed["cef_device_vendor"] == "Ubiquiti"
results["s00-raw"]["crowdsecurity/cef-logs"][3].Evt.Parsed["cef_device_version"] == "9.4.19"
results["s00-raw"]["crowdsecurity/cef-logs"][3].Evt.Parsed["cef_event_name"] == "Threat Detected and Blocked"
results["s00-raw"]["crowdsecurity/cef-logs"][3].Evt.Parsed["cef_severity"] == "9"
results["s00-raw"]["crowdsecurity/cef-logs"][3].Evt.Parsed["cef_signature_id"] == "201"
results["s00-raw"]["crowdsecurity/cef-logs"][3].Evt.Parsed["cef_version"] == "0"
results["s00-raw"]["crowdsecurity/cef-logs"][3].Evt.Parsed["hostname"] == "UDM-Gent"
results["s00-raw"]["crowdsecurity/cef-logs"][3].Evt.Parsed["logsource"] == "cef"
results["s00-raw"]["crowdsecurity/cef-logs"][3].Evt.Parsed["message"] == "proto=TCP src=192.168.100.252 spt=65020 dst=192.168.1.100 dpt=80 UNIFIcategory=Security UNIFIsubCategory=Intrusion Prevention UNIFIhost=UDM-Gent UNIFIsite=UDM-Gent UNIFIdeviceMac=78:45:58:de:fc:e7 UNIFIdeviceName=UDM-Gent UNIFIdeviceModel=UDM UNIFIdeviceIp=192.168.100.1 UNIFIdeviceVersion=4.4.0 UNIFIrisk=high UNIFIipsSessionId=1328559940562927 UNIFIipsSignature=ET USER_AGENTS Suspicious User Agent (BlackSun) UNIFIipsSignatureId=2008983 UNIFIutcTime=2025-09-08T06:32:20.613Z msg=A network intrusion attempt from Desktop d5:5d to 192.168.1.100 has been detected and blocked."
results["s00-raw"]["crowdsecurity/cef-logs"][3].Evt.Parsed["program"] == "Ubiquiti"
results["s00-raw"]["crowdsecurity/cef-logs"][3].Evt.Parsed["timestamp"] == "Sep 8 08:32:20"
basename(results["s00-raw"]["crowdsecurity/cef-logs"][3].Evt.Meta["datasource_path"]) == "cef-logs.log"
results["s00-raw"]["crowdsecurity/cef-logs"][3].Evt.Meta["datasource_type"] == "file"
results["s00-raw"]["crowdsecurity/cef-logs"][3].Evt.Whitelisted == false
len(results["success"][""]) == 0
2 changes: 2 additions & 0 deletions .tests/unifi-cef/cef-logs.log
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
CEF:0|Ubiquiti|UniFi Network|9.4.19|544|Admin Accessed UniFi Network|1|UNIFIcategory=System UNIFIsubCategory=Admin UNIFIhost=Unifi Dream Machine UNIFIaccessMethod=web UNIFIadmin=Secure Admin src=10.72.1.222 UNIFIutcTime=2025-09-04T08:32:58.445Z msg=Secure Admin accessed UniFi Network using the web. Source IP: 10.72.1.222
0|Ubiquiti|UniFi Network|9.4.19|201|Threat Detected and Blocked|7|proto=TCP src=10.0.0.100 spt=52331 dst=192.168.0.233 dpt=443 UNIFIcategory=Security UNIFIsubCategory=Intrusion Prevention UNIFIhost=Express 7 UNIFIdeviceMac=84:78:48:80:0d:86 UNIFIdeviceName=Express 7 UNIFIdeviceModel=UX7 UNIFIdeviceIp=192.168.0.1 UNIFIdeviceVersion=4.3.9 UNIFIrisk=medium UNIFIipsSessionId=54725290909450 UNIFIipsSignature=ET DROP Dshield Block Listed Source group 1 UNIFIipsSignatureId=2402000 UNIFIutcTime=2025-08-30T17:53:21.915Z msg=A network intrusion attempt has been detected and blocked.
12 changes: 12 additions & 0 deletions .tests/unifi-cef/config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
parsers:
- ./parsers/s00-raw/crowdsecurity/cef-logs.yaml
- ./parsers/s01-parse/crowdsecurity/unifi-cef.yaml
- crowdsecurity/dateparse-enrich
scenarios:
- ""
postoverflows:
- ""
collections:
- ""
log_file: cef-logs.log
log_type: cef
Loading