Machine Learning model for attack pattern on network logs

[sajjadbinraheem]

Operating System

Windows

Version Information

Current situation: I have Network log (FortiAnalyzer log) data and endpoint log (SentinelOne)

sample logs for fortianalyzer

 <189>logver=700130566 timestamp=1705663428 devname="F" devid=6 vd=root date=2024-01-19 time=11:23:48 eventtime=1705659828815774000 tz=+0100 logid="0000000013" type=traffic subtype=forward level=notice srcip=40 srcport=42489 srcintf=Server-50 srcintfrole=lan dstip=********236 dstport=514 dstintf=wan2 dstintfrole=wan srcuuid=0e9b-51eb-cc6d-9f3b180f91fc dstuuid=8da***********ea-d8f0-c1ac2598a319 srccountry=Reserved dstcountry=Netherlands sessionid=568400008 proto=17 action=deny policyid=61 policytype=policy poluuid=9ec6b-e64e-7eee61e47eab policyname=jkhdkfjhk-LAN->iNetCatchAll service=SYSLOG trandisp=noop duration=0 sentbyte=0 rcvdbyte=0 sentpkt=0 rcvdpkt=0 vwlid=3 vwlquality=Seq_num(1 wan2), alive, selected vwlname=LAN-Rule appcat=unscanned crscore=30 craction=131072 crlevel=high srchwvendor=HP devtype=Network srcfamily=Switch srchwversion=Aruba mastersrcmac=d0::cc srcmac=d0:cc srcserver=0 <189>logver=700130566 timestamp=1705663428 devname="FGT0F" devid=FG13696 vd=root date=2024-01-19 time=11:23:48 eventtime=1705659828815830080 tz=+0100 logid="0000000013" type=traffic subtype=forward level=notice srcip=1040 srcport=42489 srcintf=Server-50 srcintfrole=lan dstip=1******.236 dstport=514 dstintf=wan2 dstintfrole=wan srcuuid=0eb*********-cc6d-9f3b180f91fc dstuuid=81ea-d8f0-c1ac2598a319 srccountry=Reserved dstcountry=China sessionid=568400009 proto=17 action=deny policyid=61 policytype=policy poluuid=9eceb-e64e-7eee61e47eab policyname=-LAN->iNetCatchAll service=SYSLOG trandisp=noop duration=0 sentbyte=0 rcvdbyte=0 sentpkt=0 rcvdpkt=0 vwlid=3 vwlquality=Seq_num(1 wan2), alive, selected vwlname=LAN-Rule appcat=unscanned crscore=30 craction=131072 crlevel=high srchwvendor=HP devtype=Network srcfamily=Switch srchwversion=Aruba mastersrcmac=d0:69:e8:cc srcmac=d0:6***:e8:cc srcserver=0

sample logs of sentinel one

<12>2023-11-07 12:34:45,230 sentinel - CEF:0|SentinelOne|Mgmt|Y#20|19|New active threat - machine W023|1|osName=Windows 10 Enterprise rt=2023-11-07 12:30:36.456951 fileHash=3395856c**642f14140 filePath=\Device\HarddiskVolume3\TEMP\virus.txt cat=SystemEvent activityID=1813041897368528281 activityType=19 siteId=1080466588931492808 siteName=Technology . accountId=10815591 accountName= Technology . notificationScope=SITE <14>2023-11-07 12:34:45,231 sentinel - CEF:0|SentinelOne|Mgmt|Y#20|2001|Kill performed successfully|1|fileHash=3395ee72602f798b642f14140 filePath=\Device\HarddiskVolume3\TEMP\virus.txt osName=Windows 10 Enterprise ip=2194 cat=SystemEvent suser=WP23 rt=#arcsightDate(Tue, 07 Nov 2023, 12:30:36 UTC) activityID=181305492937 activityType=2001 siteId=10804661492808 siteName= Technology . accountId=108046591 accountName= Technology . notificationScope=SITE <14>2023-11-07 12:34:45,231 sentinel - CEF:0|SentinelOne|Mgmt|Y#20|2004|Quarantine performed successfully|1|fileHash=332b7382dee72602f798b642f14140 filePath=\Device\HarddiskVolume3\TEMP\virus.txt osName=Windows 10 Enterprise ip=2194 cat=SystemEvent suser=WP6023 rt=#arcsightDate(Tue, 07 Nov 2023, 12:30:36 UTC) activityID=1813045002 activityType=2004 siteId=10804665808 siteName= Technology . accountId=108015591 accountName= Technology . notificationScope=SITE <12>2023-11-07 12:34:45,415 sentinel - CEF:0|SentinelOne|Mgmt|Y#20|19|New active threat - machine WP*******3|1|osName=Windows 10 Enterprise rt=2023-11-07 12:30:36.456951 fileHash=3395602f798b642f14140 filePath=\Device\HarddiskVolume3\TEMP\virus.txt cat=SystemEvent activityID=18130428281 activityType=19 siteId=108092808 siteName= Technology . accountId=1080466588914715591 accountName= Technology . notificationScope=SITE

Problem statement: I would like to

1. connect (Stitch) there two logs
2. Apply an ML model to detect the anomalies (for ex: DDOS, breach) on these logs

Summary:
Is there any model on Huggingface.co or other sources using which I can detect the anomalies in the logs?
also, what is the best way to stitch the data from these two logs

Steps to reproduce

Current situation: I have Network log (FortiAnalyzer log) data and endpoint log (SentinelOne)

sample logs for fortianalyzer

 <189>logver=700130566 timestamp=1705663428 devname="F" devid=6 vd=root date=2024-01-19 time=11:23:48 eventtime=1705659828815774000 tz=+0100 logid="0000000013" type=traffic subtype=forward level=notice srcip=40 srcport=42489 srcintf=Server-50 srcintfrole=lan dstip=********236 dstport=514 dstintf=wan2 dstintfrole=wan srcuuid=0e9b-51eb-cc6d-9f3b180f91fc dstuuid=8da***********ea-d8f0-c1ac2598a319 srccountry=Reserved dstcountry=Netherlands sessionid=568400008 proto=17 action=deny policyid=61 policytype=policy poluuid=9ec6b-e64e-7eee61e47eab policyname=jkhdkfjhk-LAN->iNetCatchAll service=SYSLOG trandisp=noop duration=0 sentbyte=0 rcvdbyte=0 sentpkt=0 rcvdpkt=0 vwlid=3 vwlquality=Seq_num(1 wan2), alive, selected vwlname=LAN-Rule appcat=unscanned crscore=30 craction=131072 crlevel=high srchwvendor=HP devtype=Network srcfamily=Switch srchwversion=Aruba mastersrcmac=d0::cc srcmac=d0:cc srcserver=0 <189>logver=700130566 timestamp=1705663428 devname="FGT0F" devid=FG13696 vd=root date=2024-01-19 time=11:23:48 eventtime=1705659828815830080 tz=+0100 logid="0000000013" type=traffic subtype=forward level=notice srcip=1040 srcport=42489 srcintf=Server-50 srcintfrole=lan dstip=1******.236 dstport=514 dstintf=wan2 dstintfrole=wan srcuuid=0eb*********-cc6d-9f3b180f91fc dstuuid=81ea-d8f0-c1ac2598a319 srccountry=Reserved dstcountry=China sessionid=568400009 proto=17 action=deny policyid=61 policytype=policy poluuid=9eceb-e64e-7eee61e47eab policyname=-LAN->iNetCatchAll service=SYSLOG trandisp=noop duration=0 sentbyte=0 rcvdbyte=0 sentpkt=0 rcvdpkt=0 vwlid=3 vwlquality=Seq_num(1 wan2), alive, selected vwlname=LAN-Rule appcat=unscanned crscore=30 craction=131072 crlevel=high srchwvendor=HP devtype=Network srcfamily=Switch srchwversion=Aruba mastersrcmac=d0:69:e8:cc srcmac=d0:6***:e8:cc srcserver=0

sample logs of sentinel one

<12>2023-11-07 12:34:45,230 sentinel - CEF:0|SentinelOne|Mgmt|Y#20|19|New active threat - machine W023|1|osName=Windows 10 Enterprise rt=2023-11-07 12:30:36.456951 fileHash=3395856c**642f14140 filePath=\Device\HarddiskVolume3\TEMP\virus.txt cat=SystemEvent activityID=1813041897368528281 activityType=19 siteId=1080466588931492808 siteName=Technology . accountId=10815591 accountName= Technology . notificationScope=SITE <14>2023-11-07 12:34:45,231 sentinel - CEF:0|SentinelOne|Mgmt|Y#20|2001|Kill performed successfully|1|fileHash=3395ee72602f798b642f14140 filePath=\Device\HarddiskVolume3\TEMP\virus.txt osName=Windows 10 Enterprise ip=2194 cat=SystemEvent suser=WP23 rt=#arcsightDate(Tue, 07 Nov 2023, 12:30:36 UTC) activityID=181305492937 activityType=2001 siteId=10804661492808 siteName= Technology . accountId=108046591 accountName= Technology . notificationScope=SITE <14>2023-11-07 12:34:45,231 sentinel - CEF:0|SentinelOne|Mgmt|Y#20|2004|Quarantine performed successfully|1|fileHash=332b7382dee72602f798b642f14140 filePath=\Device\HarddiskVolume3\TEMP\virus.txt osName=Windows 10 Enterprise ip=2194 cat=SystemEvent suser=WP6023 rt=#arcsightDate(Tue, 07 Nov 2023, 12:30:36 UTC) activityID=1813045002 activityType=2004 siteId=10804665808 siteName= Technology . accountId=108015591 accountName= Technology . notificationScope=SITE <12>2023-11-07 12:34:45,415 sentinel - CEF:0|SentinelOne|Mgmt|Y#20|19|New active threat - machine WP*******3|1|osName=Windows 10 Enterprise rt=2023-11-07 12:30:36.456951 fileHash=3395602f798b642f14140 filePath=\Device\HarddiskVolume3\TEMP\virus.txt cat=SystemEvent activityID=18130428281 activityType=19 siteId=108092808 siteName= Technology . accountId=1080466588914715591 accountName= Technology . notificationScope=SITE

Problem statement: I would like to

1. connect (Stitch) there two logs
2. Apply an ML model to detect the anomalies (for ex: DDOS, breach) on these logs

Summary:
Is there any model on Huggingface.co or other sources using which I can detect the anomalies in the logs?
also, what is the best way to stitch the data from these two logs

Expected behavior

Model should detect the anomalies in the network log

Actual behavior

Model should detect the anomalies in the network log

Addition information

No response