Add Suricata Eve logging to Telegraf

[tiny-pangolin]

Important notices
Before you add a new report, we ask you kindly to acknowledge the following:

[x] I have read the contributing guide lines at https://github.com/opnsense/plugins/blob/master/CONTRIBUTING.md

[x] I have searched the existing issues and I'm convinced that mine is new.

[x] When the request is meant for an existing plugin, I've added its name to the title.

Is your feature request related to a problem? Please describe.
I am not able to send the contents of /var/log/suricata/eve.json to influxdb via telegraf
Describe the solution you'd like
I would like to be able to use the tail input plugin using the sample config below

[[inputs.tail]]
  data_format = "json"
  files = ["/var/log/suricata/eve.json"]
  name_override = "suricata"               
  tag_keys = ["event_type","src_ip","src_port","dest_ip","dest_port"]
  json_string_fields = ["*"]

Describe alternatives you've considered
I have considered use the syslog plugin but that is not exposed in the gui and would require a multiple step pipeline

Additional context
I am building an opensource RMM and would like to display suricata alerts in a monitoring dashboards

[mimugmail]

Does it work when you add it manually to the file and restart the service via CLI?

[tiny-pangolin]

Yes it does Mar 13, 2021 12:51:43 Michael ***@***.***>:

…

Does it work when you add it manually to the file and restart the service via CLI? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub[#2239 (comment)], or unsubscribe[https://github.com/notifications/unsubscribe-auth/ADFKHBFKZZYDWJ4BHEHWUP3TDOX3PANCNFSM4XTPTPGA]. [data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAEIAAABCCAYAAADjVADoAAAAAXNSR0IArs4c6QAAAARzQklUCAgICHwIZIgAAAAnSURBVHic7cEBDQAAAMKg909tDwcUAAAAAAAAAAAAAAAAAAAAAPBjRFIAASHKmKkAAAAASUVORK5CYII=###24x24:true###][Tracking image][https://github.com/notifications/beacon/ADFKHBEXI4YSAQGFT4W6JL3TDOX3PA5CNFSM4XTPTPGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOF6ODMOI.gif]

[tiny-pangolin]

I have been trying to work on this myself and I can get it to show up in the ui but it doesn't generate the correct telegraf config Below are the changes I made

I added this to /usr/local/opnsense/service/templates/OPNsense/Telegraf/telegraf.conf

{% if helpers.exists('OPNsense.telegraf.input.intrusion_detection_alerts') and OPNsense.telegraf.input.intrusion_detection_alerts == '1' %}
[[inputs.tail]]
  data_format = "json"
  files = ["/var/log/suricata/eve.json"]
  name_override = "suricata"               
  tag_keys = ["event_type","src_ip","src_port","dest_ip","dest_port"]
  json_string_fields = ["*"]
{% endif %}

and the following to /usr/local/opnsense/mvc/app/controllers/OPNsense/Telegraf/forms/input.xml

    <field>
        <id>input.intrusion_detection_alerts</id>
        <label>Intrusion Dectection Alerts</label>
        <type>checkbox</type>
        <help>Requires Intrustion detection alerts are stored locally </help>
   </field> 

[tiny-pangolin]

figured out I needed to edit /usr/local/opnsense/mvc/app/models/OPNsense/Telegraf/input.xml and it worked. I have submitted a pull request

        <intrusion_detection_alerts type="BooleanField">
            <default>0</default>
            <Required>N</Required>
        </intrusion_detection_alerts>

[psychogun]

Hey @tiny6996 - how did you get this to work? The user telegraf does not have permissions to access the directory /var/log/suricata, let alone the file /var/log/suricata/eve.json.

sudo -u telegraf more /var/log/suricata/eve.json
/var/log/suricata/eve.json: Permission denied

This permission issue is not showing up in the logs if you tail -f /var/log/telegraf/telegraf.logeither, it just looks like the telegraf service starts just fine.

This is my workaround:
Default permissions on the folder /var/log/suricata is rwx------, with user root as the owner and wheel as the group owner.

Let us add telegraf to the wheel group:

root@opnsense:/usr/local/etc/suricata # pw group mod wheel -m telegraf
root@opnsense:/usr/local/etc/suricata # pw groupshow wheel
wheel:*:0:root,telegraf

Change permissions on the folder and the file eve.json:

chmod 750 /var/log/suricata
chmod 750 /var/log/suricata/eve.json

Now running sudo -u telegraf more /var/log/suricata/eve.json - you will see that the user is able to view the file, thus the contents of eve.json can be sent to InfluxDBv2 via Telegraf.

However, this will not survive a reboot!

---

And another thing - https://docs.influxdata.com/telegraf/v1.19/data_formats/input/json/

By default the current time will be used for all created metrics, to set the time using the JSON document you can use the json_time_key and json_time_format options together to set the time to a value in the parsed document.

The json_time_key option specifies the key containing the time value and json_time_format must be set to unix, unix_ms, or the Go “reference time” which is defined to be the specific time: Mon Jan 2 15:04:05 MST 2006.

I would very much like the [[inputs.tail]] plugin to produce as much metrics as possible:

[[inputs.tail]]
  data_format = "json"
  files = ["/var/log/suricata/eve.json"]
  name_override = "suricata-alerts"
  tag_keys = ["flow_id","in_iface","event_type","src_ip","src_port","dest_ip","dest_port","proto"]
  json_string_fields = ["*"]
  json_time_key = "timestamp"
  json_time_format = "2006-01-02T15:04:05-0700" 

[psychogun]

After doing the above, disabling / enabling Intrusion Detection, the permissions on the folder / file resets.

Then this error showed up in /var/log/telegraf/telegraf.log:

2021-07-28T19:24:29Z I! Starting Telegraf 1.19.0
2021/07/28 21:42:26 FATAL -- Failed to stat file /var/log/suricata/eve.json: stat /var/log/suricata/eve.json: permission denied
goroutine 7 [running]:
runtime/debug.Stack(0xc000708180, 0x23, 0xc000d62f98)
	runtime/debug/stack.go:24 +0x9f
github.com/influxdata/tail/util.Fatal(0xfd8c69, 0x1a, 0xc000d62f98, 0x2, 0x2)
	github.com/influxdata/tail/util/util.go:22 +0xc7
github.com/influxdata/tail/watch.(*InotifyFileWatcher).ChangeEvents.func1(0xc000b3b980, 0xc000ab3b50, 0xc000c26018)
	github.com/influxdata/tail/watch/inotify.go:120 +0x305
created by github.com/influxdata/tail/watch.(*InotifyFileWatcher).ChangeEvents
	github.com/influxdata/tail/watch/inotify.go:77 +0x16f

I found out a reboot removes the user telegraf from the group wheel.

[mimugmail]

I will add a PR which lets telegraf run as run and it works with the existing checkbox:

suricata,dest_ip=165.227.139.114,dest_port=37744,event_type=anomaly,host=HQ-test,oink=hui,path=/var/log/suricata/eve.json,src_ip=172.24.68.133,src_port=25,test=va1
suricata,dest_ip=172.24.68.133,dest_port=12974,event_type=alert,host=HQ-test,oink=hui,path=/var/log/suricata/eve.json,src_ip=185.175.91.7,src_port=53,test=va1
suricata,dest_ip=172.24.68.133,dest_port=13996,event_type=alert,host=HQ-test,oink=hui,path=/var/log/suricata/eve.json,src_ip=185.175.91.8,src_port=53,test=va1
suricata,dest_ip=172.24.68.133,dest_port=19375,event_type=alert,host=HQ-test,oink=hui,path=/var/log/suricata/eve.json,src_ip=185.21.103.31,src_port=53,test=va1
suricata,dest_ip=172.24.68.133,dest_port=24075,event_type=alert,host=HQ-test,oink=hui,path=/var/log/suricata/eve.json,src_ip=185.136.96.111,src_port=53,test=va1
suricata,dest_ip=172.24.68.133,dest_port=43500,event_type=alert,host=HQ-test,oink=hui,path=/var/log/suricata/eve.json,src_ip=185.136.99.111,src_port=53,test=va1
suricata,dest_ip=172.24.68.133,dest_port=43734,event_type=alert,host=HQ-test,oink=hui,path=/var/log/suricata/eve.json,src_ip=185.175.91.5,src_port=53,test=va1
suricata,dest_ip=172.24.68.133,dest_port=56915,event_type=alert,host=HQ-test,oink=hui,path=/var/log/suricata/eve.json,src_ip=185.175.91.6,src_port=53,test=va1
suricata,dest_ip=172.24.68.133,event_type=alert,host=HQ-test,oink=hui,path=/var/log/suricata/eve.json,src_ip=1.1.1.1,test=va1
suricata,dest_ip=172.24.68.133,event_type=alert,host=HQ-test,oink=hui,path=/var/log/suricata/eve.json,src_ip=8.8.4.4,test=va1
suricata,dest_ip=172.24.68.133,event_type=alert,host=HQ-test,oink=hui,path=/var/log/suricata/eve.json,src_ip=81.24.66.1,test=va1
suricata,dest_ip=172.24.68.191,dest_port=137,event_type=alert,host=HQ-test,oink=hui,path=/var/log/suricata/eve.json,src_ip=172.24.68.171,src_port=137,test=va1
suricata,dest_ip=172.24.68.191,dest_port=137,event_type=alert,host=HQ-test,oink=hui,path=/var/log/suricata/eve.json,src_ip=172.24.68.173,src_port=55742,test=va1