Firewall dashboard issue - "Adjust autocomplete settings in kibana.yml" - "Network trasport field missing in *-pfelk-firewall*" - "20-interfaces.pfelk did not apply the aliases"

[kozistan]

Hi, having some issues firewall dashboard and 20-interfaces.pfelk is not applying the aliases field into elasticsearch.

Adjust autocomplete settings in kibana.yml is shown up at the dashboard with Interface and firewall selector.

Tried to add line into kibana.yml kibana.autocompleteTerminateAfter: 1000000 after some research but with no luck. Maybe some advise here?

Regarding Network transport protocol here is the error screenshot, in short there is missing field:

Regarding the 20-interfaces.pfelk do not know where the problem could be, here is my config:

# 20-interfaces.pfelk
################################################################################
# Version: 22.04                                                               #
# Required: No - Optional                                                      #
# Description: Adds interface.alias and network.name based on interface.name   #
# The interface.alias and network.name fields may be amended as desired        #
################################################################################
#
### firewall-1 ###
filter {
### Change first.network.local to pfSesne or OPNsense host name ###
  if [host][name] == "blabla.bla" {
    ### Change interface as desired ###
    if [interface][name] =~ /^vtnet0$/ {
      mutate {
        add_field => { "[interface][alias]" => "WAN" }
        add_field => { "[network][name]" => "FiOS" }
      }
    }
    ### Change interface as desired ###
    if [interface][name] =~ /^vlan0.1110$/ {
      mutate {
        add_field => { "[interface][alias]" => "MNGMNT" }
        add_field => { "[network][name]" => "vlan1110" }
      }
    }
    ### Change interface as desired ###
    if [interface][name] =~ /^vlan0.1010$/ {
      mutate {
        add_field => { "[interface][alias]" => "MARKETING" }
        add_field => { "[network][name]" => "vlan1010" }
      }
    }
    ### Change interface as desired ###
    if [interface][name] =~ /^vlan0.1020$/ {
      mutate {
        add_field => { "[interface][alias]" => "SALES" }
        add_field => { "[network][name]" => "vlan1020" }
      }
    }
    ### Change interface as desired ###
    if [interface][name] =~ /^vlan0.1030$/ {
      mutate {
        add_field => { "[interface][alias]" => "ACCOUNT" }
        add_field => { "[network][name]" => "vlan1030" }
      }
    }
    ### Change interface as desired ###
    if [interface][name] =~ /^vlan0.1040$/ {
      mutate {
        add_field => { "[interface][alias]" => "HEAD" }
        add_field => { "[network][name]" => "vlan1040" }
      }
    }
    ### Change interface as desired ###
    if [interface][name] =~ /^vlan0.1050$/ {
      mutate {
        add_field => { "[interface][alias]" => "LEGAL" }
        add_field => { "[network][name]" => "vlan1050" }
      }
    }
    ### Change interface as desired ###
    if [interface][name] =~ /^vlan0.1060$/ {
      mutate {
        add_field => { "[interface][alias]" => "FINANCE" }
        add_field => { "[network][name]" => "vlan1060" }
      }
    }
    ### Change interface as desired ###
    if [interface][name] =~ /^vlan0.1070$/ {
      mutate {
        add_field => { "[interface][alias]" => "OFFICE" }
        add_field => { "[network][name]" => "vlan1070" }
      }
    }
    ### Change interface as desired ###
    if [interface][name] =~ /^vlan0.1080$/ {
      mutate {
        add_field => { "[interface][alias]" => "VENDOR" }
        add_field => { "[network][name]" => "vlan1080" }
      }
    }
    ### Change interface as desired ###
    if [interface][name] =~ /^vlan0.1200$/ {
      mutate {
        add_field => { "[interface][alias]" => "GUEST" }
        add_field => { "[network][name]" => "vlan1200" }
      }
    }
    ### Change interface as desired ###
    if [interface][name] =~ /^vlan0.1500$/ {
      mutate {
        add_field => { "[interface][alias]" => "EQUIPMENT" }
        add_field => { "[network][name]" => "vlan1500" }
      }
    }
    ### Change interface as desired ###
    if [interface][name] =~ /^vlan0.1600$/ {
      mutate {
        add_field => { "[interface][alias]" => "CCTV" }
        add_field => { "[network][name]" => "vlan1600" }
      }
    }
    ### Change interface as desired ###
    if [interface][name] =~ /^wg4$/ {
      mutate {
        add_field => { "[interface][alias]" => "PROXY09" }
        add_field => { "[network][name]" => "coProxy09" }
      }
    }
    ### Change interface as desired ###
    if [interface][name] =~ /^wg3$/ {
      mutate {
        add_field => { "[interface][alias]" => "PROXY11" }
        add_field => { "[network][name]" => "coProxy11" }
      }
    }
    ### Change interface as desired ###
    if [interface][name] =~ /^lo0$/ {
      mutate {
        add_field => { "[interface][alias]" => "Link-Local" }
        update => { "[network][direction]" => "%{[network][direction]}bound" }
        update => { "[network][type]" => "ipv%{[network][type]}" }
      }
    }
    ### Fallback interface ###
    if ![interface][alias] and [interface][name] {
        mutate {
          add_field => { "[interface][alias]" => "%{[interface][name]}" }
          add_field => { "[network][name]" => "%{[interface][name]}" }
      }
    }
  }
}

I'm running latest opnsense 23.7.4 on FreeBSD 13.2-RELEASE-p3 amd64 as VM on Proxmox 8.0.4 and latest elasticsearch/lkibana and logstash versions 8.10

Thanks in advance for any help/suggestions.

[a3ilson]

1. 20-interfaces.pfelk

• The issue regarding the 20-interfaces.pfelk
  • The referenced file is outdated and references the host by [host][name] updating will spend that field to read [log][syslog][hostname]
  • The initial filer (01-inputs.pfelk) enriches the syslog message via the pfelk.grok pattern
    • pfelk.grok pattern: [event][created], [log][syslog][hostname], [log][syslog][appname], and [log][syslog][procid] are created.
  • The 20-interfaces.pfelk (optional) adds additional fields based on the [log][syslog][hostname] field value but requires minor configuration (e.g., [log][syslog][hostname] and [interface][name])

2. Network Transport

• More context is needed...the [network][transport] is utilized within the pfelk.grok pattern file but is associated to Snort, Suricate, OpenVPN and the depreciated PF Grok patterns for which (02-firewall.pfelk) filters based on comma (structured PF logging).

[kozistan]

Thanks a lot helping me with that, actually what i did is replaced the [host][name] for [log][syslog][hostname] and everything started to work with 20-interfaces.pfelk, the rest additional fields are parsed.

Regarding [network][transport] i've replaced the field with [network][protocol] and is enough for me to have separated protocols as output.

I'm not using Snort and Suticata, well and OpnVPN is another part of what i would have fixed :) Have seen ad add a comment to closed issue, where you've been trying to fix with no luck.
There is a comment from my in that issue, so would appreciate to continue with that if you do not mind, OpenVPN is a often used protocol and regarding the right logging would be nice to have it on.

As you're not interacting with the Kibana autocomplete issue i understand that it just not belong here, just thought you've been dealing with that too, if not just forget about it.

Thanks again for your participation @613377

[a3ilson]

Glad to hear.

1. OpenVPN has been a long overdue item and is listed within this project. The piece I'm struggling with is the output format of OpenVPN which is unfortunately a bit messy (too many variations) coupled with the lack of time to dedicate. I'll get to it eventually...

2. Feel free to close this issue out and open another for the Kibana Autocomplete...I'll look into that one as well.