Skip to content

Commit

Permalink
feat: source port config validation
Browse files Browse the repository at this point in the history
  • Loading branch information
ikheifets-splunk committed May 6, 2024
1 parent 3109c3e commit 5ef0b72
Show file tree
Hide file tree
Showing 5 changed files with 60 additions and 0 deletions.
14 changes: 14 additions & 0 deletions docs/troubleshooting/troubleshoot_SC4S_server.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,20 @@ just _one_ bad index will "taint" the entire batch (in this case, 1000 events) a
imperative that the container logs be free of these kinds of errors in production._ You can use the alternate HEC debug destination (below)
to help debug this condition by sending direct "curl" commands to the HEC endpoint outside of the SC4S setting.

## Invalid SC4S listen ports

SC4S can exclusively grant some port to device by using environment var `SC4S_LISTEN_{vendor}_{product}_{TCP/UDP/TLS}_PORT={port}`.

During startup SC4S validating that listen ports configured correctly and in case of constraint error you will see it on logs.

Example of error messages when listen ports for `MERAKI SWITCHES` configured incorrectly:

```
SC4S_LISTEN_MERAKI_SWITCHES_TCP_PORT: Impossible to use default ports like 514
SC4S_LISTEN_MERAKI_SWITCHES_UDP_PORT: 7000 already used in another source, use unique port
SC4S_LISTEN_MERAKI_SWITCHES_TLS_PORT: 999999999999 should be valid int [0, 10000]
```

## SC4S Local Disk Resource Considerations
* Check the HEC connection to Splunk. If the connection is down for a long period of time, the local disk buffer used for backup will exhaust local
disk resources. The size of the local disk buffer is configured in the env_file: [Disk buffer configuration](https://splunk-connect-for-syslog.readthedocs.io/en/latest/configuration/#disk-buffer-variables)
Expand Down
1 change: 1 addition & 0 deletions package/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,7 @@ COPY package/etc/local_config /etc/syslog-ng/local_config
COPY package/etc/local_config /etc/syslog-ng/local_config
COPY package/sbin/entrypoint.sh /
COPY package/sbin/healthcheck.sh /
COPY package/sbin/source_ports_validator.py /

ENV SC4S_CONTAINER_OPTS=--no-caps
ARG VERSION=unknown
Expand Down
1 change: 1 addition & 0 deletions package/Dockerfile.lite
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,7 @@ COPY package/lite/etc/addons /etc/syslog-ng/addons

COPY package/sbin/entrypoint.sh /
COPY package/sbin/healthcheck.sh /
COPY package/sbin/source_ports_validator.py /


RUN chmod -R 755 /etc/syslog-ng/
Expand Down
2 changes: 2 additions & 0 deletions package/sbin/entrypoint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -201,6 +201,8 @@ fi
export SOURCE_SIMPLE_SET=$(printenv | grep '^SC4S_LISTEN_SIMPLE_.*_PORT=.' | sed 's/^SC4S_LISTEN_SIMPLE_//;s/_..._PORT\=.*//;s/_[^_]*_PORT\=.*//' | sort | uniq | xargs echo | sed 's/ /,/g' | tr '[:upper:]' '[:lower:]' )
export SOURCE_ALL_SET=$(printenv | grep '^SC4S_LISTEN_.*_PORT=.' | grep -v "disabled" | sed 's/^SC4S_LISTEN_//;s/_..._PORT\=.*//;s/_[^_]*_PORT\=.*//' | sort | uniq | xargs echo | sed 's/ /,/g' | tr '[:lower:]' '[:upper:]' )

python3 /source_ports_validator.py

syslog-ng --no-caps --preprocess-into=- | grep vendor_product | grep set | grep -v 'set(.\$' | sed 's/^ *//' | grep 'value("fields.sc4s_vendor_product"' | grep -v "\`vendor_product\`" | sed s/^set\(// | cut -d',' -f1 | sed 's/\"//g' >/tmp/keys
syslog-ng --no-caps --preprocess-into=- | grep 'meta_key(.' | sed 's/^ *meta_key(.//' | sed "s/')//" >>/tmp/keys
rm -f $SC4S_ETC/conf.d/local/context/splunk_metadata.csv.example >/dev/null || true
Expand Down
42 changes: 42 additions & 0 deletions package/sbin/source_ports_validator.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
import os
import logging


logger = logging.getLogger(__name__)


def is_valid_port(raw_port: str) -> bool:
return raw_port.isdigit() and int(raw_port) < 10000


def validate_source_ports(sources: list[str]) -> None:
source_ports = []
for source in sources:
tcp_ports = os.getenv(f"SC4S_LISTEN_{source}_TCP_PORT", "disabled").split(",")
udp_ports = os.getenv(f"SC4S_LISTEN_{source}_UDP_PORT", "disabled").split(",")
tls_ports = os.getenv(f"SC4S_LISTEN_{source}_TLS_PORT", "disabled").split(",")

source_ports.extend((source, port, "TCP") for port in tcp_ports)
source_ports.extend((source, port, "UDP") for port in udp_ports)
source_ports.extend((source, port, "TLS") for port in tls_ports)


busy_ports = set()
for source, port, proto in source_ports:
env_var = f"SC4S_LISTEN_{source}_{port}_PORT"

if port in ["disabled", ""]:
continue
elif not is_valid_port(port):
logger.error(f"{env_var}: {port} should be valid int [0, 10000]")
elif source != "DEFAULT" and port in ["514", "614", "6514"]:
logger.error(f"{env_var}: Impossible to use default ports like {port}")
elif (port, proto) in busy_ports:
logger.error(f"{env_var}: {port} already used in another source, use unique port")
else:
busy_ports.add((port, proto))


if __name__ == "__main__":
sources = os.getenv("SOURCE_ALL_SET").split(",")
validate_source_ports(sources)

0 comments on commit 5ef0b72

Please sign in to comment.