Promtail does not work with Syslog via UDP without line break

[pstrobl96]

Hello I occurred an issue while using Promtail / Grafana Agent. I need to use Syslog via UDP. RFC5424 is used however Promtail is unable to process logs.

When I was sending logs to Promtail I was getting no error log at all. Then I tried Grafana Agent and Grafana Agent in flow mode - I was experimenting with Agent so this was an excuse why to try flow mode. But flow mode logs very useful information

agent | ts=2024-03-28T15:30:34.892686659Z level=warn msg="error parsing syslog stream" component=loki.source.syslog.logs_prusa_syslog err="unexpected EOF"

This led to investigation of log source and it was discovered that printers do not sent any line break. However in RFC5424 does not specify line break. When it's added then it works but without it does not.

Source of the logs is Prusa-Firmware-Buddy and I'm using logs in prusa_exporter - I need to process logs via code and Promtail scrapes file right now. I would prefer use only Promtail.

Configuration file

logs:
  positions_directory: /var/lib/grafana-agent
  configs:
  - name: prusa
    clients:
    - url: http://loki:3100/loki/api/v1/push
    scrape_configs:
    - job_name: syslog
      syslog:
        listen_address: 0.0.0.0:10007
        listen_protocol: udp
        idle_timeout: 120
        label_structured_data: true
        use_incoming_timestamp: false
        labels:
          job: "syslog-buddy"
          board: "buddy"
      relabel_configs:
        - source_labels: ['__syslog_message_hostname']
          target_label: 'mac_address'
        - source_labels: ['__syslog_message_app_name']
          target_label: 'app'
        - source_labels: ['__syslog_connection_ip_address']
          target_label: 'ip'

[lukash]

To further clarify, promtail expects a line break at the end of the message, but that doesn't seem to be mentioned in the RFC, the more relevant one is perhaps RFC 5426 for transporting syslog via UDP. It says the transfer MUST be done one record per datagram and doesn't mention a line break at the end.

[rarrr]

I'm getting the same thing when trying to send pfsense logs to grafana agent, pfsense is sending RFC 5424 logs but without a line ending resulting in the same error. If I manually send the log using netcat and put a line ending it all gets digested nicely.