Skip to content

[Pfsense] The fix changes the grok pattern for pfSense filterlog events from %{WORD:rule.id} to %{DATA:rule.id} in the firewall ingest pipeline.#19027

Merged
haetamoudi merged 2 commits into
elastic:mainfrom
haetamoudi:sentinel-mirror-pfsense-run-1778774717596297984
May 15, 2026
Merged

[Pfsense] The fix changes the grok pattern for pfSense filterlog events from %{WORD:rule.id} to %{DATA:rule.id} in the firewall ingest pipeline.#19027
haetamoudi merged 2 commits into
elastic:mainfrom
haetamoudi:sentinel-mirror-pfsense-run-1778774717596297984

Conversation

@haetamoudi
Copy link
Copy Markdown
Contributor

@haetamoudi haetamoudi commented May 15, 2026

Suggested label: Enhancement (pick the matching GitHub label)

Proposed commit message

Squash subject line (from Sentinel / primary PR):

[Pfsense] The fix changes the grok pattern for pfSense filterlog events from `%{WORD:rule.id}` to `%{DATA:rule.id}` in the firewall ingest pipeline.

Executive summary

The fix changes the grok pattern for pfSense filterlog events from %{WORD:rule.id} to %{DATA:rule.id} in the firewall ingest pipeline. %{WORD} only matches alphanumeric characters and underscores, which caused it to fail on UUID-format tracker IDs containing hyphens (e.g., 89a1d5c1-2b3e-4f67-8a9b-0c1d2e3f4a5b). The interface name field was also updated from %{WORD} to %{DATA} for consistency, and a new test case was added to validate UUID-format rule IDs. Test timestamps across all expected files were also updated from 2025 to 2026.

Root cause

The PF_LOG_DATA grok pattern uses %{WORD:rule.id}, which only matches [a-zA-Z0-9_]+. pfSense can generate UUID-format tracker IDs containing hyphens (e.g. 89a1d5c1-2b3e-4f67-8a9b-0c1d2e3f4a5b), causing grok to fail for those events.

Approach

Replace %{WORD:rule.id} with %{DATA:rule.id} in the PF_LOG_DATA pattern definition within firewall.yml. The DATA pattern matches any characters up to the next delimiter (comma), covering both numeric tracker IDs (e.g. 1535324496) and UUID-format trackers (e.g. 89a1d5c1-2b3e-4f67-8a9b-0c1d2e3f4a5b). Add a new test fixture line with a UUID tracker and update the expected output. Bump the changelog to 1.25.3 as a bugfix.

Pipeline changes

  • Modify the PF_LOG_DATA pattern in the grok processor (tag: grok_message_518a3bd8) in firewall.yml: change %{WORD:rule.id} to %{DATA:rule.id} to match hyphen-containing UUID tracker IDs in addition to purely alphanumeric tracker IDs.

Field / mapping changes

Sanitized log (event_sanitized excerpt)

<134>May  8 14:04:06 host-1.example.local filterlog[69300]: 52,,,89a1d5c1-2b3e-4f67-8a9b-0c1d2e3f4a5b,igc2,match,pass,in,4,0x0,,64,9146,0,none,17,udp,94,198.51.100.10,198.51.100.1,42590,53,74

Reviewer concerns

• Changing observer.ingress.interface.name from %{WORD} to %{DATA} is broader than necessary — while interface names with hyphens (e.g., igb1.12) already worked because they're captured before the comma delimiter, using %{DATA} is greedy and relies on the comma delimiter to stop matching, which is correct here but worth noting.
• The %{DATA:rule.id} pattern is also comma-delimited, so it will correctly stop at the next comma — no risk of over-capture in practice.
• Timestamp year bumps (2025→2026) across all expected files suggest these are time-sensitive fixtures that will need updating again next year; consider using year-independent test data in the future.
• No concerns about the pipeline logic itself — the fix is minimal and targeted.

Risk and classification

  • Plan risk level: low
  • Tags: pipeline, test-fixture, processors, ingest
  • Impact: medium

Links

  • Issue: (no issue number)
  • Issue title: pfsense.log [MISSING_CASE]: Processor 'grok' with tag 'grok_message_518a3bd8' in pipeline 'logs-pfse…
  • Pipeline case: 5d7b209a26b90578 (Integration Sentinel)

This pull request was opened from the Integration Sentinel agent fix flow.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

  • [ ]

How to test this PR locally

Add a new log line with a UUID-format tracker to test-pfsense-bsd.log and add the corresponding expected output entry to test-pfsense-bsd.log-expected.json. Run 'elastic-package test pipeline -d log' to validate all test fixtures pass, including the new UUID-tracker case and all existing numeric-tracker cases.
From a checkout of this branch:

cd packages/pfsense
elastic-package test

Screenshots

None attached from Sentinel — add if applicable.

…ts from `%{WORD:rule.id}` to `%{DATA:rule.id}` in the firewall ingest pipeline.
@haetamoudi haetamoudi added enhancement New feature or request Integration:pfsense pfSense (Community supported) Team:Integration-Experience Security Integrations Integration Experience [elastic/integration-experience] labels May 15, 2026
@elastic-vault-github-plugin-prod
Copy link
Copy Markdown

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

@elasticmachine
Copy link
Copy Markdown

💚 Build Succeeded

@haetamoudi haetamoudi added the source:integration_sentinel The PR was created via the Integration Sentinel pipeline label May 15, 2026
@haetamoudi haetamoudi marked this pull request as ready for review May 15, 2026 12:26
@haetamoudi haetamoudi requested a review from a team as a code owner May 15, 2026 12:26
@infra-vault-gh-plugin-prod
Copy link
Copy Markdown

Pinging @elastic/integration-experience (Team:Integration-Experience)

@haetamoudi haetamoudi merged commit 801e8d1 into elastic:main May 15, 2026
8 checks passed
@elastic-vault-github-plugin-prod
Copy link
Copy Markdown

Package pfsense - 1.25.3 containing this change is available at https://epr.elastic.co/package/pfsense/1.25.3/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request Integration:pfsense pfSense (Community supported) source:integration_sentinel The PR was created via the Integration Sentinel pipeline Team:Integration-Experience Security Integrations Integration Experience [elastic/integration-experience]

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants