Skip to content

crowdstrike.fdr: Add if conditions and opt-out config for geoip processors#17584

Merged
kcreddy merged 4 commits intoelastic:mainfrom
kcreddy:crowdstrike-fdr-geoip-perf
Feb 27, 2026
Merged

crowdstrike.fdr: Add if conditions and opt-out config for geoip processors#17584
kcreddy merged 4 commits intoelastic:mainfrom
kcreddy:crowdstrike-fdr-geoip-perf

Conversation

@kcreddy
Copy link
Contributor

@kcreddy kcreddy commented Feb 26, 2026 •
edited
Loading

Proposed commit message

crowdstrike.fdr: Add if conditions and opt-out config for geoip processors

Avoid unnecessary geoip processor invocations when IP fields are absent.
Without an "if" guard, each geoip processor is still entered on every
document — triggering a database validity check[1] that traverses cluster 
state metadata on every call — before discovering the field is missing 
and returning early.
Adding "if: ctx.<field> != null" skips the processor entirely, avoiding
this overhead for the majority of documents that lack the relevant IP.

Additionally, expose three new configuration options —
enable_geoip_observer_ip, enable_geoip_source_ip, and
enable_geoip_destination_ip — that allow users to selectively disable
GeoIP enrichment per IP field. All default to true for backward
compatibility.

New test sample log copied from existing sample, but with disabled
new configuration options to verify "geo" fields are not populated.

[1] https://github.com/elastic/elasticsearch/blob/70266af5b6e9bd5a40a918cb34fc5d1c3fe680aa/modules/ingest-geoip/src/main/java/org/elasticsearch/ingest/geoip/GeoIpProcessor.java#L103

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices

Related issues

@kcreddy kcreddy self-assigned this Feb 26, 2026
@kcreddy kcreddy added enhancement New feature or request Integration:crowdstrike CrowdStrike Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations] labels Feb 26, 2026
@kcreddy kcreddy marked this pull request as ready for review February 26, 2026 11:02
@kcreddy kcreddy requested a review from a team as a code owner February 26, 2026 11:02
@elasticmachine
Copy link

elasticmachine commented Feb 26, 2026

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

@kcreddy kcreddy requested a review from efd6 February 26, 2026 11:03
@kcreddy kcreddy mentioned this pull request Feb 26, 2026
Closed
efd6
efd6 reviewed Feb 26, 2026
View reviewed changes
Copy link
Contributor

@efd6 efd6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure that this is a close; there was extension work suggested in the issue to add options to skip the observer.ip (and possibly the others) GeoIP look-up entirely. This needs to be discussed to determine whether we think it's a worthwhile thing to do, or whether the change here is enough.

packages/crowdstrike/data_stream/fdr/elasticsearch/ingest_pipeline/default.yml Outdated
# IP Geolocation Lookup.
- geoip:
tag: geoip_observer_ip_into_observer_geo_0729ba64
if: ctx.observer?.ip != null
Copy link
Contributor

@efd6 efd6 Feb 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we make sure that this is also included in the generator that @navnit-elastic is using for his work?

Copy link
Contributor Author

@kcreddy kcreddy Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@navnit-elastic LMK where your current generator lies, I can add this change to be included there.

@kcreddy
Copy link
Contributor Author

kcreddy commented Feb 27, 2026 •
edited
Loading

I'm not sure that this is a close; there was extension work suggested in the issue to add options to skip the observer.ip (and possibly the others) GeoIP look-up entirely. This needs to be discussed to determine whether we think it's a worthwhile thing to do, or whether the change here is enough.

@efd6, in 2f6b0c0, I added the flags which allows for disabling the geoip enrichment. PTAL

@kcreddy kcreddy requested a review from efd6 February 27, 2026 06:21
efd6
efd6 approved these changes Feb 27, 2026
View reviewed changes
Copy link
Contributor

@efd6 efd6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM after changelog addition.

@kcreddy kcreddy changed the title crowdstrike.fdr: Add if conditions to geoip processors crowdstrike.fdr: Add if conditions and opt-out config for geoip processors Feb 27, 2026
@elasticmachine
Copy link

elasticmachine commented Feb 27, 2026

💚 Build Succeeded

History

cc @kcreddy

@kcreddy kcreddy added the Category: Integration quality Category: Quality used for SI planning label Feb 27, 2026
@kcreddy kcreddy merged commit 7afb469 into elastic:main Feb 27, 2026
10 checks passed
@elastic-vault-github-plugin-prod
Copy link

elastic-vault-github-plugin-prod bot commented Feb 27, 2026

Package crowdstrike - 3.6.0 containing this change is available at https://epr.elastic.co/package/crowdstrike/3.6.0/

ilyannn pushed a commit that referenced this pull request Feb 28, 2026
@kcreddy @ilyannn
crowdstrike.fdr: Add if conditions and opt-out config for geoip proce…
685c226 
…ssors (#17584)

Avoid unnecessary geoip processor invocations when IP fields are absent.
Without an "if" guard, each geoip processor is still entered on every
document — triggering a database validity check[1] that traverses cluster 
state metadata on every call — before discovering the field is missing 
and returning early.
Adding "if: ctx.<field> != null" skips the processor entirely, avoiding
this overhead for the majority of documents that lack the relevant IP.

Additionally, expose three new configuration options —
enable_geoip_observer_ip, enable_geoip_source_ip, and
enable_geoip_destination_ip — that allow users to selectively disable
GeoIP enrichment per IP field. All default to true for backward
compatibility.

New test sample log copied from existing sample, but with disabled
new configuration options to verify "geo" fields are not populated.

[1] https://github.com/elastic/elasticsearch/blob/70266af5b6e9bd5a40a918cb34fc5d1c3fe680aa/modules/ingest-geoip/src/main/java/org/elasticsearch/ingest/geoip/GeoIpProcessor.java#L103
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@efd6 efd6 efd6 approved these changes

Assignees

@kcreddy kcreddy

Labels

Category: Integration quality Category: Quality used for SI planning enhancement New feature or request Integration:crowdstrike CrowdStrike Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations]

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

crowdstrike: fdr pipeline geoip processors lack if conditions causing unnecessary CPU overhead at scale

3 participants

@kcreddy @elasticmachine @efd6