crowdstrike: remove redundant GeoIP look-ups

[efd6]

What does this PR do?

Removes redundant GeoIP look-ups. These appear ~twice (the first instance is a subset of the second — this change removes the first).

integrations/packages/crowdstrike/data_stream/fdr/elasticsearch/ingest_pipeline/default.yml

Lines 1463 to 1505 in 96e8c05

	## IP Geolocation Lookup
	- geoip:
	field: source.ip
	target_field: source.geo
	ignore_missing: true
	- geoip:
	field: destination.ip
	target_field: destination.geo
	ignore_missing: true
	## IP Autonomous System (AS) Lookup
	- geoip:
	database_file: GeoLite2-ASN.mmdb
	field: source.ip
	target_field: source.as
	properties:
	- asn
	- organization_name
	ignore_missing: true
	- geoip:
	database_file: GeoLite2-ASN.mmdb
	field: destination.ip
	target_field: destination.as
	properties:
	- asn
	- organization_name
	ignore_missing: true
	- rename:
	field: source.as.asn
	target_field: source.as.number
	ignore_missing: true
	- rename:
	field: source.as.organization_name
	target_field: source.as.organization.name
	ignore_missing: true
	- rename:
	field: destination.as.asn
	target_field: destination.as.number
	ignore_missing: true
	- rename:
	field: destination.as.organization_name
	target_field: destination.as.organization.name
	ignore_missing: true

integrations/packages/crowdstrike/data_stream/fdr/elasticsearch/ingest_pipeline/default.yml

Lines 1979 to 2025 in 96e8c05

	## IP Geolocation Lookup
	- geoip:
	field: observer.ip
	target_field: observer.geo
	ignore_missing: true
	- geoip:
	field: source.ip
	target_field: source.geo
	ignore_missing: true
	- geoip:
	field: destination.ip
	target_field: destination.geo
	ignore_missing: true
	## IP Autonomous System (AS) Lookup
	- geoip:
	database_file: GeoLite2-ASN.mmdb
	field: source.ip
	target_field: source.as
	properties:
	- asn
	- organization_name
	ignore_missing: true
	- geoip:
	database_file: GeoLite2-ASN.mmdb
	field: destination.ip
	target_field: destination.as
	properties:
	- asn
	- organization_name
	ignore_missing: true
	- rename:
	field: source.as.asn
	target_field: source.as.number
	ignore_missing: true
	- rename:
	field: source.as.organization_name
	target_field: source.as.organization.name
	ignore_missing: true
	- rename:
	field: destination.as.asn
	target_field: destination.as.number
	ignore_missing: true
	- rename:
	field: destination.as.organization_name
	target_field: destination.as.organization.name
	ignore_missing: true

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

• [ ]

How to test this PR locally

Related issues

• Relates crowdstrike: conditionally support Windows NT timestamps #5168 (comment)

Screenshots

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2023-03-17T23:18:40.411+0000

• Duration: 15 min 20 sec

Test stats 🧪

Test	Results
Failed	0
Passed	14
Skipped	0
Total	14

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

[metalshanked]

@efd6 - had a question on the below line.
It seems to create many events with the current timestamp due to the _ingest.timestamp block after the below.
Shouldn't the current timestamp be used for @timestamp like a last resort after some other potential date fields can be used for @timestamp?
Example: crowdstrike.FirstSeen or crowdstrike.Time etc?

{"processors": [{"set": {"tag": "set-timestamp", "field": "@timestamp", "copy_from": "event.created", "if": "ctx.event?.created != null && (ctx.crowdstrike?.ContextTimeStamp == null || ctx.crowdstrike?.ContextTimeStamp == \"\")"}}]}

  - set:
      tag: set-timestamp
      field: "@timestamp"
      copy_from: event.created
      if: ctx.event?.created != null && (ctx.crowdstrike?.ContextTimeStamp == null || ctx.crowdstrike?.ContextTimeStamp == "")
  - set:
      tag: set-timestamp-ingest
      field: "@timestamp"
      copy_from: _ingest.timestamp
      if: ctx["@timestamp"] == null
      

[elasticmachine]

🌐 Coverage report

Name	Metrics % (covered/total)	Diff
Packages	100.0% (2/2)	💚
Files	100.0% (9/9)	💚 3.432
Classes	100.0% (9/9)	💚 3.432
Methods	100.0% (53/53)	💚 8.84
Lines	95.111% (2646/2782)	👍 3.729
Conditionals	100.0% (0/0)	💚

[elasticmachine]

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

[efd6]

@metalshanked The second set processor only sets the @timestamp from _ingest.timestamp if it is not already set, as a fallback. If you are having problems with this, please file an issue with details so we can look into it.

[metalshanked]

Thanks @efd6 . Yes, due to ctx.crowdstrike?.ContextTimeStamp == null || ctx.crowdstrike?.ContextTimeStamp == ""
Lot of messages get marked with the _ingest.timestamp as a fallback. So was wondering if other timestamps in the Crowdstrike fields crowdstrike.FirstSeen or crowdstrike.Time can also be attempted before the _ingest.timestamp is applied.

[kcreddy]

LGTM

[elasticmachine]

Package crowdstrike - 1.10.2 containing this change is available at https://epr.elastic.co/search?package=crowdstrike