Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Crowdstrike] FDR - Fixes parsing errors in aip and ContextTimeStamp fields #5655

Merged
merged 1 commit into from
Mar 23, 2023

Conversation

kcreddy
Copy link
Contributor

@kcreddy kcreddy commented Mar 23, 2023

What does this PR do?

For FDR datastreaM:

  • Fixes aip and ContextTimeStamp fields parsing errors
  • Adds additional fields

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

How to test this PR locally

Related issues

Screenshots

@kcreddy kcreddy force-pushed the sdh-3136-crowdstrike-parsingerr branch from c059a32 to c73c50f Compare March 23, 2023 06:54
@kcreddy kcreddy changed the title [Crowdstrike] Multiple IPs in aip field and add new fields [Crowdstrike] FDR - Fixes parsing errors in aip and ContextTimeStamp fields Mar 23, 2023
@kcreddy kcreddy self-assigned this Mar 23, 2023
@elasticmachine
Copy link

elasticmachine commented Mar 23, 2023

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2023-03-23T10:54:35.090+0000

  • Duration: 14 min 19 sec

Test stats 🧪

Test Results
Failed 0
Passed 14
Skipped 0
Total 14

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

@kcreddy kcreddy marked this pull request as ready for review March 23, 2023 06:57
@kcreddy kcreddy requested a review from a team as a code owner March 23, 2023 06:57
@elasticmachine
Copy link

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

@kcreddy kcreddy force-pushed the sdh-3136-crowdstrike-parsingerr branch from c73c50f to e3b26c4 Compare March 23, 2023 07:14
@elasticmachine
Copy link

elasticmachine commented Mar 23, 2023

🌐 Coverage report

Name Metrics % (covered/total) Diff
Packages 100.0% (2/2) 💚
Files 100.0% (9/9) 💚
Classes 100.0% (9/9) 💚
Methods 100.0% (54/54) 💚 6.667
Lines 95.194% (2694/2830) 👎 -1.847
Conditionals 100.0% (0/0) 💚

@kcreddy kcreddy force-pushed the sdh-3136-crowdstrike-parsingerr branch 2 times, most recently from 168ebb2 to b1c8e50 Compare March 23, 2023 08:06
- convert:
field: crowdstrike.localipCount
type: integer
if: (ctx.crowdstrike?.localipCount != null && ctx.crowdstrike?.localipCount != "")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a thought looking at lot of repeated code..
Can we not run a painless script before all convert / set with copy_from processors to remove fields that are empty [""] / null

Like

Suggested change
if: (ctx.crowdstrike?.localipCount != null && ctx.crowdstrike?.localipCount != "")
- script:
lang: painless
description: This script processor iterates over the whole document to remove fields with null / empty values.
source: |
void handleMap(Map map) {
for (def x : map.values()) {
if (x instanceof Map) {
handleMap(x);
} else if (x instanceof List) {
handleList(x);
}
}
map.values().removeIf(v -> v == null || v == "" || v == "-" || v == "\"-\"" || ((v instanceof List || v instanceof Map) && v.isEmpty()));
}
void handleList(List list) {
for (def x : list) {
if (x instanceof Map) {
handleMap(x);
} else if (x instanceof List) {
handleList(x);
}
}
}
handleMap(ctx.crowdstrike);

@kcreddy kcreddy force-pushed the sdh-3136-crowdstrike-parsingerr branch from be8049c to 11fd9be Compare March 23, 2023 10:54
Copy link
Member

@P1llus P1llus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its a shame that observer.* ends up as an array, but not much we can do if aip can often be arrays, because its impossible to know which to choose.
Since the field type is still keyword then its still fine.

@kcreddy kcreddy merged commit 97f4971 into elastic:main Mar 23, 2023
@elasticmachine
Copy link

Package crowdstrike - 1.11.1 containing this change is available at https://epr.elastic.co/search?package=crowdstrike

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Integration:crowdstrike CrowdStrike
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Crowdstrike] aip and ContextTimeStamp parsing errors
4 participants