Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

updated Github Secret Scanning fingerprint with resolved_at #3802

Merged
merged 5 commits into from
Jul 25, 2022

Conversation

kcreddy
Copy link
Contributor

@kcreddy kcreddy commented Jul 22, 2022

What does this PR do?

Fixes bug in Github Secret Scanning alerts where events are not getting ingested into ES when an alert is resolved.

  • This PR adds resolved_at timestamp into fingerprint processor to ingest the resolved events

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

Related issues

@kcreddy kcreddy marked this pull request as ready for review July 22, 2022 06:25
@kcreddy kcreddy requested a review from a team as a code owner July 22, 2022 06:25
@elasticmachine
Copy link

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

@kcreddy kcreddy marked this pull request as draft July 22, 2022 06:26
@elasticmachine
Copy link

elasticmachine commented Jul 22, 2022

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2022-07-22T17:29:13.354+0000

  • Duration: 17 min 59 sec

Test stats 🧪

Test Results
Failed 0
Passed 29
Skipped 0
Total 29

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

@kcreddy kcreddy changed the title updated fingerprint with resolved_at updated Github Secret Scanning fingerprint with resolved_at Jul 22, 2022
@elasticmachine
Copy link

elasticmachine commented Jul 22, 2022

🌐 Coverage report

Name Metrics % (covered/total) Diff
Packages 100.0% (3/3) 💚
Files 100.0% (3/3) 💚 2.941
Classes 100.0% (3/3) 💚 2.941
Methods 100.0% (33/33) 💚 10.269
Lines 95.754% (451/471) 👍 5.171
Conditionals 100.0% (0/0) 💚

@kcreddy kcreddy marked this pull request as ready for review July 22, 2022 07:07
Copy link
Contributor

@r00tu53r r00tu53r left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look okay. But the test data does not test the change made. It's missing the resolved_at field and value.

@kcreddy
Copy link
Contributor Author

kcreddy commented Jul 22, 2022

@r00tu53r Any suggestion where I can add this test? Pipeline tests doesn't seem to contain _id field to show that it has changed successfully.

@r00tu53r
Copy link
Contributor

@r00tu53r Any suggestion where I can add this test? Pipeline tests doesn't seem to contain _id field to show that it has changed successfully.

@kcreddy I meant testing the date processor with resolved_at.

@r00tu53r
Copy link
Contributor

@kcreddy am curious why compute hash to assign it as the document id ? Why not use what elasticsearch generates ?

@kcreddy
Copy link
Contributor Author

kcreddy commented Jul 22, 2022

@r00tu53r The reason why we are using fingerprint processor to compute _id with these specific fields instead of default is to avoid duplicates. We only want to ingest events when there is some update or resolution to it. If we put it as default, it would create documents for every time the polling happens.

@kcreddy
Copy link
Contributor Author

kcreddy commented Jul 22, 2022

@kcreddy I meant testing the date processor with resolved_at

I tested that resolved_at field exists in Kibana with date and also the field can be seen in expected json log from pipeline test. I also verified the _id functionality manually in Kibana and against the value generated from _simulate API with fingerprint processor

Copy link
Contributor

@adriansr adriansr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting a change for performance

@kcreddy kcreddy requested a review from adriansr July 23, 2022 06:41
Copy link
Contributor

@adriansr adriansr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@kcreddy kcreddy merged commit fb2eef1 into elastic:main Jul 25, 2022
@kcreddy kcreddy deleted the github_secret_fix_resolved_tat branch July 26, 2022 03:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants