New integration - ESET Threat Intelligence #9255

MichalVisnovsky · 2024-03-01T08:29:32Z

What does this PR do?

PR adds integration package for ESET Threat Intelligence data feeds.

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.
I have added an entry to my package's changelog.yml file.
I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

[ ]

cla-checker-service · 2024-03-01T08:29:37Z

💚 CLA has been signed

kcreddy · 2024-03-04T08:20:14Z

/test

MichalVisnovsky · 2024-03-04T09:23:13Z

I changed my primary email to the one that was in the CLA, as it was set to my personal mail. Please try building it now.

kcreddy · 2024-03-04T09:32:02Z

/test

kcreddy · 2024-03-04T10:36:08Z

/test

elasticmachine · 2024-03-05T09:10:45Z

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

kcreddy · 2024-03-05T09:11:02Z

/test

kcreddy · 2024-03-05T09:15:40Z

@MichalVisnovsky Can you please add following line into .github/CODEOWNERS file:

/packages/ti_eset @elastic/security-service-integrations

kcreddy · 2024-03-05T09:22:08Z

/test

kcreddy

Thanks for PR once again 😄
I like the dashboards, just quick question:
Were they pointing at destination index, so we are only querying the latest active indicators? if not, you can query using _index: <alias name> or with NOT labels.is_ioc_transform_source: true

packages/ti_eset/_dev/build/build.yml

packages/ti_eset/_dev/build/docs/README.md

packages/ti_eset/manifest.yml

packages/ti_eset/_dev/build/docs/README.md

packages/ti_eset/data_stream/apt/manifest.yml

packages/ti_eset/elasticsearch/transform/apt_latest_ioc/transform.yml

packages/ti_eset/data_stream/apt/elasticsearch/ingest_pipeline/default.yml

packages/ti_eset/manifest.yml

packages/ti_eset/data_stream/apt/fields/fields.yml

packages/ti_eset/data_stream/apt/manifest.yml

kcreddy · 2024-03-14T07:33:45Z

/test

kcreddy · 2024-03-14T08:02:12Z

@polakovicp Looks like the CI is failing on README file. Can you run elastic-package build && elastic-package format && elastic-package lint && elastic-package check && elastic-package build and commit the README files generated.

Also, please let me know when all the review comments are answered, I can take a look again.

polakovicp · 2024-03-15T09:46:21Z

@polakovicp Looks like the CI is failing on README file. Can you run elastic-package build && elastic-package format && elastic-package lint && elastic-package check && elastic-package build and commit the README files generated.

Also, please let me know when all the review comments are answered, I can take a look again.

@kcreddy thank you for your response. I found problem with paging #9372 and need some assistance :)
I'll update the PR once it's solved.

polakovicp · 2024-03-25T07:14:49Z

@polakovicp Looks like the CI is failing on README file. Can you run elastic-package build && elastic-package format && elastic-package lint && elastic-package check && elastic-package build and commit the README files generated.

Also, please let me know when all the review comments are answered, I can take a look again.

PR updated. Thanks for #9372, pagination and cursor is now working as expected.
All you comments were addressed, plus did few fixes in apt pipeline.

kcreddy · 2024-03-27T12:57:57Z

/test

kcreddy

Thanks for clearing all of the requested changes.

Looks like the CI is failing on README file changes.
Can you re-run system tests: eval "$(elastic-package stack shellinit)" && elastic-package test system --generate -v followed by elastic-package build && elastic-package format && elastic-package lint && elastic-package check && elastic-package build and commit the generated files?

kcreddy · 2024-03-27T13:02:46Z

packages/ti_eset/_dev/build/docs/README.md

+There are some minimum requirements for running Elastic Agent and for more information,
+refer to the link [here](https://www.elastic.co/guide/en/fleet/current/elastic-agent-installation.html).
+
+The minimum **Kibana version** required is **8.12.0**.


Update manifest.yml accordingly

conditions: kibana: version: "^8.12.0"

kcreddy · 2024-03-27T13:09:24Z

packages/ti_eset/data_stream/botnet/elasticsearch/ilm/default_policy.json

+                "actions": {
+                    "delete": {}
+                },
+                "min_age": "3d"


Suggested change

"min_age": "3d"

"min_age": "7d"

As per the README doc. Same for other datastreams that have 7d in README doc.

polakovicp · 2024-03-27T14:46:01Z

Thanks for clearing all of the requested changes.

Looks like the CI is failing on README file changes. Can you re-run system tests: eval "$(elastic-package stack shellinit)" && elastic-package test system --generate -v followed by elastic-package build && elastic-package format && elastic-package lint && elastic-package check && elastic-package build and commit the generated files?

@kcreddy PR updated.

kcreddy · 2024-03-27T14:48:57Z

/test

kcreddy · 2024-03-27T15:32:30Z

I ran your test locally and it seems to be an issue with pipeline tests.
Can you run eval "$(elastic-package stack shellinit)" && elastic-package test pipeline --generate -v and commit the generated file(s).

polakovicp · 2024-03-27T15:41:15Z

I ran your test locally and it seems to be an issue with pipeline tests. Can you run eval "$(elastic-package stack shellinit)" && elastic-package test pipeline --generate -v and commit the generated file(s).

Updated

kcreddy · 2024-03-27T15:44:06Z

/test

elasticmachine · 2024-03-27T16:02:45Z

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

elasticmachine · 2024-03-27T16:03:18Z

💚 Build Succeeded

Buildkite Build
Commit: 9e833df

History

💔 Build #9911 failed fcadfee
💔 Build #9910 failed 2c6d96f
💔 Build #9607 failed 0f4071f
💔 Build #9337 failed d159cc2
💔 Build #9336 failed 620c8f3

elastic-sonarqube · 2024-03-27T16:03:21Z

Quality Gate passed

Kudos, no new issues were introduced!

0 New issues
0 Security Hotspots
90.0% 90.0% Coverage on New Code
0.0% 0.0% Duplication on New Code

See analysis details on SonarQube

kcreddy

LGTM 👍🏼

elasticmachine · 2024-03-27T16:33:10Z

Package ti_eset - 0.1.0 containing this change is available at https://epr.elastic.co/search?package=ti_eset

…es (#9465) This PR aims to improve compatibility with threat intelligence and other security integrations, including ESET Threat Intelligence #9255, by lowercasing the ECS hash fields threat.indicator.file.hash.sha1, file.hash.sha and related.hash, which is more in line with similar security integrations. This would enable one to use indicator match rules more easily as well. Also fixes a parsing error when object_uri equals script.

MichalVisnovsky added 2 commits March 5, 2024 08:56

Add ESET Threat Intelligence package

4f88e6b

Change PR link

620c8f3

MichalVisnovsky force-pushed the main branch from 4dba656 to 620c8f3 Compare March 5, 2024 07:57

kcreddy added New Integration Integration:ESET Threat Intelligence Team:Security-Service Integrations Security Service Integrations Team labels Mar 5, 2024

Add package to code owners

d159cc2

kcreddy added Team:Security-Service Integrations Security Service Integrations Team and removed Team:Security-Service Integrations Security Service Integrations Team labels Mar 5, 2024

kcreddy reviewed Mar 5, 2024

View reviewed changes

kcreddy reviewed Mar 6, 2024

View reviewed changes

packages/ti_eset/data_stream/apt/manifest.yml Show resolved Hide resolved

polakovicp added 3 commits March 11, 2024 10:55

Implement PR review comments

b625a6a

Fix assert.hit_count

33e560a

Use ti_eset_latest index in Kibana dashboards

0f4071f

polakovicp force-pushed the main branch from 0410435 to 0f4071f Compare March 13, 2024 07:28

Pull request changes

2c6d96f

kcreddy reviewed Mar 27, 2024

View reviewed changes

PR changes

fcadfee

Update APT expected pipeline result

9e833df

kcreddy approved these changes Mar 27, 2024

View reviewed changes

kcreddy merged commit bb0d25a into elastic:main Mar 27, 2024
5 checks passed

This was referenced Mar 28, 2024

[ESET PROTECT] Make IoC ECS fields compatible with TI integrations #9459

Closed

[ESET PROTECT] Normalize ECS hash fields enabling indicator match rules #9465

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New integration - ESET Threat Intelligence #9255

New integration - ESET Threat Intelligence #9255

MichalVisnovsky commented Mar 1, 2024

cla-checker-service bot commented Mar 1, 2024 •

edited

Loading

kcreddy commented Mar 4, 2024

MichalVisnovsky commented Mar 4, 2024

kcreddy commented Mar 4, 2024

kcreddy commented Mar 4, 2024

elasticmachine commented Mar 5, 2024

kcreddy commented Mar 5, 2024

kcreddy commented Mar 5, 2024

kcreddy commented Mar 5, 2024

kcreddy left a comment

kcreddy commented Mar 14, 2024

kcreddy commented Mar 14, 2024

polakovicp commented Mar 15, 2024

polakovicp commented Mar 25, 2024

kcreddy commented Mar 27, 2024

kcreddy left a comment

kcreddy Mar 27, 2024

polakovicp Mar 27, 2024

kcreddy Mar 27, 2024

polakovicp Mar 27, 2024

polakovicp commented Mar 27, 2024

kcreddy commented Mar 27, 2024

kcreddy commented Mar 27, 2024

polakovicp commented Mar 27, 2024

kcreddy commented Mar 27, 2024

elasticmachine commented Mar 27, 2024

elasticmachine commented Mar 27, 2024

elastic-sonarqube bot commented Mar 27, 2024

kcreddy left a comment

elasticmachine commented Mar 27, 2024

New integration - ESET Threat Intelligence #9255

New integration - ESET Threat Intelligence #9255

Conversation

MichalVisnovsky commented Mar 1, 2024

What does this PR do?

Checklist

Author's Checklist

cla-checker-service bot commented Mar 1, 2024 • edited Loading

kcreddy commented Mar 4, 2024

MichalVisnovsky commented Mar 4, 2024

kcreddy commented Mar 4, 2024

kcreddy commented Mar 4, 2024

elasticmachine commented Mar 5, 2024

kcreddy commented Mar 5, 2024

kcreddy commented Mar 5, 2024

kcreddy commented Mar 5, 2024

kcreddy left a comment

Choose a reason for hiding this comment

kcreddy commented Mar 14, 2024

kcreddy commented Mar 14, 2024

polakovicp commented Mar 15, 2024

polakovicp commented Mar 25, 2024

kcreddy commented Mar 27, 2024

kcreddy left a comment

Choose a reason for hiding this comment

kcreddy Mar 27, 2024

Choose a reason for hiding this comment

polakovicp Mar 27, 2024

Choose a reason for hiding this comment

kcreddy Mar 27, 2024

Choose a reason for hiding this comment

polakovicp Mar 27, 2024

Choose a reason for hiding this comment

polakovicp commented Mar 27, 2024

kcreddy commented Mar 27, 2024

kcreddy commented Mar 27, 2024

polakovicp commented Mar 27, 2024

kcreddy commented Mar 27, 2024

elasticmachine commented Mar 27, 2024

🚀 Benchmarks report

elasticmachine commented Mar 27, 2024

💚 Build Succeeded

History

elastic-sonarqube bot commented Mar 27, 2024

Quality Gate passed

kcreddy left a comment

Choose a reason for hiding this comment

elasticmachine commented Mar 27, 2024

cla-checker-service bot commented Mar 1, 2024 •

edited

Loading