[cisco_ftd] Add network and dns ECS fields #5522

LaZyDK · 2023-03-13T14:31:02Z

What does this PR do?

This will sum up source.bytes and destination.bytes into network.bytes, along with extracting dns.question.registered_domain from dns.question.name.
user.name will be removed if the string is "Not Found".

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.
I have added an entry to my package's changelog.yml file.
I have verified that Kibana version constraints are current according to guidelines.

elasticmachine · 2023-03-13T14:34:14Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2023-03-28T07:23:50.629+0000
Duration: 20 min 31 sec

Test stats 🧪

Test	Results
Failed	0
Passed	20
Skipped	0
Total	20

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.

packages/cisco_ftd/data_stream/log/elasticsearch/ingest_pipeline/default.yml

efd6 · 2023-03-13T23:22:43Z

/test

…ne/default.yml Co-authored-by: Dan Kortschak <90160302+efd6@users.noreply.github.com>

elasticmachine · 2023-03-13T23:35:39Z

🌐 Coverage report

Name	Metrics % (`covered/total`)	Diff
Packages	100.0% (`1/1`)	💚
Files	100.0% (`1/1`)	💚
Classes	100.0% (`1/1`)	💚
Methods	100.0% (`19/19`)	💚
Lines	67.861% (`1288/1898`)	❕
Conditionals	100.0% (`0/0`)	💚

file-detected events should not be alert kind

In line with 430005 events that are not malware.

According to ECS

LaZyDK · 2023-03-16T08:36:07Z

I need help fixing usernames from FTD integrated with Passive Authentication.
This will generate the user.name with the format:
STRING\username

The string can be a custom description of the domain used for user lookup, but it is not required to be an actual domain name. (Eg. AD or COMPANYAD etc.)
I am using this in a custom pipeline, but I am not sure this is the correct way to implement the fix for example when the data source is Cisco ASA and not FTD.
(?:%{DATA}\\)?%{GREEDYDATA:user.name}

efd6

The pipeline test expects need to be updated for this.

Also needs additions to fields/ecs.yml:

[0] field "dns.question.registered_domain" is undefined
[1] field "dns.question.subdomain" is undefined
[2] field "dns.question.top_level_domain" is undefined

packages/cisco_ftd/changelog.yml

packages/cisco_ftd/data_stream/log/elasticsearch/ingest_pipeline/default.yml

efd6 · 2023-03-16T23:18:32Z

For #5522 (comment) do you have examples where they would differ and how that would cause an issue?

…ne/default.yml Co-authored-by: Dan Kortschak <90160302+efd6@users.noreply.github.com>

LaZyDK · 2023-03-17T08:33:59Z

For #5522 (comment) do you have examples where they would differ and how that would cause an issue?

I cannot comprehend the different options here, hence me asking.
Looking at some Cisco ASA data i find the user.name field filled with different things like:

real usernames (when cisco.asa.destination_username: LOCAL\user then user.name: user)
IPv4 addresses from Site2site tunnels (event.code: 602304)
"DefaultL2LGroup" and "Unknown" from IKEv2 tunnels (event.code: 750002 and 750003)

efd6 · 2023-03-17T09:10:02Z

Yeah, you really need to glean that on a case-by-case basis from the Cisco documentation.

efd6 · 2023-03-17T09:10:09Z

/test

LaZyDK · 2023-03-17T10:33:19Z

My mistake.

efd6 · 2023-03-17T23:20:13Z

/test

elasticmachine · 2023-03-20T03:05:41Z

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

efd6 · 2023-03-20T03:06:05Z

Needs an elastic-package build.

LaZyDK · 2023-03-21T10:56:02Z

Ready for review

efd6 · 2023-03-21T22:47:25Z

/test

LaZyDK · 2023-03-23T10:06:47Z

should be good now

efd6 · 2023-03-27T00:15:07Z

/test

efd6

Can you provide test cases for the new event.outcome classifications?

...es/cisco_ftd/data_stream/log/_dev/test/pipeline/test-security-file-malware.log-expected.json

packages/cisco_ftd/data_stream/log/elasticsearch/ingest_pipeline/default.yml

LaZyDK · 2023-03-27T08:46:54Z

Can you provide test cases for the new event.outcome classifications?

Added test case for event.outcome "monitor".
Still need to find events for "trust", "block with reset" and "domain not found".

LaZyDK · 2023-03-27T09:40:29Z

Added test cases for event.outcome "trust" and "block with reset".
Still need to find event for "domain not found".

LaZyDK · 2023-03-27T11:42:20Z

I found another value for event.outcome, so now I am looking for test data for both "pass" and "domain not found".

LaZyDK · 2023-03-27T15:32:23Z

I need help fixing usernames from FTD integrated with Passive Authentication. This will generate the user.name with the format: STRING\username

The string can be a custom description of the domain used for user lookup, but it is not required to be an actual domain name. (Eg. AD or COMPANYAD etc.) I am using this in a custom pipeline, but I am not sure this is the correct way to implement the fix for example when the data source is Cisco ASA and not FTD. (?:%{DATA}\\)?%{GREEDYDATA:user.name}

Fixed with this: https://github.com/elastic/integrations/pull/5522/files#diff-753433e478ec043eaee29b5dcd39b3f7153a0f858323be73401e0e1aab3e7255R2137

efd6 · 2023-03-27T20:40:12Z

/test

efd6

LGTM with minor nit.

packages/cisco_ftd/data_stream/log/_dev/test/pipeline/test-intrusion.log-expected.json

LaZyDK · 2023-03-28T07:26:24Z

Minor nit removed. All good from here. Ready to go.

elasticmachine · 2023-03-28T08:41:03Z

Package cisco_ftd - 2.9.2 containing this change is available at https://epr.elastic.co/search?package=cisco_ftd

Update default.yml

c49a073

LaZyDK requested a review from a team as a code owner March 13, 2023 14:31

LaZyDK added 2 commits March 13, 2023 15:33

Update changelog.yml

ba3bc81

Update manifest.yml

bae7762

efd6 reviewed Mar 13, 2023

View reviewed changes

packages/cisco_ftd/data_stream/log/elasticsearch/ingest_pipeline/default.yml Outdated Show resolved Hide resolved

Update packages/cisco_ftd/data_stream/log/elasticsearch/ingest_pipeli…

0444881

…ne/default.yml Co-authored-by: Dan Kortschak <90160302+efd6@users.noreply.github.com>

LaZyDK added 6 commits March 15, 2023 12:11

Change file-detected event kind

327017b

file-detected events should not be alert kind

Change file-detected category to file

a3fdc2f

In line with 430005 events that are not malware.

Define outcome actions

ff82dff

According to ECS

Merge branch 'main' into patch-3

3b5dd2c

Update default.yml

3cd18df

Fix user.name

36eebac

efd6 reviewed Mar 16, 2023

View reviewed changes

packages/cisco_ftd/changelog.yml Show resolved Hide resolved

packages/cisco_ftd/data_stream/log/elasticsearch/ingest_pipeline/default.yml Outdated Show resolved Hide resolved

LaZyDK and others added 3 commits March 17, 2023 09:17

Update ecs.yml

1686854

Update manifest.yml

8df6d3d

Update packages/cisco_ftd/data_stream/log/elasticsearch/ingest_pipeli…

f630a47

…ne/default.yml Co-authored-by: Dan Kortschak <90160302+efd6@users.noreply.github.com>

Merge branch 'elastic:main' into patch-3

c822c48

LaZyDK closed this Mar 17, 2023

LaZyDK deleted the patch-3 branch March 17, 2023 10:26

LaZyDK restored the patch-3 branch March 17, 2023 10:33

LaZyDK reopened this Mar 17, 2023

efd6 added enhancement New feature or request Team:Security-External Integrations Integration:CiscoFTD Cisco FTD Firepower Threat Defense labels Mar 20, 2023

LaZyDK added 2 commits March 21, 2023 11:27

elastic-package build

de15280

build tests

73c17cf

elastic-package build

5f34081

efd6 reviewed Mar 27, 2023

View reviewed changes

Remove user.id and add test case

cfec626

Add tests

210a126

LaZyDK added 2 commits March 27, 2023 14:02

Sanitize user.name and add tests

6215d4d

add outcome pass test

da5d2b6

efd6 approved these changes Mar 27, 2023

View reviewed changes

packages/cisco_ftd/data_stream/log/_dev/test/pipeline/test-intrusion.log-expected.json Outdated Show resolved Hide resolved

Remove empty user field

c525393

efd6 merged commit 890323c into elastic:main Mar 28, 2023
3 checks passed

LaZyDK deleted the patch-3 branch March 28, 2023 08:22

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[cisco_ftd] Add network and dns ECS fields #5522

[cisco_ftd] Add network and dns ECS fields #5522

LaZyDK commented Mar 13, 2023 •

edited

elasticmachine commented Mar 13, 2023 •

edited

Build stats

Test stats 🧪

efd6 commented Mar 13, 2023

elasticmachine commented Mar 13, 2023 •

edited

LaZyDK commented Mar 16, 2023 •

edited

efd6 left a comment

efd6 commented Mar 16, 2023

LaZyDK commented Mar 17, 2023

efd6 commented Mar 17, 2023

efd6 commented Mar 17, 2023

LaZyDK commented Mar 17, 2023

efd6 commented Mar 17, 2023

elasticmachine commented Mar 20, 2023

efd6 commented Mar 20, 2023

LaZyDK commented Mar 21, 2023

efd6 commented Mar 21, 2023

LaZyDK commented Mar 23, 2023

efd6 commented Mar 27, 2023

efd6 left a comment

LaZyDK commented Mar 27, 2023 •

edited

LaZyDK commented Mar 27, 2023

LaZyDK commented Mar 27, 2023

LaZyDK commented Mar 27, 2023

efd6 commented Mar 27, 2023

efd6 left a comment

LaZyDK commented Mar 28, 2023

elasticmachine commented Mar 28, 2023

[cisco_ftd] Add network and dns ECS fields #5522

[cisco_ftd] Add network and dns ECS fields #5522

Conversation

LaZyDK commented Mar 13, 2023 • edited

What does this PR do?

Checklist

elasticmachine commented Mar 13, 2023 • edited

💚 Build Succeeded

Build stats

Test stats 🧪

🤖 GitHub comments

efd6 commented Mar 13, 2023

elasticmachine commented Mar 13, 2023 • edited

🌐 Coverage report

LaZyDK commented Mar 16, 2023 • edited

efd6 left a comment

Choose a reason for hiding this comment

efd6 commented Mar 16, 2023

LaZyDK commented Mar 17, 2023

efd6 commented Mar 17, 2023

efd6 commented Mar 17, 2023

LaZyDK commented Mar 17, 2023

efd6 commented Mar 17, 2023

elasticmachine commented Mar 20, 2023

efd6 commented Mar 20, 2023

LaZyDK commented Mar 21, 2023

efd6 commented Mar 21, 2023

LaZyDK commented Mar 23, 2023

efd6 commented Mar 27, 2023

efd6 left a comment

Choose a reason for hiding this comment

LaZyDK commented Mar 27, 2023 • edited

LaZyDK commented Mar 27, 2023

LaZyDK commented Mar 27, 2023

LaZyDK commented Mar 27, 2023

efd6 commented Mar 27, 2023

efd6 left a comment

Choose a reason for hiding this comment

LaZyDK commented Mar 28, 2023

elasticmachine commented Mar 28, 2023

LaZyDK commented Mar 13, 2023 •

edited

elasticmachine commented Mar 13, 2023 •

edited

elasticmachine commented Mar 13, 2023 •

edited

LaZyDK commented Mar 16, 2023 •

edited

LaZyDK commented Mar 27, 2023 •

edited