[forescout] Initial release of the forescout by sharadcrest · Pull Request #16426 · elastic/integrations

sharadcrest · 2025-12-09T09:48:34Z

Proposed commit message

The initial release includes event data stream, associated dashboard
and visualizations.

forescout fields are mapped to their corresponding ECS fields where possible.

Test samples were derived from live data samples, which were subsequently
sanitized.

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.
I have added an entry to my package's changelog.yml file.
I have verified that Kibana version constraints are current according to guidelines.
I have verified that any added dashboard complies with Kibana's Dashboard good practices

How to test this PR locally

To test the forescout package:

Clone integrations repo.
Install elastic package locally.
Start elastic stack using elastic-package.
Move to integrations/packages/forescout directory.
Run the following command to run tests.

elastic-package test

Run asset tests for the package
2025/12/09 15:08:54  INFO License text found in "/home/devuser/github/integrations/LICENSE.txt" will be included in package
--- Test results for package: forescout - START ---
╭───────────┬─────────────┬───────────┬────────────────────────────────────────────────────────────────────┬────────┬──────────────╮
│ PACKAGE   │ DATA STREAM │ TEST TYPE │ TEST NAME                                                          │ RESULT │ TIME ELAPSED │
├───────────┼─────────────┼───────────┼────────────────────────────────────────────────────────────────────┼────────┼──────────────┤
│ forescout │             │ asset     │ dashboard forescout-52ac29ab-2ce2-4d68-937d-cac1cb92ab47 is loaded │ PASS   │        921ns │
│ forescout │             │ asset     │ search forescout-09d5d60b-1995-4851-b518-9337a4761cae is loaded    │ PASS   │        136ns │
│ forescout │ event       │ asset     │ index_template logs-forescout.event is loaded                      │ PASS   │        261ns │
│ forescout │ event       │ asset     │ ingest_pipeline logs-forescout.event-0.1.0 is loaded               │ PASS   │        203ns │
╰───────────┴─────────────┴───────────┴────────────────────────────────────────────────────────────────────┴────────┴──────────────╯
--- Test results for package: forescout - END   ---
Done
Run pipeline tests for the package
--- Test results for package: forescout - START ---
╭───────────┬─────────────┬───────────┬────────────────────────────────────────────────────┬────────┬──────────────╮
│ PACKAGE   │ DATA STREAM │ TEST TYPE │ TEST NAME                                          │ RESULT │ TIME ELAPSED │
├───────────┼─────────────┼───────────┼────────────────────────────────────────────────────┼────────┼──────────────┤
│ forescout │ event       │ pipeline  │ (ingest pipeline warnings test-pipeline-event.log) │ PASS   │ 353.515412ms │
│ forescout │ event       │ pipeline  │ test-pipeline-event.log                            │ PASS   │ 183.035682ms │
╰───────────┴─────────────┴───────────┴────────────────────────────────────────────────────┴────────┴──────────────╯
--- Test results for package: forescout - END   ---
Done
Run policy tests for the package
--- Test results for package: forescout - START ---
No test results
--- Test results for package: forescout - END   ---
Done
Run script tests for the package
--- Test results for package: forescout - START ---
PKG forescout
[no test files]
--- Test results for package: forescout - END ---
Done
Run static tests for the package
--- Test results for package: forescout - START ---
╭───────────┬─────────────┬───────────┬──────────────────────────┬────────┬──────────────╮
│ PACKAGE   │ DATA STREAM │ TEST TYPE │ TEST NAME                │ RESULT │ TIME ELAPSED │
├───────────┼─────────────┼───────────┼──────────────────────────┼────────┼──────────────┤
│ forescout │ event       │ static    │ Verify sample_event.json │ PASS   │ 125.779254ms │
╰───────────┴─────────────┴───────────┴──────────────────────────┴────────┴──────────────╯
--- Test results for package: forescout - END   ---
Done
Run system tests for the package
2025/12/09 15:08:59  INFO Installing package...
2025/12/09 15:08:59  INFO License text found in "/home/devuser/github/integrations/LICENSE.txt" will be included in package
2025/12/09 15:09:11  INFO Running test for data_stream "event" with configuration 'udp'
2025/12/09 15:09:20  INFO Setting up independent Elastic Agent...
2025/12/09 15:09:31  INFO Setting up service...
2025/12/09 15:09:53  INFO Validating test case...
2025/12/09 15:09:54  INFO Tearing down service...
2025/12/09 15:09:55  INFO Write container logs to file: /home/devuser/github/integrations/build/container-logs/forescout-event-udp-1765273195104041584.log
2025/12/09 15:09:58  INFO Tearing down agent...
2025/12/09 15:09:58  INFO Write container logs to file: /home/devuser/github/integrations/build/container-logs/elastic-agent-1765273198432716438.log
2025/12/09 15:10:11  INFO Running test for data_stream "event" with configuration 'tcp'
2025/12/09 15:10:20  INFO Setting up independent Elastic Agent...
2025/12/09 15:10:31  INFO Setting up service...
2025/12/09 15:10:51  INFO Validating test case...
2025/12/09 15:10:52  INFO Tearing down service...
2025/12/09 15:10:53  INFO Write container logs to file: /home/devuser/github/integrations/build/container-logs/forescout-event-tcp-1765273253274622964.log
2025/12/09 15:10:56  INFO Tearing down agent...
2025/12/09 15:10:56  INFO Write container logs to file: /home/devuser/github/integrations/build/container-logs/elastic-agent-1765273256451904211.log
2025/12/09 15:11:09  INFO Uninstalling package...
--- Test results for package: forescout - START ---
╭───────────┬─────────────┬───────────┬───────────┬────────┬───────────────╮
│ PACKAGE   │ DATA STREAM │ TEST TYPE │ TEST NAME │ RESULT │  TIME ELAPSED │
├───────────┼─────────────┼───────────┼───────────┼────────┼───────────────┤
│ forescout │ event       │ system    │ tcp       │ PASS   │ 41.396462621s │
│ forescout │ event       │ system    │ udp       │ PASS   │ 43.344074374s │
╰───────────┴─────────────┴───────────┴───────────┴────────┴───────────────╯
--- Test results for package: forescout - END   ---
Done

Related issues

Relates [New Integration] Forescout #14069

Screenshots

Go Code for Ingest Pipeline Generation

The event data stream pipeline is generated using Go code with the help of Dispear library.

package main

import (
	"fmt"
	"strings"

	. "github.com/efd6/dispear"
)

const (
	ECSVersion = "9.2.0"
	PkgRoot    = "json"
)
const errorFormat = "Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}"

// safeNavigateAndCheck converts a dot-separated field path to a safe navigation string.
//
// Example:
// "forescout.event.message" -> "ctx.forescout?.event?.message"
func safeNavigateAndCheck(field string) string {
	parts := strings.Split(field, ".")
	condition := "ctx"
	for i, part := range parts {
		if i > 0 { // Skip the first part which is already included in the condition
			condition += fmt.Sprintf("?.%s", part)
		} else {
			condition += fmt.Sprintf(".%s", part)
		}
	}
	return condition
}

func main() {

	// Initial processors of pipeline

	DESCRIPTION("Pipeline for processing event logs.")

	SET("ecs.version").VALUE(ECSVersion).TAG("set ecs.version to 9.2.0")

	// Setting event.* fields
	BLANK()
	BLANK().COMMENT("Set event.* fields")

	SET("event.kind").
		TAG(fmt.Sprintf("set %s to %s", "event.kind", "event")).
		VALUE("event")

	// extract fields from message
	BLANK()
	BLANK().COMMENT("extract fields from message")

	DISSECT("message", "%{forescout.event.service} : TTY=%{forescout.event.tty} ; PWD=%{forescout.event.pwd} ; USER=%{forescout.event.user} ; COMMAND=%{forescout.event.command}").
		IF(safeNavigateAndCheck("message") + " != null").
		DESCRIPTION("Extract fields from the `message` field using the Dissect processor.").
		IGNORE_FAILURE(true)

	// Map custom fields to corresponding ECS and related fields.
	BLANK()
	BLANK().COMMENT("Map custom fields to corresponding ECS and related fields.")

	SET("forescout.event.message").
		TAG(fmt.Sprintf("set %s to %s", "forescout.event.message", "message")).
		COPY_FROM("message").
		IGNORE_EMPTY(true)

	for _, mapping := range []struct {
		field, targetField string
	}{
		{field: "forescout.event.pwd", targetField: "process.working_directory"},
		{field: "forescout.event.user", targetField: "user.name"},
		{field: "forescout.event.user", targetField: "process.user.name"},
		{field: "forescout.event.command", targetField: "process.command_line"},
	} {
		SET(mapping.targetField).
			COPY_FROM(mapping.field).
			TAG(fmt.Sprintf("set %s from %s", mapping.targetField, strings.ReplaceAll(mapping.field, ".", "_"))).
			IGNORE_EMPTY(true)
	}

	// Map related fields
	BLANK()
	BLANK().COMMENT("Append related.* fields")

	APPEND("related.user", "{{{process.user.name}}}").
		ALLOW_DUPLICATES(false).IF(safeNavigateAndCheck("forescout.event.user") + " != null")
	APPEND("related.hosts", "{{{log.syslog.hostname}}}").
		ALLOW_DUPLICATES(false).IF(safeNavigateAndCheck("log.syslog.hostname") + " != null")

	// Remove duplicate custom fields
	BLANK()
	BLANK().COMMENT("Remove duplicate custom fields")
	REMOVE(
		"forescout.event.message",
		"forescout.event.pwd",
		"forescout.event.user",
		"forescout.event.command",
	).
		IF("ctx.tags == null || !ctx.tags.contains('preserve_duplicate_custom_fields')").
		TAG("remove_custom_duplicate_fields").
		IGNORE_MISSING(true)

	// Clean up script

	BLANK()
	BLANK().COMMENT("Cleanup")

	SCRIPT().
		TAG("script_to_drop_null_values").
		DESCRIPTION("This script processor iterates over the whole document to remove fields with null values.").
		LANG("painless").
		SOURCE(`
		void handleMap(Map map) {
			map.values().removeIf(v -> {
				if (v instanceof Map) {
					handleMap(v);
				} else if (v instanceof List) {
					handleList(v);
				}
				return v == null || v == '' || (v instanceof Map && v.size() == 0) || (v instanceof List && v.size() == 0)
			});
		}
		void handleList(List list) {
			list.removeIf(v -> {
				if (v instanceof Map) {
					handleMap(v);
				} else if (v instanceof List) {
					handleList(v);
				}
				return v == null || v == '' || (v instanceof Map && v.size() == 0) || (v instanceof List && v.size() == 0)
			});
		}
		handleMap(ctx);
		`)

	// Set and Append processor on last

	SET("event.kind").VALUE("pipeline_error").
		TAG("set event.kind to pipeline_error").
		IF("ctx.error?.message != null")
	APPEND("tags", "preserve_original_event").
		IF("ctx.error?.message != null").
		ALLOW_DUPLICATES(false)

	// Global on failure processor

	ON_FAILURE(
		APPEND("error.message", errorFormat),
		SET("event.kind").VALUE("pipeline_error").
			TAG("set event.kind to pipeline_error"),
		APPEND("tags", "preserve_original_event").
			ALLOW_DUPLICATES(false),
	)

	// Generate the pipeline

	Generate()
}

botelastic · 2026-01-14T07:48:44Z

Hi! We just realized that we haven't looked into this PR in a while. We're sorry! We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!

narph · 2026-02-06T09:35:43Z

 /packages/first_epss @elastic/security-service-integrations
 /packages/fleet_server @elastic/fleet
 /packages/forcepoint_web @elastic/security-service-integrations
+/packages/forescout @elastic/security-service-integrations


@cpascale43 , @qcorporation from a quick glance this integration belongs to the integration experience team, using tcp/udp input

@narph this makes a lot of sense.
@sharadcrest, can you update the owner to @elastic/integration-experience and assign it to us for review

narph · 2026-02-10T09:52:53Z

+        title: Collect logs from Forescout via UDP
+        description: Collecting logs from Forescout via UDP.
+owner:
+  github: elastic/security-service-integrations


@sharadcrest , a team update is also necessary here

taylor-swanson

~~Overall this looks great, just a few comments.~~

Edit: I missed the bigger issues with unparsed logs that @ilyannn pointed out. See my updated response below.

taylor-swanson · 2026-02-19T14:48:32Z

+  - set:
+      tag: set_event_kind_to_event_de80643c
+      field: event.kind
+      value: event


Looks like we are missing the following ECS categorization fields:

event.category

event.type

event.type can be a info but for event.category I couldn't identify suitable value. I’d really appreciate it if you could take a look and share your suggestions or guidance.

Yeah this highly depends on the content of the log. To give you an idea of how varied this can be, this is a large script from cisco_asa that sets these fields. It's a bit "easier" in that integration as there are message IDs we can key off of to reliably set the fields. It doesn't look like we have that luxury with this integration.

I'm okay with this being deferred to a follow-up task, but is something that should be addressed before the integration is GA-ed.

ilyannn · 2026-02-19T20:09:52Z

+                    "message": "Accepted publickey for root from 172.20.10.101 port 46018 ssh2: RSA SHA256:WokXPUll0YJnJwbAFK1xYfYR+DaVN2RVFEyK6lMW78c"
+                }
+            },
+            "message": "Accepted publickey for root from 172.20.10.101 port 46018 ssh2: RSA SHA256:WokXPUll0YJnJwbAFK1xYfYR+DaVN2RVFEyK6lMW78c",


This event is unparsed – is that expected?

ilyannn · 2026-02-19T20:11:10Z

+            },
+            "forescout": {
+                "event": {
+                    "message": "New session 14906 of user root."


This is also unparsed; in fact there is only one DISSECT pattern and most of the test logs are not processed by it. Is this a conscious decision or an oversight?

ilyannn · 2026-02-19T20:20:41Z

+  external: ecs
+  type: constant_keyword
+  value: RFC3164
+- name: observer.vendor


Should we also add the observer.product etc.?

The syslog data we’re collecting is forwarded through the syslog plugin, which is a core module of Forescout. So we can either set observer.product to Forescout which will be same as observer.vendor or leave it unmapped as it is in the current implementation. I’m happy to go with whichever approach you think is more appropriate.

taylor-swanson

So looking at the example logs (particularly the system logs as they are the full syslog messages), it looks like we are receiving all of the system messages (see logs from sshd and systemd), not just the Forescout service (_fsservice).

To echo @ilyannn's question, is this intended? Looking at the syslog docs for Forescout, I suppose it is intentional, since you can enable system logs. It appears it just forwards regular systemd and other service logs.

This is problematic since in order to properly parse these, not only do we have to write parsers for Forescout-specific logs, we also have to write parsers for regular linux-type logs. However, if we just want to generically ingest them, then the current solution is "okay", although I would prefer if the dissect keys off of the syslog app name, rather than blindly running the dissect and relying on ignore_failure to recover it. This is bad because a legitimate failure will be silenced by ignore_failure.

Since it sounds like there are other types of events to handle (of which I don't think the current dissect can handle?), a better solution needs to be found to handle other log types.

sharadcrest · 2026-02-23T06:15:40Z

Regarding the comments #16426 and #16426, I agree with @taylor-swanson(#16426). It was intentionally left unparsed.

According to RFC 3164 log format, the MSG portion of a syslog message contains the tag and message text, and it doesn’t enforce a structured format. Because of that, even if we extract important values like IP or port, it’s not always possible to determine their semantic role (e.g., source.ip vs destination.ip vs host.ip) from the free-form content.

taylor-swanson

Regarding the comments #16426 and #16426, I agree with @taylor-swanson(#16426). It was intentionally left unparsed.

According to RFC 3164 log format, the MSG portion of a syslog message contains the tag and message text, and it doesn’t enforce a structured format. Because of that, even if we extract important values like IP or port, it’s not always possible to determine their semantic role (e.g., source.ip vs destination.ip vs host.ip) from the free-form content.

I would think that exporting system logs (or really anything non-Forescout) should be turned off, but it's still a good idea to gracefully handle these logs coming through, even if it's just indexing them as-is.

I was going to mention looking at log.syslog.appname again, but then I looked closer at the example logs. It seems like all of these were run through sudo, which isn't a bad thing, it's just that the appname is going to be sudo instead of _fsservice. I was going to suggest something like: "If the appname is NOT _fsservice, then maybe think about dropping the log," but that may not be as easy as I think. It's probably safer right now to just index everything. If a user doesn't want those logs, then the best course of action would be to turn off the export of those logs.

taylor-swanson · 2026-02-23T16:50:21Z

/test

elastic-vault-github-plugin-prod · 2026-02-23T17:12:58Z

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

elasticmachine · 2026-02-23T17:13:22Z

💚 Build Succeeded

Buildkite Build
Commit: 3541147

sharadcrest · 2026-03-17T05:42:57Z

Hey @taylor-swanson, @ilyannn,

Could you please let me know if anything is still pending from my side? If everything looks good, can we proceed with the merge?

ilyannn

Since it seems to be fine that we just pass through and not parse the other logs, LGTM

ilyannn · 2026-03-19T12:28:22Z

I would think that exporting system logs (or really anything non-Forescout) should be turned off, but it's still a good idea to gracefully handle these logs coming through, even if it's just indexing them as-is.

I think it would be good to add this information to the README.

taylor-swanson

LGTM as well

@elastic/integrations-triaging, could someone take a look (@qcorporation perhaps)?

qcorporation

@sharadcrest
can you please update ./.github/integrations_bug.yml and integration_feature_request.yml please

qcorporation · 2026-03-19T14:23:39Z

@sharadcrest can you please update ./.github/integrations_bug.yml and integration_feature_request.yml please

@sharadcrest it would be also nice if you used the LLM docs generation - to generate the docs. @narph have we introduced this tooling to @sharadcrest ?

sharadcrest · 2026-03-23T07:43:22Z

@qcorporation in the integrations_bug.yml and integration_feature_request.yml files, apart from adding the integration name to the dropdown, is there anything else you would like me to update?

qcorporation · 2026-03-23T20:04:29Z

@qcorporation in the integrations_bug.yml and integration_feature_request.yml files, apart from adding the integration name to the dropdown, is there anything else you would like me to update?

@sharadcrest
If you can add the lines in both files that would be great, that will satisfy my review of this PR as the triaging portion.

Have you talked to @narph about docs generation? We have an LLM based docs generation workflow that I feel could be applied to this new integration - are you aware of it and do you think you should be using it to update the docs instead of writing it manually?

sharadcrest · 2026-03-24T07:01:49Z

Have you talked to @narph about docs generation?

I haven’t connected with @narph yet regarding the docs generation workflow. I’ll reach out and get guidance on the LLM-based docs generation process. If it fits well for this integration, I’ll use that approach instead of writing the documentation manually.

sharadcrest · 2026-03-30T05:57:50Z

@narph Does LLM based doc generation tool is ready to use?

piyush-elastic · 2026-03-31T12:50:59Z

@qcorporation — I noticed that we are currently waiting for the LLM-based documentation generation tool. I connected with @ShourieG , and it appears that it is still in progress. Would it be possible to proceed with this PR for now? If there are any changes required later, we can address them as enhancements. This would help us avoid blocking other PR that are currently in draft cause on this.
CC- @narph , @cpascale43

qcorporation · 2026-03-31T14:52:53Z

@qcorporation — I noticed that we are currently waiting for the LLM-based documentation generation tool. I connected with @ShourieG , and it appears that it is still in progress. Would it be possible to proceed with this PR for now? If there are any changes required later, we can address them as enhancements. This would help us avoid blocking other PR that are currently in draft cause on this. CC- @narph , @cpascale43

sound good

sharadcrest added 2 commits December 9, 2025 14:57

Initial release

cc794d2

Update changelog entry

426fa4b

sharadcrest marked this pull request as ready for review December 15, 2025 07:08

sharadcrest requested a review from a team as a code owner December 15, 2025 07:08

botelastic bot added the Stalled label Jan 14, 2026

narph reviewed Feb 6, 2026

View reviewed changes

botelastic bot removed the Stalled label Feb 6, 2026

update codeowner

c042159

sharadcrest requested a review from qcorporation February 9, 2026 08:16

narph requested a review from a team February 10, 2026 09:50

narph reviewed Feb 10, 2026

View reviewed changes

update owner

1e46831

sharadcrest requested a review from narph February 10, 2026 10:06

taylor-swanson reviewed Feb 19, 2026

View reviewed changes

ilyannn reviewed Feb 19, 2026

View reviewed changes

taylor-swanson reviewed Feb 19, 2026

View reviewed changes

resolve review comments

63ad318

sharadcrest requested review from ilyannn and taylor-swanson February 23, 2026 06:52

taylor-swanson reviewed Feb 23, 2026

View reviewed changes

Comment thread packages/forescout/manifest.yml Outdated

Comment thread packages/forescout/changelog.yml Outdated

revert owner and changelog entry

3541147

ilyannn approved these changes Mar 19, 2026

View reviewed changes

taylor-swanson approved these changes Mar 19, 2026

View reviewed changes

qcorporation reviewed Mar 19, 2026

View reviewed changes

Add note in readme and update logo

12e23a1

sharadcrest requested a review from qcorporation March 23, 2026 07:43

sharadcrest changed the base branch from main to feature/forescout-0.1.0 March 24, 2026 11:36

Update issue tempaltes

99bc9c1

sharadcrest requested a review from a team as a code owner March 26, 2026 05:12

qcorporation approved these changes Mar 31, 2026

View reviewed changes

qcorporation merged commit 34cd237 into elastic:feature/forescout-0.1.0 Apr 1, 2026
9 checks passed

janvi-elastic mentioned this pull request Apr 17, 2026

[forescout] Initial release of Forescout with Event and Host data streams #18493

Draft

5 tasks

akshraj-crest pushed a commit to akshraj-crest/integrations that referenced this pull request Apr 20, 2026

[forescout] Initial release of the forescout (elastic#16426)

4cc5883

Conversation

sharadcrest commented Dec 9, 2025

Proposed commit message

Checklist

How to test this PR locally

To test the forescout package:

Related issues

Screenshots

Go Code for Ingest Pipeline Generation

Uh oh!

botelastic bot commented Jan 14, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

taylor-swanson left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

taylor-swanson left a comment

Choose a reason for hiding this comment

Uh oh!

sharadcrest commented Feb 23, 2026

Uh oh!

taylor-swanson left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

taylor-swanson commented Feb 23, 2026

Uh oh!

elastic-vault-github-plugin-prod bot commented Feb 23, 2026

🚀 Benchmarks report

Uh oh!

elasticmachine commented Feb 23, 2026

💚 Build Succeeded

Uh oh!

sharadcrest commented Mar 17, 2026

Uh oh!

ilyannn left a comment

Choose a reason for hiding this comment

Uh oh!

ilyannn commented Mar 19, 2026

Uh oh!

taylor-swanson left a comment

Choose a reason for hiding this comment

Uh oh!

qcorporation left a comment

Choose a reason for hiding this comment

Uh oh!

qcorporation commented Mar 19, 2026

Uh oh!

sharadcrest commented Mar 23, 2026

Uh oh!

qcorporation commented Mar 23, 2026

Uh oh!

sharadcrest commented Mar 24, 2026

taylor-swanson left a comment •

edited

Loading