[Forcepoint Web Security] New integration

[colin-stubbs]

Enhancement

What does this PR do?

Adds initial release of forcepoint_web integration

Addresses issue #1208

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

Not sure. We're actually using this in production already just via custom injection of components that this will supercede.

How to test this PR locally

Use the packages/forcepoint_web/_dev/deploy/docker/sample_logs/full-sample.log as a log file to be ingested by the integration, e.g. docker cp packages/forcepoint_web/_dev/deploy/docker/sample_logs/full-sample.log CONTAINER_ID:/var/log/forcepoint-web-1.log

Related issues

• Closes Forcepoint Web Security Cloud #1208
• Relates Forcepoint CASB #1140 ( may be a solution )

Screenshots

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2023-01-23T06:28:49.991+0000

• Duration: 14 min 33 sec

Test stats 🧪

Test	Results
Failed	0
Passed	6
Skipped	0
Total	6

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

[elasticmachine]

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

[efd6]

/test

[elasticmachine]

🌐 Coverage report

Name	Metrics % (covered/total)	Diff
Packages	100.0% (1/1)	💚
Files	100.0% (1/1)	💚
Classes	100.0% (1/1)	💚
Methods	95.0% (19/20)	👍 20.0
Lines	86.017% (203/236)	👎 -13.983
Conditionals	100.0% (0/0)	💚

[efd6]

Looks good.

[efd6]

Almost passing locally. Needs the following change

diff --git a/packages/forcepoint_web/data_stream/logs/_dev/test/pipeline/test-forcepoint-web.json-expected.json b/packages/forcepoint_web/data_stream/logs/_dev/test/pipeline/test-forcepoint-web.json-expected.json
index f12487324..49fbf04a9 100644
--- a/packages/forcepoint_web/data_stream/logs/_dev/test/pipeline/test-forcepoint-web.json-expected.json
+++ b/packages/forcepoint_web/data_stream/logs/_dev/test/pipeline/test-forcepoint-web.json-expected.json
@@ -1,6 +1,7 @@
 {
     "expected": [
         {
+            "@timestamp": "2022-12-16T07:05:25.000Z",
             "data_stream": {
                 "dataset": "forcepoint_web.logs",
                 "namespace": "default",
@@ -52,7 +53,6 @@
                     "None"
                 ],
                 "time": "07:05:25",
-                "timestamp": "2022-12-16T07:05:25.000Z",
                 "user": "anonymous",
                 "user_agent_string": "Java/11.0.6"
             },

[colin-stubbs]

Odd but definitely not intended... I've modified as suggested. Tests passing. Additional commit/push made to resolve comments.

[efd6]

/test

[efd6]

This will need to be bumped.

[2023-01-20T07:21:15.967Z] Error: error running package asset tests: could not complete test run: can't install the package: can't install the package: could not install package; API status code = 422; response body = {"statusCode":422,"error":"Unprocessable Entity","message":"Document \"forcepoint_web-05cb8903-79fb-4aa1-a20e-db1eb073a8e9\" has property \"dashboard\" which belongs to a more recent version of Kibana [8.5.0]. The last known version is [8.0.0]"}

[colin-stubbs]

Bumped to 8.5.1

[efd6]

/test

[efd6]

This looks good, but I'm wondering if we need to full 2.7k lines of system tests; there are sets of lines that differ little from each other that do not appear to add significantly to test coverage.

[colin-stubbs]

Ah yeap, that'll be the full_sample.log ... not really used by the system tests, can definitely be removed from the release.

[efd6]

/test

[efd6]

Thanks

[elasticmachine]

Package forcepoint_web - 0.0.1 containing this change is available at https://epr.elastic.co/search?package=forcepoint_web