[ti_crowdstrike] Initial release for ti_crowdstrike. #8789

mohitjha-elastic · 2023-12-22T15:28:59Z

What does this PR do?

Generated the skeleton of the ti_crowdstrike integration package.
Added data stream (Intel and IOC).
Added data collection logic for the data stream.
Added the ingest pipeline for the data stream.
Added Transform pipeline.
Mapped fields according to the ECS schema and added field metadata in the appropriate yml files.
Added test for pipeline for the data stream.

Integration release checklist

This checklist is intended for integration maintainers to ensure consistency when creating or updating a Package, Module, or Dataset for an Integration.

All changes

Change follows the contributing guidelines
Supported versions of the monitoring target are documented
Supported operating systems are documented (if applicable)
Integration or System tests exist
Documentation exists
Fields follow ECS and naming conventions
At least a manual test with ES / Kibana / Agent has been performed.
Required Kibana version set to: ^8.11.0

New Package

Screenshot of the "Add Integration" page on Fleet added

Log dataset changes

Pipeline tests exist (if applicable)
Generated output for at least 1 log file exists
Sample event (sample_event.json) exists

How to test this PR locally

Clone integrations repo.
Install elastic package locally.
Start elastic stack using elastic-package.
Move to integrations/packages/ti_crowdstrike directory.
Run the following command to run tests.

elastic-package test

Automated Test

2023/12/22 11:16:37 DEBUG Enable verbose logging
2023/12/22 11:16:37 DEBUG latest version (cached): v0.95.0. Download from: https://github.com/elastic/elastic-package/releases/tag/v0.95.0 (Timestamp 2023-12-22 11:08:54.364664205 +0530 IST)
Run test suite for the package
Run asset tests for the package
2023/12/22 11:16:37 DEBUG output command: /usr/bin/docker ps -a --filter label=com.docker.compose.project=elastic-package-stack --format {{.ID}}
2023/12/22 11:16:37 DEBUG output command: /usr/bin/docker inspect c2e6686fb732 e18b7a41a8f3 229bcddfb9e2 8a052b09a641 ca4b4a1a1c82 838569e6ecb6 4b876610ea09 68c4e40b5c8b 958d0c894646 71c0a6ac8960
2023/12/22 11:16:37 DEBUG Connecting with Elasticsearch host from current profile (profile: default, host: "https://127.0.0.1:9200")
2023/12/22 11:16:37 DEBUG output command: /usr/bin/docker ps -a --filter label=com.docker.compose.project=elastic-package-stack --format {{.ID}}
2023/12/22 11:16:37 DEBUG output command: /usr/bin/docker inspect c2e6686fb732 e18b7a41a8f3 229bcddfb9e2 8a052b09a641 ca4b4a1a1c82 838569e6ecb6 4b876610ea09 68c4e40b5c8b 958d0c894646 71c0a6ac8960
2023/12/22 11:16:37 DEBUG Connecting with Kibana host from current profile (profile: default, host: "https://127.0.0.1:5601")
2023/12/22 11:16:37 DEBUG GET https://127.0.0.1:5601/api/status
2023/12/22 11:16:37 DEBUG installing package...
2023/12/22 11:16:37 DEBUG Build directory: /root/develop/integrations/build/packages/ti_crowdstrike/0.1.0
2023/12/22 11:16:37 DEBUG Clear target directory (path: /root/develop/integrations/build/packages/ti_crowdstrike/0.1.0)
2023/12/22 11:16:37 DEBUG Copy package content (source: /root/develop/integrations/packages/ti_crowdstrike)
2023/12/22 11:16:37 DEBUG Copy license file if needed
2023/12/22 11:16:37  INFO License text found in "/root/develop/integrations/LICENSE.txt" will be included in package
2023/12/22 11:16:37 DEBUG Encode dashboards
2023/12/22 11:16:37 DEBUG Resolve external fields
2023/12/22 11:16:37 DEBUG Package has external dependencies defined
2023/12/22 11:16:37 DEBUG data_stream/intel/fields/base-fields.yml: source file hasn't been changed
2023/12/22 11:16:37 DEBUG data_stream/intel/fields/beats.yml: source file hasn't been changed
2023/12/22 11:16:37 DEBUG data_stream/intel/fields/fields.yml: source file hasn't been changed
2023/12/22 11:16:37 DEBUG data_stream/ioc/fields/base-fields.yml: source file hasn't been changed
2023/12/22 11:16:37 DEBUG data_stream/ioc/fields/beats.yml: source file hasn't been changed
2023/12/22 11:16:37 DEBUG data_stream/ioc/fields/fields.yml: source file hasn't been changed
2023/12/22 11:16:37 DEBUG elasticsearch/transform/latest/fields/fields.yml: source file hasn't been changed
2023/12/22 11:16:37  INFO Import ECS mappings into the built package (technical preview)
2023/12/22 11:16:37 DEBUG Build zipped package
2023/12/22 11:16:37 DEBUG Compress using archiver.Zip (destination: /root/develop/integrations/build/packages/ti_crowdstrike-0.1.0.zip)
2023/12/22 11:16:37 DEBUG Create work directory for archiving: /tmp/elastic-package-3624985307/ti_crowdstrike-0.1.0
2023/12/22 11:16:37 DEBUG Skip validation of the built .zip package
2023/12/22 11:16:37 DEBUG POST https://127.0.0.1:5601/api/fleet/epm/packages
2023/12/22 11:16:38 DEBUG removing package...
2023/12/22 11:16:38 DEBUG DELETE https://127.0.0.1:5601/api/fleet/epm/packages/ti_crowdstrike/0.1.0
2023/12/22 11:16:38  WARN failed to uninstall package "ti_crowdstrike": can't remove the package: could not remove package; API status code = 400; response body = {"statusCode":400,"error":"Bad Request","message":"unable to remove package with existing package policy(s) in use by agent(s)"}
--- Test results for package: ti_crowdstrike - START ---
╭────────────────┬─────────────┬───────────┬─────────────────────────────────────────────────────────────────────────┬────────┬──────────────╮
│ PACKAGE        │ DATA STREAM │ TEST TYPE │ TEST NAME                                                               │ RESULT │ TIME ELAPSED │
├────────────────┼─────────────┼───────────┼─────────────────────────────────────────────────────────────────────────┼────────┼──────────────┤
│ ti_crowdstrike │             │ asset     │ dashboard ti_crowdstrike-37a9ef30-9993-11ee-9b44-fd906664033c is loaded │ PASS   │      1.614µs │
│ ti_crowdstrike │             │ asset     │ dashboard ti_crowdstrike-b81b5020-9e64-11ee-9777-1d1f44d25bb5 is loaded │ PASS   │        171ns │
│ ti_crowdstrike │ intel       │ asset     │ index_template logs-ti_crowdstrike.intel is loaded                      │ PASS   │        233ns │
│ ti_crowdstrike │ intel       │ asset     │ ingest_pipeline logs-ti_crowdstrike.intel-0.1.0 is loaded               │ PASS   │         97ns │
│ ti_crowdstrike │ ioc         │ asset     │ index_template logs-ti_crowdstrike.ioc is loaded                        │ PASS   │        239ns │
│ ti_crowdstrike │ ioc         │ asset     │ ingest_pipeline logs-ti_crowdstrike.ioc-0.1.0 is loaded                 │ PASS   │        104ns │
╰────────────────┴─────────────┴───────────┴─────────────────────────────────────────────────────────────────────────┴────────┴──────────────╯
--- Test results for package: ti_crowdstrike - END   ---
Done
Run pipeline tests for the package
2023/12/22 11:16:38 DEBUG output command: /usr/bin/docker ps -a --filter label=com.docker.compose.project=elastic-package-stack --format {{.ID}}
2023/12/22 11:16:38 DEBUG output command: /usr/bin/docker inspect c2e6686fb732 e18b7a41a8f3 229bcddfb9e2 8a052b09a641 ca4b4a1a1c82 838569e6ecb6 4b876610ea09 68c4e40b5c8b 958d0c894646 71c0a6ac8960
2023/12/22 11:16:38 DEBUG Connecting with Elasticsearch host from current profile (profile: default, host: "https://127.0.0.1:9200")
--- Test results for package: ti_crowdstrike - START ---
╭────────────────┬─────────────┬───────────┬────────────────┬────────┬──────────────╮
│ PACKAGE        │ DATA STREAM │ TEST TYPE │ TEST NAME      │ RESULT │ TIME ELAPSED │
├────────────────┼─────────────┼───────────┼────────────────┼────────┼──────────────┤
│ ti_crowdstrike │ intel       │ pipeline  │ test-intel.log │ PASS   │   2.986288ms │
│ ti_crowdstrike │ ioc         │ pipeline  │ test-ioc.log   │ PASS   │   2.028767ms │
╰────────────────┴─────────────┴───────────┴────────────────┴────────┴──────────────╯
--- Test results for package: ti_crowdstrike - END   ---
Done
Run static tests for the package
2023/12/22 11:16:39 DEBUG output command: /usr/bin/docker ps -a --filter label=com.docker.compose.project=elastic-package-stack --format {{.ID}}
2023/12/22 11:16:39 DEBUG output command: /usr/bin/docker inspect c2e6686fb732 e18b7a41a8f3 229bcddfb9e2 8a052b09a641 ca4b4a1a1c82 838569e6ecb6 4b876610ea09 68c4e40b5c8b 958d0c894646 71c0a6ac8960
2023/12/22 11:16:39 DEBUG Connecting with Elasticsearch host from current profile (profile: default, host: "https://127.0.0.1:9200")
--- Test results for package: ti_crowdstrike - START ---
╭────────────────┬─────────────┬───────────┬──────────────────────────┬────────┬──────────────╮
│ PACKAGE        │ DATA STREAM │ TEST TYPE │ TEST NAME                │ RESULT │ TIME ELAPSED │
├────────────────┼─────────────┼───────────┼──────────────────────────┼────────┼──────────────┤
│ ti_crowdstrike │ intel       │ static    │ Verify sample_event.json │ PASS   │  96.615393ms │
│ ti_crowdstrike │ ioc         │ static    │ Verify sample_event.json │ PASS   │  117.55266ms │
╰────────────────┴─────────────┴───────────┴──────────────────────────┴────────┴──────────────╯
--- Test results for package: ti_crowdstrike - END   ---
Done
Run system tests for the package
2023/12/22 11:16:39 DEBUG output command: /usr/bin/docker ps -a --filter label=com.docker.compose.project=elastic-package-stack --format {{.ID}}
2023/12/22 11:16:39 DEBUG output command: /usr/bin/docker inspect c2e6686fb732 e18b7a41a8f3 229bcddfb9e2 8a052b09a641 ca4b4a1a1c82 838569e6ecb6 4b876610ea09 68c4e40b5c8b 958d0c894646 71c0a6ac8960
2023/12/22 11:16:39 DEBUG Connecting with Elasticsearch host from current profile (profile: default, host: "https://127.0.0.1:9200")
2023/12/22 11:16:39 DEBUG output command: /usr/bin/docker ps -a --filter label=com.docker.compose.project=elastic-package-stack --format {{.ID}}
2023/12/22 11:16:39 DEBUG output command: /usr/bin/docker inspect c2e6686fb732 e18b7a41a8f3 229bcddfb9e2 8a052b09a641 ca4b4a1a1c82 838569e6ecb6 4b876610ea09 68c4e40b5c8b 958d0c894646 71c0a6ac8960
2023/12/22 11:16:39 DEBUG Connecting with Kibana host from current profile (profile: default, host: "https://127.0.0.1:5601")
2023/12/22 11:16:39 DEBUG GET https://127.0.0.1:5601/api/status
2023/12/22 11:16:39 DEBUG Running system tests for data stream
2023/12/22 11:16:39 DEBUG running test with configuration 'default'
2023/12/22 11:16:39 DEBUG setting up service...
2023/12/22 11:16:39 DEBUG setting up service using Docker Compose service deployer
2023/12/22 11:16:39 DEBUG running command: /usr/local/bin/docker-compose version --short
2023/12/22 11:16:40 DEBUG Determined Docker Compose version: 1.27.4, the tool will use Compose V1
2023/12/22 11:16:40 DEBUG output command: /usr/bin/docker network inspect elastic-package-stack_default
2023/12/22 11:16:40 DEBUG running command: /usr/local/bin/docker-compose -f /root/develop/integrations/packages/ti_crowdstrike/_dev/deploy/docker/docker-compose.yml -p elastic-package-service up --build -d
Creating network "elastic-package-service_default" with the default driver
Creating elastic-package-service_ti_crowdstrike_1 ... 
�[1A�[2K
Creating elastic-package-service_ti_crowdstrike_1 ... �[32mdone�[0m
�[1B2023/12/22 11:16:41 DEBUG running command: /usr/local/bin/docker-compose -f /root/develop/integrations/packages/ti_crowdstrike/_dev/deploy/docker/docker-compose.yml -p elastic-package-service ps -a -q
2023/12/22 11:16:42 DEBUG Wait for healthy containers: 857df61a7e42e2bf3ce4683dac3599214051739d7b80afb29ef4fdb8b5a5fd68
2023/12/22 11:16:42 DEBUG output command: /usr/bin/docker inspect 857df61a7e42e2bf3ce4683dac3599214051739d7b80afb29ef4fdb8b5a5fd68
2023/12/22 11:16:42 DEBUG Container status: {"Config":{"Image":"docker.elastic.co/observability/stream:v0.13.0","Labels":{"BRANCH_NAME":"v0.13.0","GIT_SHA":"0d6bbf6c679d6352c079112c679b40e73aebe092","GO_VERSION":"1.19.5","TIMESTAMP":"2023-11-08_16:50","com.docker.compose.config-hash":"01ddf53b81056b30ba56eb901deb1053d5a066c71b2949fd997d15378fc06167","com.docker.compose.container-number":"1","com.docker.compose.oneoff":"False","com.docker.compose.project":"elastic-package-service","com.docker.compose.project.config_files":"/root/develop/integrations/packages/ti_crowdstrike/_dev/deploy/docker/docker-compose.yml","com.docker.compose.project.working_dir":"/root/develop/integrations/packages/ti_crowdstrike/_dev/deploy/docker","com.docker.compose.service":"ti_crowdstrike","com.docker.compose.version":"1.27.4"}},"ID":"857df61a7e42e2bf3ce4683dac3599214051739d7b80afb29ef4fdb8b5a5fd68","State":{"Status":"running","ExitCode":0,"Health":null}}
2023/12/22 11:16:42 DEBUG run command: /usr/bin/docker network connect elastic-package-stack_default elastic-package-service_ti_crowdstrike_1
2023/12/22 11:16:42 DEBUG adding service container elastic-package-service_ti_crowdstrike_1 internal ports to context
2023/12/22 11:16:42 DEBUG running command: /usr/local/bin/docker-compose -f /root/develop/integrations/packages/ti_crowdstrike/_dev/deploy/docker/docker-compose.yml -p elastic-package-service config
2023/12/22 11:16:43 DEBUG Installing package...
2023/12/22 11:16:43 DEBUG Build directory: /root/develop/integrations/build/packages/ti_crowdstrike/0.1.0
2023/12/22 11:16:43 DEBUG Clear target directory (path: /root/develop/integrations/build/packages/ti_crowdstrike/0.1.0)
2023/12/22 11:16:43 DEBUG Copy package content (source: /root/develop/integrations/packages/ti_crowdstrike)
2023/12/22 11:16:43 DEBUG Copy license file if needed
2023/12/22 11:16:43  INFO License text found in "/root/develop/integrations/LICENSE.txt" will be included in package
2023/12/22 11:16:43 DEBUG Encode dashboards
2023/12/22 11:16:43 DEBUG Resolve external fields
2023/12/22 11:16:43 DEBUG Package has external dependencies defined
2023/12/22 11:16:43 DEBUG data_stream/intel/fields/base-fields.yml: source file hasn't been changed
2023/12/22 11:16:43 DEBUG data_stream/intel/fields/beats.yml: source file hasn't been changed
2023/12/22 11:16:43 DEBUG data_stream/intel/fields/fields.yml: source file hasn't been changed
2023/12/22 11:16:43 DEBUG data_stream/ioc/fields/base-fields.yml: source file hasn't been changed
2023/12/22 11:16:43 DEBUG data_stream/ioc/fields/beats.yml: source file hasn't been changed
2023/12/22 11:16:43 DEBUG data_stream/ioc/fields/fields.yml: source file hasn't been changed
2023/12/22 11:16:43 DEBUG elasticsearch/transform/latest/fields/fields.yml: source file hasn't been changed
2023/12/22 11:16:43  INFO Import ECS mappings into the built package (technical preview)
2023/12/22 11:16:43 DEBUG Build zipped package
2023/12/22 11:16:43 DEBUG Compress using archiver.Zip (destination: /root/develop/integrations/build/packages/ti_crowdstrike-0.1.0.zip)
2023/12/22 11:16:43 DEBUG Create work directory for archiving: /tmp/elastic-package-3268241116/ti_crowdstrike-0.1.0
2023/12/22 11:16:43 DEBUG Skip validation of the built .zip package
2023/12/22 11:16:43 DEBUG POST https://127.0.0.1:5601/api/fleet/epm/packages
2023/12/22 11:16:44 DEBUG creating test policy...
2023/12/22 11:16:44 DEBUG POST https://127.0.0.1:5601/api/fleet/agent_policies
2023/12/22 11:16:50 DEBUG adding package data stream to test policy...
2023/12/22 11:16:50 DEBUG POST https://127.0.0.1:5601/api/fleet/package_policies
2023/12/22 11:16:52 DEBUG deleting old data in data stream...
2023/12/22 11:16:52 DEBUG found 0 hits in logs-ti_crowdstrike.intel-ep data stream
2023/12/22 11:16:52 DEBUG GET https://127.0.0.1:5601/api/fleet/agents
2023/12/22 11:16:52 DEBUG filter agents using criteria: NamePrefix=docker-fleet-agent
2023/12/22 11:16:52 DEBUG found 1 enrolled agent(s)
2023/12/22 11:16:52 DEBUG GET https://127.0.0.1:5601/api/fleet/agent_policies/7d36a470-a08d-11ee-a169-955e60687673
2023/12/22 11:16:52 DEBUG assigning package data stream to agent...
2023/12/22 11:16:52 DEBUG PUT https://127.0.0.1:5601/api/fleet/agents/5a7d7e5e-2def-4b46-bbc3-628e1fd09853/reassign
2023/12/22 11:16:54 DEBUG GET https://127.0.0.1:5601/api/fleet/agents/5a7d7e5e-2def-4b46-bbc3-628e1fd09853
2023/12/22 11:16:54 DEBUG Agent data: {"id":"5a7d7e5e-2def-4b46-bbc3-628e1fd09853","policy_id":"7d36a470-a08d-11ee-a169-955e60687673","local_metadata":{"host":{"name":"docker-fleet-agent"}}}
2023/12/22 11:16:54 DEBUG Wait until the policy (ID: 7d36a470-a08d-11ee-a169-955e60687673, revision: 2) is assigned to the agent (ID: 5a7d7e5e-2def-4b46-bbc3-628e1fd09853)...
2023/12/22 11:16:56 DEBUG GET https://127.0.0.1:5601/api/fleet/agents/5a7d7e5e-2def-4b46-bbc3-628e1fd09853
2023/12/22 11:16:56 DEBUG Agent data: {"id":"5a7d7e5e-2def-4b46-bbc3-628e1fd09853","policy_id":"7d36a470-a08d-11ee-a169-955e60687673","local_metadata":{"host":{"name":"docker-fleet-agent"}}}
2023/12/22 11:16:56 DEBUG Wait until the policy (ID: 7d36a470-a08d-11ee-a169-955e60687673, revision: 2) is assigned to the agent (ID: 5a7d7e5e-2def-4b46-bbc3-628e1fd09853)...
2023/12/22 11:16:58 DEBUG GET https://127.0.0.1:5601/api/fleet/agents/5a7d7e5e-2def-4b46-bbc3-628e1fd09853
2023/12/22 11:16:58 DEBUG Agent data: {"id":"5a7d7e5e-2def-4b46-bbc3-628e1fd09853","policy_id":"7d36a470-a08d-11ee-a169-955e60687673","local_metadata":{"host":{"name":"docker-fleet-agent"}}}
2023/12/22 11:16:58 DEBUG Wait until the policy (ID: 7d36a470-a08d-11ee-a169-955e60687673, revision: 2) is assigned to the agent (ID: 5a7d7e5e-2def-4b46-bbc3-628e1fd09853)...
2023/12/22 11:17:00 DEBUG GET https://127.0.0.1:5601/api/fleet/agents/5a7d7e5e-2def-4b46-bbc3-628e1fd09853
2023/12/22 11:17:00 DEBUG Agent data: {"id":"5a7d7e5e-2def-4b46-bbc3-628e1fd09853","policy_id":"7d36a470-a08d-11ee-a169-955e60687673","policy_revision":2,"local_metadata":{"host":{"name":"docker-fleet-agent"}}}
2023/12/22 11:17:00 DEBUG Policy revision assigned to the agent (ID: 5a7d7e5e-2def-4b46-bbc3-628e1fd09853)...
2023/12/22 11:17:00 DEBUG checking for expected data in data stream...
2023/12/22 11:17:00 DEBUG found 0 hits in logs-ti_crowdstrike.intel-ep data stream
2023/12/22 11:17:01 DEBUG found 0 hits in logs-ti_crowdstrike.intel-ep data stream
2023/12/22 11:17:02 DEBUG found 0 hits in logs-ti_crowdstrike.intel-ep data stream
2023/12/22 11:17:03 DEBUG found 1 hits in logs-ti_crowdstrike.intel-ep data stream
2023/12/22 11:17:07 DEBUG found 1 hits in logs-ti_crowdstrike.intel-ep data stream
2023/12/22 11:17:07 DEBUG running command: /usr/local/bin/docker-compose version --short
2023/12/22 11:17:08 DEBUG Determined Docker Compose version: 1.27.4, the tool will use Compose V1
2023/12/22 11:17:08 DEBUG running command: /usr/local/bin/docker-compose -f /root/develop/integrations/packages/ti_crowdstrike/_dev/deploy/docker/docker-compose.yml -p elastic-package-service ps -a -q ti_crowdstrike
2023/12/22 11:17:08 DEBUG output command: /usr/bin/docker inspect 857df61a7e42e2bf3ce4683dac3599214051739d7b80afb29ef4fdb8b5a5fd68
2023/12/22 11:17:08 DEBUG check whether or not synthetics is enabled (component template logs-ti_crowdstrike.intel@package)...
2023/12/22 11:17:08 DEBUG data stream logs-ti_crowdstrike.intel-ep has synthetics enabled: false
2023/12/22 11:17:08 DEBUG assert hit count expected 1, observed 1
2023/12/22 11:17:08 DEBUG checking transform "latest"
2023/12/22 11:17:08 DEBUG reassigning original policy back to agent...
2023/12/22 11:17:08 DEBUG PUT https://127.0.0.1:5601/api/fleet/agents/5a7d7e5e-2def-4b46-bbc3-628e1fd09853/reassign
2023/12/22 11:17:10 DEBUG GET https://127.0.0.1:5601/api/fleet/agents/5a7d7e5e-2def-4b46-bbc3-628e1fd09853
2023/12/22 11:17:10 DEBUG Agent data: {"id":"5a7d7e5e-2def-4b46-bbc3-628e1fd09853","policy_id":"elastic-agent-managed-ep","local_metadata":{"host":{"name":"docker-fleet-agent"}}}
2023/12/22 11:17:10 DEBUG Wait until the policy (ID: elastic-agent-managed-ep, revision: 7) is assigned to the agent (ID: 5a7d7e5e-2def-4b46-bbc3-628e1fd09853)...
2023/12/22 11:17:12 DEBUG GET https://127.0.0.1:5601/api/fleet/agents/5a7d7e5e-2def-4b46-bbc3-628e1fd09853
2023/12/22 11:17:12 DEBUG Agent data: {"id":"5a7d7e5e-2def-4b46-bbc3-628e1fd09853","policy_id":"elastic-agent-managed-ep","local_metadata":{"host":{"name":"docker-fleet-agent"}}}
2023/12/22 11:17:12 DEBUG Wait until the policy (ID: elastic-agent-managed-ep, revision: 7) is assigned to the agent (ID: 5a7d7e5e-2def-4b46-bbc3-628e1fd09853)...
2023/12/22 11:17:14 DEBUG GET https://127.0.0.1:5601/api/fleet/agents/5a7d7e5e-2def-4b46-bbc3-628e1fd09853
2023/12/22 11:17:14 DEBUG Agent data: {"id":"5a7d7e5e-2def-4b46-bbc3-628e1fd09853","policy_id":"elastic-agent-managed-ep","local_metadata":{"host":{"name":"docker-fleet-agent"}}}
2023/12/22 11:17:14 DEBUG Wait until the policy (ID: elastic-agent-managed-ep, revision: 7) is assigned to the agent (ID: 5a7d7e5e-2def-4b46-bbc3-628e1fd09853)...
2023/12/22 11:17:16 DEBUG GET https://127.0.0.1:5601/api/fleet/agents/5a7d7e5e-2def-4b46-bbc3-628e1fd09853
2023/12/22 11:17:16 DEBUG Agent data: {"id":"5a7d7e5e-2def-4b46-bbc3-628e1fd09853","policy_id":"elastic-agent-managed-ep","policy_revision":7,"local_metadata":{"host":{"name":"docker-fleet-agent"}}}
2023/12/22 11:17:16 DEBUG Policy revision assigned to the agent (ID: 5a7d7e5e-2def-4b46-bbc3-628e1fd09853)...
2023/12/22 11:17:16 DEBUG deleting test policy...
2023/12/22 11:17:16 DEBUG POST https://127.0.0.1:5601/api/fleet/agent_policies/delete
2023/12/22 11:17:19 DEBUG DELETE https://127.0.0.1:5601/api/fleet/epm/packages/ti_crowdstrike/0.1.0
2023/12/22 11:17:19  WARN failed to uninstall package "ti_crowdstrike": can't remove the package: could not remove package; API status code = 400; response body = {"statusCode":400,"error":"Bad Request","message":"unable to remove package with existing package policy(s) in use by agent(s)"}
2023/12/22 11:17:19 DEBUG tearing down service...
2023/12/22 11:17:19 DEBUG tearing down service using Docker Compose runner
2023/12/22 11:17:19 DEBUG running command: /usr/local/bin/docker-compose version --short
2023/12/22 11:17:20 DEBUG Determined Docker Compose version: 1.27.4, the tool will use Compose V1
2023/12/22 11:17:20 DEBUG running command: /usr/local/bin/docker-compose -f /root/develop/integrations/packages/ti_crowdstrike/_dev/deploy/docker/docker-compose.yml -p elastic-package-service logs
2023/12/22 11:17:20  INFO Write container logs to file: /root/develop/integrations/build/container-logs/ti_crowdstrike-1703224040703838742.log
2023/12/22 11:17:20 DEBUG running command: /usr/local/bin/docker-compose -f /root/develop/integrations/packages/ti_crowdstrike/_dev/deploy/docker/docker-compose.yml -p elastic-package-service down --volumes
Stopping elastic-package-service_ti_crowdstrike_1 ... 
�[1A�[2K
Stopping elastic-package-service_ti_crowdstrike_1 ... �[32mdone�[0m
�[1BRemoving elastic-package-service_ti_crowdstrike_1 ... 
�[1A�[2K
Removing elastic-package-service_ti_crowdstrike_1 ... �[32mdone�[0m
�[1BRemoving network elastic-package-service_default
2023/12/22 11:17:21 DEBUG deleting data in data stream...
2023/12/22 11:17:21 DEBUG Dump Elastic stack data
2023/12/22 11:17:21 DEBUG Dump stack logs (location: /tmp/test-system-2951520813)
2023/12/22 11:17:21 DEBUG output command: /usr/bin/docker ps -a --filter label=com.docker.compose.project=elastic-package-stack --format {{.ID}}
2023/12/22 11:17:21 DEBUG output command: /usr/bin/docker inspect c2e6686fb732 e18b7a41a8f3 229bcddfb9e2 8a052b09a641 ca4b4a1a1c82 838569e6ecb6 4b876610ea09 68c4e40b5c8b 958d0c894646 71c0a6ac8960
2023/12/22 11:17:21 DEBUG Dump stack logs for elastic-agent
2023/12/22 11:17:21 DEBUG running command: /usr/local/bin/docker-compose version --short
2023/12/22 11:17:22 DEBUG Determined Docker Compose version: 1.27.4, the tool will use Compose V1
2023/12/22 11:17:22 DEBUG running command: /usr/local/bin/docker-compose -f /root/.elastic-package/profiles/default/stack/snapshot.yml -p elastic-package-stack logs elastic-agent
2023/12/22 11:17:24 DEBUG running command: /usr/local/bin/docker-compose version --short
2023/12/22 11:17:24 DEBUG Determined Docker Compose version: 1.27.4, the tool will use Compose V1
2023/12/22 11:17:24 DEBUG run command: /usr/bin/docker cp elastic-package-stack_elastic-agent_1:/usr/share/elastic-agent/state/data/logs/ /tmp/test-system-2951520813/logs/elastic-agent-internal
2023/12/22 11:17:25 DEBUG Dump stack logs for fleet-server
2023/12/22 11:17:25 DEBUG running command: /usr/local/bin/docker-compose version --short
2023/12/22 11:17:25 DEBUG Determined Docker Compose version: 1.27.4, the tool will use Compose V1
2023/12/22 11:17:25 DEBUG running command: /usr/local/bin/docker-compose -f /root/.elastic-package/profiles/default/stack/snapshot.yml -p elastic-package-stack logs fleet-server
2023/12/22 11:17:27 DEBUG running command: /usr/local/bin/docker-compose version --short
2023/12/22 11:17:27 DEBUG Determined Docker Compose version: 1.27.4, the tool will use Compose V1
2023/12/22 11:17:27 DEBUG run command: /usr/bin/docker cp elastic-package-stack_fleet-server_1:/usr/share/elastic-agent/state/data/logs/ /tmp/test-system-2951520813/logs/fleet-server-internal
2023/12/22 11:17:27 DEBUG Dump stack logs for kibana
2023/12/22 11:17:27 DEBUG running command: /usr/local/bin/docker-compose version --short
2023/12/22 11:17:28 DEBUG Determined Docker Compose version: 1.27.4, the tool will use Compose V1
2023/12/22 11:17:28 DEBUG running command: /usr/local/bin/docker-compose -f /root/.elastic-package/profiles/default/stack/snapshot.yml -p elastic-package-stack logs kibana
2023/12/22 11:17:29 DEBUG Dump stack logs for package-registry
2023/12/22 11:17:29 DEBUG running command: /usr/local/bin/docker-compose version --short
2023/12/22 11:17:29 DEBUG Determined Docker Compose version: 1.27.4, the tool will use Compose V1
2023/12/22 11:17:29 DEBUG running command: /usr/local/bin/docker-compose -f /root/.elastic-package/profiles/default/stack/snapshot.yml -p elastic-package-stack logs package-registry
2023/12/22 11:17:31 DEBUG Dump stack logs for elasticsearch
2023/12/22 11:17:31 DEBUG running command: /usr/local/bin/docker-compose version --short
2023/12/22 11:17:32 DEBUG Determined Docker Compose version: 1.27.4, the tool will use Compose V1
2023/12/22 11:17:32 DEBUG running command: /usr/local/bin/docker-compose -f /root/.elastic-package/profiles/default/stack/snapshot.yml -p elastic-package-stack logs elasticsearch
2023/12/22 11:17:33 DEBUG skipped malformed docker-compose log line: Attaching to elastic-package-stack_elastic-agent_1
2023/12/22 11:17:33 DEBUG Running system tests for data stream
2023/12/22 11:17:33 DEBUG running test with configuration 'default'
2023/12/22 11:17:33 DEBUG setting up service...
2023/12/22 11:17:33 DEBUG setting up service using Docker Compose service deployer
2023/12/22 11:17:33 DEBUG running command: /usr/local/bin/docker-compose version --short
2023/12/22 11:17:33 DEBUG Determined Docker Compose version: 1.27.4, the tool will use Compose V1
2023/12/22 11:17:33 DEBUG output command: /usr/bin/docker network inspect elastic-package-stack_default
2023/12/22 11:17:33 DEBUG running command: /usr/local/bin/docker-compose -f /root/develop/integrations/packages/ti_crowdstrike/_dev/deploy/docker/docker-compose.yml -p elastic-package-service up --build -d
Creating network "elastic-package-service_default" with the default driver
Creating elastic-package-service_ti_crowdstrike_1 ... 
�[1A�[2K
Creating elastic-package-service_ti_crowdstrike_1 ... �[32mdone�[0m
�[1B2023/12/22 11:17:35 DEBUG running command: /usr/local/bin/docker-compose -f /root/develop/integrations/packages/ti_crowdstrike/_dev/deploy/docker/docker-compose.yml -p elastic-package-service ps -a -q
2023/12/22 11:17:36 DEBUG Wait for healthy containers: fc72bd1c03bcf8f5cdd2f1d12ae5cf7b06efe4d52c7e2536e441ea6f2404b7ee
2023/12/22 11:17:36 DEBUG output command: /usr/bin/docker inspect fc72bd1c03bcf8f5cdd2f1d12ae5cf7b06efe4d52c7e2536e441ea6f2404b7ee
2023/12/22 11:17:36 DEBUG Container status: {"Config":{"Image":"docker.elastic.co/observability/stream:v0.13.0","Labels":{"BRANCH_NAME":"v0.13.0","GIT_SHA":"0d6bbf6c679d6352c079112c679b40e73aebe092","GO_VERSION":"1.19.5","TIMESTAMP":"2023-11-08_16:50","com.docker.compose.config-hash":"01ddf53b81056b30ba56eb901deb1053d5a066c71b2949fd997d15378fc06167","com.docker.compose.container-number":"1","com.docker.compose.oneoff":"False","com.docker.compose.project":"elastic-package-service","com.docker.compose.project.config_files":"/root/develop/integrations/packages/ti_crowdstrike/_dev/deploy/docker/docker-compose.yml","com.docker.compose.project.working_dir":"/root/develop/integrations/packages/ti_crowdstrike/_dev/deploy/docker","com.docker.compose.service":"ti_crowdstrike","com.docker.compose.version":"1.27.4"}},"ID":"fc72bd1c03bcf8f5cdd2f1d12ae5cf7b06efe4d52c7e2536e441ea6f2404b7ee","State":{"Status":"running","ExitCode":0,"Health":null}}
2023/12/22 11:17:36 DEBUG run command: /usr/bin/docker network connect elastic-package-stack_default elastic-package-service_ti_crowdstrike_1
2023/12/22 11:17:36 DEBUG adding service container elastic-package-service_ti_crowdstrike_1 internal ports to context
2023/12/22 11:17:36 DEBUG running command: /usr/local/bin/docker-compose -f /root/develop/integrations/packages/ti_crowdstrike/_dev/deploy/docker/docker-compose.yml -p elastic-package-service config
2023/12/22 11:17:36 DEBUG Installing package...
2023/12/22 11:17:36 DEBUG Build directory: /root/develop/integrations/build/packages/ti_crowdstrike/0.1.0
2023/12/22 11:17:36 DEBUG Clear target directory (path: /root/develop/integrations/build/packages/ti_crowdstrike/0.1.0)
2023/12/22 11:17:36 DEBUG Copy package content (source: /root/develop/integrations/packages/ti_crowdstrike)
2023/12/22 11:17:36 DEBUG Copy license file if needed
2023/12/22 11:17:36  INFO License text found in "/root/develop/integrations/LICENSE.txt" will be included in package
2023/12/22 11:17:36 DEBUG Encode dashboards
2023/12/22 11:17:36 DEBUG Resolve external fields
2023/12/22 11:17:36 DEBUG Package has external dependencies defined
2023/12/22 11:17:36 DEBUG data_stream/intel/fields/base-fields.yml: source file hasn't been changed
2023/12/22 11:17:36 DEBUG data_stream/intel/fields/beats.yml: source file hasn't been changed
2023/12/22 11:17:36 DEBUG data_stream/intel/fields/fields.yml: source file hasn't been changed
2023/12/22 11:17:36 DEBUG data_stream/ioc/fields/base-fields.yml: source file hasn't been changed
2023/12/22 11:17:36 DEBUG data_stream/ioc/fields/beats.yml: source file hasn't been changed
2023/12/22 11:17:36 DEBUG data_stream/ioc/fields/fields.yml: source file hasn't been changed
2023/12/22 11:17:36 DEBUG elasticsearch/transform/latest/fields/fields.yml: source file hasn't been changed
2023/12/22 11:17:36  INFO Import ECS mappings into the built package (technical preview)
2023/12/22 11:17:36 DEBUG Build zipped package
2023/12/22 11:17:36 DEBUG Compress using archiver.Zip (destination: /root/develop/integrations/build/packages/ti_crowdstrike-0.1.0.zip)
2023/12/22 11:17:36 DEBUG Create work directory for archiving: /tmp/elastic-package-3474806357/ti_crowdstrike-0.1.0
2023/12/22 11:17:36 DEBUG Skip validation of the built .zip package
2023/12/22 11:17:36 DEBUG POST https://127.0.0.1:5601/api/fleet/epm/packages
2023/12/22 11:17:38 DEBUG creating test policy...
2023/12/22 11:17:38 DEBUG POST https://127.0.0.1:5601/api/fleet/agent_policies
2023/12/22 11:17:44 DEBUG adding package data stream to test policy...
2023/12/22 11:17:44 DEBUG POST https://127.0.0.1:5601/api/fleet/package_policies
2023/12/22 11:17:46 DEBUG deleting old data in data stream...
2023/12/22 11:17:46 DEBUG found 0 hits in logs-ti_crowdstrike.ioc-ep data stream
2023/12/22 11:17:46 DEBUG GET https://127.0.0.1:5601/api/fleet/agents
2023/12/22 11:17:46 DEBUG filter agents using criteria: NamePrefix=docker-fleet-agent
2023/12/22 11:17:46 DEBUG found 1 enrolled agent(s)
2023/12/22 11:17:46 DEBUG GET https://127.0.0.1:5601/api/fleet/agent_policies/9d761ae0-a08d-11ee-a169-955e60687673
2023/12/22 11:17:46 DEBUG assigning package data stream to agent...
2023/12/22 11:17:46 DEBUG PUT https://127.0.0.1:5601/api/fleet/agents/5a7d7e5e-2def-4b46-bbc3-628e1fd09853/reassign
2023/12/22 11:17:48 DEBUG GET https://127.0.0.1:5601/api/fleet/agents/5a7d7e5e-2def-4b46-bbc3-628e1fd09853
2023/12/22 11:17:48 DEBUG Agent data: {"id":"5a7d7e5e-2def-4b46-bbc3-628e1fd09853","policy_id":"9d761ae0-a08d-11ee-a169-955e60687673","local_metadata":{"host":{"name":"docker-fleet-agent"}}}
2023/12/22 11:17:48 DEBUG Wait until the policy (ID: 9d761ae0-a08d-11ee-a169-955e60687673, revision: 2) is assigned to the agent (ID: 5a7d7e5e-2def-4b46-bbc3-628e1fd09853)...
2023/12/22 11:17:50 DEBUG GET https://127.0.0.1:5601/api/fleet/agents/5a7d7e5e-2def-4b46-bbc3-628e1fd09853
2023/12/22 11:17:50 DEBUG Agent data: {"id":"5a7d7e5e-2def-4b46-bbc3-628e1fd09853","policy_id":"9d761ae0-a08d-11ee-a169-955e60687673","local_metadata":{"host":{"name":"docker-fleet-agent"}}}
2023/12/22 11:17:50 DEBUG Wait until the policy (ID: 9d761ae0-a08d-11ee-a169-955e60687673, revision: 2) is assigned to the agent (ID: 5a7d7e5e-2def-4b46-bbc3-628e1fd09853)...
2023/12/22 11:17:52 DEBUG GET https://127.0.0.1:5601/api/fleet/agents/5a7d7e5e-2def-4b46-bbc3-628e1fd09853
2023/12/22 11:17:52 DEBUG Agent data: {"id":"5a7d7e5e-2def-4b46-bbc3-628e1fd09853","policy_id":"9d761ae0-a08d-11ee-a169-955e60687673","local_metadata":{"host":{"name":"docker-fleet-agent"}}}
2023/12/22 11:17:52 DEBUG Wait until the policy (ID: 9d761ae0-a08d-11ee-a169-955e60687673, revision: 2) is assigned to the agent (ID: 5a7d7e5e-2def-4b46-bbc3-628e1fd09853)...
2023/12/22 11:17:54 DEBUG GET https://127.0.0.1:5601/api/fleet/agents/5a7d7e5e-2def-4b46-bbc3-628e1fd09853
2023/12/22 11:17:54 DEBUG Agent data: {"id":"5a7d7e5e-2def-4b46-bbc3-628e1fd09853","policy_id":"9d761ae0-a08d-11ee-a169-955e60687673","policy_revision":2,"local_metadata":{"host":{"name":"docker-fleet-agent"}}}
2023/12/22 11:17:54 DEBUG Policy revision assigned to the agent (ID: 5a7d7e5e-2def-4b46-bbc3-628e1fd09853)...
2023/12/22 11:17:54 DEBUG checking for expected data in data stream...
2023/12/22 11:17:54 DEBUG found 0 hits in logs-ti_crowdstrike.ioc-ep data stream
2023/12/22 11:17:55 DEBUG found 0 hits in logs-ti_crowdstrike.ioc-ep data stream
2023/12/22 11:17:56 DEBUG found 0 hits in logs-ti_crowdstrike.ioc-ep data stream
2023/12/22 11:17:57 DEBUG found 0 hits in logs-ti_crowdstrike.ioc-ep data stream
2023/12/22 11:17:58 DEBUG found 1 hits in logs-ti_crowdstrike.ioc-ep data stream
2023/12/22 11:18:02 DEBUG found 1 hits in logs-ti_crowdstrike.ioc-ep data stream
2023/12/22 11:18:02 DEBUG running command: /usr/local/bin/docker-compose version --short
2023/12/22 11:18:03 DEBUG Determined Docker Compose version: 1.27.4, the tool will use Compose V1
2023/12/22 11:18:03 DEBUG running command: /usr/local/bin/docker-compose -f /root/develop/integrations/packages/ti_crowdstrike/_dev/deploy/docker/docker-compose.yml -p elastic-package-service ps -a -q ti_crowdstrike
2023/12/22 11:18:03 DEBUG output command: /usr/bin/docker inspect fc72bd1c03bcf8f5cdd2f1d12ae5cf7b06efe4d52c7e2536e441ea6f2404b7ee
2023/12/22 11:18:03 DEBUG check whether or not synthetics is enabled (component template logs-ti_crowdstrike.ioc@package)...
2023/12/22 11:18:03 DEBUG data stream logs-ti_crowdstrike.ioc-ep has synthetics enabled: false
2023/12/22 11:18:03 DEBUG assert hit count expected 1, observed 1
2023/12/22 11:18:03 DEBUG transform "latest" does not match "logs-ti_crowdstrike.ioc-ep" as source (sources: [logs-ti_crowdstrike.intel-*])
2023/12/22 11:18:03 DEBUG reassigning original policy back to agent...
2023/12/22 11:18:03 DEBUG PUT https://127.0.0.1:5601/api/fleet/agents/5a7d7e5e-2def-4b46-bbc3-628e1fd09853/reassign
2023/12/22 11:18:05 DEBUG GET https://127.0.0.1:5601/api/fleet/agents/5a7d7e5e-2def-4b46-bbc3-628e1fd09853
2023/12/22 11:18:05 DEBUG Agent data: {"id":"5a7d7e5e-2def-4b46-bbc3-628e1fd09853","policy_id":"elastic-agent-managed-ep","local_metadata":{"host":{"name":"docker-fleet-agent"}}}
2023/12/22 11:18:05 DEBUG Wait until the policy (ID: elastic-agent-managed-ep, revision: 7) is assigned to the agent (ID: 5a7d7e5e-2def-4b46-bbc3-628e1fd09853)...
2023/12/22 11:18:07 DEBUG GET https://127.0.0.1:5601/api/fleet/agents/5a7d7e5e-2def-4b46-bbc3-628e1fd09853
2023/12/22 11:18:07 DEBUG Agent data: {"id":"5a7d7e5e-2def-4b46-bbc3-628e1fd09853","policy_id":"elastic-agent-managed-ep","local_metadata":{"host":{"name":"docker-fleet-agent"}}}
2023/12/22 11:18:07 DEBUG Wait until the policy (ID: elastic-agent-managed-ep, revision: 7) is assigned to the agent (ID: 5a7d7e5e-2def-4b46-bbc3-628e1fd09853)...
2023/12/22 11:18:09 DEBUG GET https://127.0.0.1:5601/api/fleet/agents/5a7d7e5e-2def-4b46-bbc3-628e1fd09853
2023/12/22 11:18:09 DEBUG Agent data: {"id":"5a7d7e5e-2def-4b46-bbc3-628e1fd09853","policy_id":"elastic-agent-managed-ep","local_metadata":{"host":{"name":"docker-fleet-agent"}}}
2023/12/22 11:18:09 DEBUG Wait until the policy (ID: elastic-agent-managed-ep, revision: 7) is assigned to the agent (ID: 5a7d7e5e-2def-4b46-bbc3-628e1fd09853)...
2023/12/22 11:18:11 DEBUG GET https://127.0.0.1:5601/api/fleet/agents/5a7d7e5e-2def-4b46-bbc3-628e1fd09853
2023/12/22 11:18:11 DEBUG Agent data: {"id":"5a7d7e5e-2def-4b46-bbc3-628e1fd09853","policy_id":"elastic-agent-managed-ep","policy_revision":7,"local_metadata":{"host":{"name":"docker-fleet-agent"}}}
2023/12/22 11:18:11 DEBUG Policy revision assigned to the agent (ID: 5a7d7e5e-2def-4b46-bbc3-628e1fd09853)...
2023/12/22 11:18:11 DEBUG deleting test policy...
2023/12/22 11:18:11 DEBUG POST https://127.0.0.1:5601/api/fleet/agent_policies/delete
2023/12/22 11:18:14 DEBUG DELETE https://127.0.0.1:5601/api/fleet/epm/packages/ti_crowdstrike/0.1.0
2023/12/22 11:18:14  WARN failed to uninstall package "ti_crowdstrike": can't remove the package: could not remove package; API status code = 400; response body = {"statusCode":400,"error":"Bad Request","message":"unable to remove package with existing package policy(s) in use by agent(s)"}
2023/12/22 11:18:14 DEBUG tearing down service...
2023/12/22 11:18:14 DEBUG tearing down service using Docker Compose runner
2023/12/22 11:18:14 DEBUG running command: /usr/local/bin/docker-compose version --short
2023/12/22 11:18:15 DEBUG Determined Docker Compose version: 1.27.4, the tool will use Compose V1
2023/12/22 11:18:15 DEBUG running command: /usr/local/bin/docker-compose -f /root/develop/integrations/packages/ti_crowdstrike/_dev/deploy/docker/docker-compose.yml -p elastic-package-service logs
2023/12/22 11:18:15  INFO Write container logs to file: /root/develop/integrations/build/container-logs/ti_crowdstrike-1703224095710251327.log
2023/12/22 11:18:15 DEBUG running command: /usr/local/bin/docker-compose -f /root/develop/integrations/packages/ti_crowdstrike/_dev/deploy/docker/docker-compose.yml -p elastic-package-service down --volumes
Stopping elastic-package-service_ti_crowdstrike_1 ... 
�[1A�[2K
Stopping elastic-package-service_ti_crowdstrike_1 ... �[32mdone�[0m
�[1BRemoving elastic-package-service_ti_crowdstrike_1 ... 
�[1A�[2K
Removing elastic-package-service_ti_crowdstrike_1 ... �[32mdone�[0m
�[1BRemoving network elastic-package-service_default
2023/12/22 11:18:16 DEBUG deleting data in data stream...
2023/12/22 11:18:16 DEBUG Dump Elastic stack data
2023/12/22 11:18:16 DEBUG Dump stack logs (location: /tmp/test-system-639402898)
2023/12/22 11:18:16 DEBUG output command: /usr/bin/docker ps -a --filter label=com.docker.compose.project=elastic-package-stack --format {{.ID}}
2023/12/22 11:18:16 DEBUG output command: /usr/bin/docker inspect c2e6686fb732 e18b7a41a8f3 229bcddfb9e2 8a052b09a641 ca4b4a1a1c82 838569e6ecb6 4b876610ea09 68c4e40b5c8b 958d0c894646 71c0a6ac8960
2023/12/22 11:18:16 DEBUG Dump stack logs for elastic-agent
2023/12/22 11:18:16 DEBUG running command: /usr/local/bin/docker-compose version --short
2023/12/22 11:18:17 DEBUG Determined Docker Compose version: 1.27.4, the tool will use Compose V1
2023/12/22 11:18:17 DEBUG running command: /usr/local/bin/docker-compose -f /root/.elastic-package/profiles/default/stack/snapshot.yml -p elastic-package-stack logs elastic-agent
2023/12/22 11:18:19 DEBUG running command: /usr/local/bin/docker-compose version --short
2023/12/22 11:18:20 DEBUG Determined Docker Compose version: 1.27.4, the tool will use Compose V1
2023/12/22 11:18:20 DEBUG run command: /usr/bin/docker cp elastic-package-stack_elastic-agent_1:/usr/share/elastic-agent/state/data/logs/ /tmp/test-system-639402898/logs/elastic-agent-internal
2023/12/22 11:18:20 DEBUG Dump stack logs for fleet-server
2023/12/22 11:18:20 DEBUG running command: /usr/local/bin/docker-compose version --short
2023/12/22 11:18:20 DEBUG Determined Docker Compose version: 1.27.4, the tool will use Compose V1
2023/12/22 11:18:20 DEBUG running command: /usr/local/bin/docker-compose -f /root/.elastic-package/profiles/default/stack/snapshot.yml -p elastic-package-stack logs fleet-server
2023/12/22 11:18:22 DEBUG running command: /usr/local/bin/docker-compose version --short
2023/12/22 11:18:22 DEBUG Determined Docker Compose version: 1.27.4, the tool will use Compose V1
2023/12/22 11:18:22 DEBUG run command: /usr/bin/docker cp elastic-package-stack_fleet-server_1:/usr/share/elastic-agent/state/data/logs/ /tmp/test-system-639402898/logs/fleet-server-internal
2023/12/22 11:18:22 DEBUG Dump stack logs for kibana
2023/12/22 11:18:22 DEBUG running command: /usr/local/bin/docker-compose version --short
2023/12/22 11:18:23 DEBUG Determined Docker Compose version: 1.27.4, the tool will use Compose V1
2023/12/22 11:18:23 DEBUG running command: /usr/local/bin/docker-compose -f /root/.elastic-package/profiles/default/stack/snapshot.yml -p elastic-package-stack logs kibana
2023/12/22 11:18:24 DEBUG Dump stack logs for package-registry
2023/12/22 11:18:24 DEBUG running command: /usr/local/bin/docker-compose version --short
2023/12/22 11:18:24 DEBUG Determined Docker Compose version: 1.27.4, the tool will use Compose V1
2023/12/22 11:18:24 DEBUG running command: /usr/local/bin/docker-compose -f /root/.elastic-package/profiles/default/stack/snapshot.yml -p elastic-package-stack logs package-registry
2023/12/22 11:18:26 DEBUG Dump stack logs for elasticsearch
2023/12/22 11:18:26 DEBUG running command: /usr/local/bin/docker-compose version --short
2023/12/22 11:18:27 DEBUG Determined Docker Compose version: 1.27.4, the tool will use Compose V1
2023/12/22 11:18:27 DEBUG running command: /usr/local/bin/docker-compose -f /root/.elastic-package/profiles/default/stack/snapshot.yml -p elastic-package-stack logs elasticsearch
2023/12/22 11:18:28 DEBUG skipped malformed docker-compose log line: Attaching to elastic-package-stack_elastic-agent_1
--- Test results for package: ti_crowdstrike - START ---
╭────────────────┬─────────────┬───────────┬───────────┬────────┬───────────────╮
│ PACKAGE        │ DATA STREAM │ TEST TYPE │ TEST NAME │ RESULT │  TIME ELAPSED │
├────────────────┼─────────────┼───────────┼───────────┼────────┼───────────────┤
│ ti_crowdstrike │ intel       │ system    │ default   │ PASS   │ 29.223019844s │
│ ti_crowdstrike │ ioc         │ system    │ default   │ PASS   │ 30.515751895s │
╰────────────────┴─────────────┴───────────┴───────────┴────────┴───────────────╯
--- Test results for package: ti_crowdstrike - END   ---
Done

Screenshot

kcreddy

For dashboards, the preferred approach is to use labels.is_ioc_transform_source and not _index field for querying the destination index.

Also, on what basis/fields are you expiring your indicators from the destination index?

packages/ti_crowdstrike/_dev/build/docs/README.md

packages/ti_crowdstrike/data_stream/intel/agent/stream/cel.yml.hbs

packages/ti_crowdstrike/data_stream/intel/elasticsearch/ingest_pipeline/default.yml

packages/ti_crowdstrike/elasticsearch/transform/latest/transform.yml

packages/ti_crowdstrike/manifest.yml

packages/ti_crowdstrike/data_stream/intel/fields/base-fields.yml

packages/ti_crowdstrike/data_stream/ioc/fields/base-fields.yml

packages/ti_crowdstrike/data_stream/intel/fields/fields.yml

Add transform pipeline for IOC. Add threat_intel in category in manifest. Update reamde, add details of the IOC and Intel endpoints. Add pagination in system test. Add IOC expiration duration user parameter. Add missing ECS mappings. Add threat.feed.name.

packages/ti_crowdstrike/_dev/build/docs/README.md

packages/ti_crowdstrike/data_stream/intel/_dev/test/pipeline/test-intel.log-expected.json

packages/ti_crowdstrike/data_stream/ioc/_dev/test/pipeline/test-ioc.log-expected.json

packages/ti_crowdstrike/data_stream/ioc/_dev/test/system/test-default-config.yml

packages/ti_crowdstrike/elasticsearch/transform/intel/transform.yml

...ges/ti_crowdstrike/kibana/dashboard/ti_crowdstrike-37a9ef30-9993-11ee-9b44-fd906664033c.json

packages/ti_crowdstrike/data_stream/intel/elasticsearch/ingest_pipeline/default.yml

Update the filter in dashboard. Update the name of transforms to latest_ioc and latest_intel respectively. Update threat.file.name to file.name ecs mapping Add more details in readme regarding transform pipeline.

kcreddy

LGTM 👍🏼

kcreddy · 2024-01-12T12:58:28Z

/test

elasticmachine · 2024-01-12T13:16:18Z

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

elasticmachine · 2024-01-12T13:29:54Z

Package ti_crowdstrike - 0.1.0 containing this change is available at https://epr.elastic.co/search?package=ti_crowdstrike

elasticmachine · 2024-01-12T13:59:14Z

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

mohitjha-elastic added 2 commits December 22, 2023 14:28

Initial release for ti_crowdstrike.

5f4f0c0

Update the changelog entry.

db46e75

kcreddy reviewed Dec 26, 2023

View reviewed changes

mohitjha-elastic mentioned this pull request Jan 9, 2024

threat.indicator.first_seen ECS Field Coming as a Keyword on Kibana elastic/elastic-package#1618

Open

Resolve Review Comments.

faa21db

Add transform pipeline for IOC. Add threat_intel in category in manifest. Update reamde, add details of the IOC and Intel endpoints. Add pagination in system test. Add IOC expiration duration user parameter. Add missing ECS mappings. Add threat.feed.name.

mohitjha-elastic requested a review from kcreddy January 10, 2024 11:41

kcreddy reviewed Jan 11, 2024

View reviewed changes

packages/ti_crowdstrike/data_stream/intel/elasticsearch/ingest_pipeline/default.yml Outdated Show resolved Hide resolved

Resolve review comments.

c8ed402

Update the filter in dashboard. Update the name of transforms to latest_ioc and latest_intel respectively. Update threat.file.name to file.name ecs mapping Add more details in readme regarding transform pipeline.

mohitjha-elastic requested a review from kcreddy January 12, 2024 12:07

kcreddy approved these changes Jan 12, 2024

View reviewed changes

kcreddy added the New Integration label Jan 12, 2024

kcreddy added the Integration:TI Crowdstrike label Jan 12, 2024

kcreddy merged commit 2f76c8a into elastic:main Jan 12, 2024
3 checks passed

kcreddy added the Team:Security-External Integrations label Jan 12, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ti_crowdstrike] Initial release for ti_crowdstrike. #8789

[ti_crowdstrike] Initial release for ti_crowdstrike. #8789

mohitjha-elastic commented Dec 22, 2023

kcreddy left a comment

kcreddy left a comment

kcreddy commented Jan 12, 2024

elasticmachine commented Jan 12, 2024

elasticmachine commented Jan 12, 2024

elasticmachine commented Jan 12, 2024

[ti_crowdstrike] Initial release for ti_crowdstrike. #8789

[ti_crowdstrike] Initial release for ti_crowdstrike. #8789

Conversation

mohitjha-elastic commented Dec 22, 2023

What does this PR do?

Integration release checklist

All changes

New Package

Log dataset changes

How to test this PR locally

Automated Test

Screenshot

kcreddy left a comment

Choose a reason for hiding this comment

kcreddy left a comment

Choose a reason for hiding this comment

kcreddy commented Jan 12, 2024

elasticmachine commented Jan 12, 2024

🚀 Benchmarks report

elasticmachine commented Jan 12, 2024

elasticmachine commented Jan 12, 2024