DTT1 - Execution time analysis #5191

rauldpm · 2024-04-09T13:39:04Z

Description

We need to perform an analysis of DTT1 execution times so that we have reference times. The idea of this issue is to collect the following information for future improvements:

Execution time of each module (Allocation -> Provision -> Testing)
Execution time of a complete process

This analysis must be carried out in a way that considers different deployments such as time-based when using N agents and M managers.

Since local execution will vary from machine to machine, this analysis will be performed in AWS.

Due to the #5193 limitation, we will use the following system matrix:

1 Manager - N agents
- Manager: Debian 12
- Agents [1]: Ubuntu 22
- Agents [6]: Ubuntu 22, Ubuntu 20, Ubuntu 18, Debian 11, Debian 10, CentOS 8
- Agents [10]: Ubuntu 22, Ubuntu 20, Ubuntu 18, Debian 11, Debian 10, CentOS 8, CentOS 7, Oracle 9, RedHat 9, RedHat 8
- Agents [13]: Ubuntu 22, Ubuntu 20, Ubuntu 18, Debian 11, Debian 10, CentOS 8, CentOS 7, Oracle 9, RedHat 9, RedHat 8, RedHat 7, Amazon Linux 2, Amazon Linux 2023
M Manager - 0 agent*
- Manager [1]: Ubuntu 22
- Manager [6]: Ubuntu 22, Ubuntu 20, Ubuntu 18, Debian 11, Debian 10, CentOS 8
- Manager [10]: Ubuntu 22, Ubuntu 20, Ubuntu 18, Debian 11, Debian 10, CentOS 8, CentOS 7, Oracle 9, RedHat 9, RedHat 8
- Manager [13]: Ubuntu 22, Ubuntu 20, Ubuntu 18, Debian 12, Debian 11, Debian 10, CentOS 8, CentOS 7, Oracle 9, RedHat 9, RedHat 8, RedHat 7, Amazon Linux 2

Note: The workflow will be executed locally, but the allocation, provision, and testing should be done in AWS
Note*: Manager test does not use agents in their workflow, this will only test the manager's side

This issue is not a stopper for DTT1 but it is related to DTT1 to improve it

Tasks

Analyze results of issue DTT1 - Iteration 3 - Test module - Battery test #5125 and provide times in a single comment
Complete time data specified in the Deliverable 1 and 2 tables
Create plots using the data collected (this task should be done after the previous one is completed) based on the Deliverables plot requirements

Deliverable 1 - Concurrence

Table: {Number of manager-Number of agents}-{Time-per-module-module}-{Time-total}
- Time should be provided in minutes
- Each execution should be provided in a single comment
- Due to DTT1 restrictions, 1:M and N:1 are only supported, N:M relations are not supported
- Tests should be executed parallelized if possible (launch and wait)

1 Manager	N Agent	Allocation (Time)	Provision (Time)	Testing (Time)	Cleanup (Time)	Total (Time)
1	1	--	--	--	--	--
1	6	--	--	--	--	--
1	10	--	--	--	--	--
1	13	--	--	--	--	--

Plot 1:N (1 manager - N agents)
- X-axis: Manager-Agent (number)
- Y-axis: Total time (minutes)

N Manager	Allocation (Time)	Provision (Time)	Testing (Time)	Cleanup (Time)	Total (Time)
2**	--	--	--	--	--
6	--	--	--	--	--
10	--	--	--	--	--
13	--	--	--	--	--

Plot M:1 (M manager - 0 agent)
- X-axis: Manager-Agent (number)
- Y-axis: Total time (minutes)

Note**: The procedure needs two managers to be in a cluster to test the deployment: #5191

Deliverable 2 - Multithreading

2 threads

1 Manager	N Agent	Allocation (Time)	Provision (Time)	Testing (Time)	Cleanup (Time)	Total (Time)
1	1	--	--	--	--	--
1	6	--	--	--	--	--
1	10	--	--	--	--	--
1	13	--	--	--	--	--

Plot 1:N (1 manager - N agents)
- X-axis: Manager-Agent (number)
- Y-axis: Total time (minutes)

N Manager	Allocation (Time)	Provision (Time)	Testing (Time)	Cleanup (Time)	Total (Time)
2**	--	--	--	--	--
6	--	--	--	--	--
10	--	--	--	--	--
13	--	--	--	--	--

Plot M:1 (M manager - 0 agent)
- X-axis: Manager-Agent (number)
- Y-axis: Total time (minutes)

4 threads

1 Manager	N Agent	Allocation (Time)	Provision (Time)	Testing (Time)	Cleanup (Time)	Total (Time)
1	1	--	--	--	--	--
1	6	--	--	--	--	--
1	10	--	--	--	--	--
1	13	--	--	--	--	--

Plot 1:N (1 manager - N agents)
- X-axis: Manager-Agent (number)
- Y-axis: Total time (minutes)

N Manager	Allocation (Time)	Provision (Time)	Testing (Time)	Cleanup (Time)	Total (Time)
2**	--	--	--	--	--
6	--	--	--	--	--
10	--	--	--	--	--
13	--	--	--	--	--

Plot M:1 (M manager - 0 agent)
- X-axis: Manager-Agent (number)
- Y-axis: Total time (minutes)

6 threads

1 Manager	N Agent	Allocation (Time)	Provision (Time)	Testing (Time)	Cleanup (Time)	Total (Time)
1	1	--	--	--	--	--
1	6	--	--	--	--	--
1	10	--	--	--	--	--
1	13	--	--	--	--	--

Plot 1:N (1 manager - N agents)
- X-axis: Manager-Agent (number)
- Y-axis: Total time (minutes)

N Manager	Allocation (Time)	Provision (Time)	Testing (Time)	Cleanup (Time)	Total (Time)
2**	--	--	--	--	--
6	--	--	--	--	--
10	--	--	--	--	--
13	--	--	--	--	--

Plot M:1 (M manager - 0 agent)
- X-axis: Manager-Agent (number)
- Y-axis: Total time (minutes)

8 threads

1 Manager	N Agent	Allocation (Time)	Provision (Time)	Testing (Time)	Cleanup (Time)	Total (Time)
1	1	--	--	--	--	--
1	6	--	--	--	--	--
1	10	--	--	--	--	--
1	13	--	--	--	--	--

Plot 1:N (1 manager - N agents)
- X-axis: Manager-Agent (number)
- Y-axis: Total time (minutes)

N Manager	Allocation (Time)	Provision (Time)	Testing (Time)	Cleanup (Time)	Total (Time)
2**	--	--	--	--	--
6	--	--	--	--	--
10	--	--	--	--	--
13	--	--	--	--	--

Plot M:1 (M manager - 0 agent)
- X-axis: Manager-Agent (number)
- Y-axis: Total time (minutes)

10 threads

1 Manager	N Agent	Allocation (Time)	Provision (Time)	Testing (Time)	Cleanup (Time)	Total (Time)
1	1	--	--	--	--	--
1	6	--	--	--	--	--
1	10	--	--	--	--	--
1	13	--	--	--	--	--

Plot 1:N (1 manager - N agents)
- X-axis: Manager-Agent (number)
- Y-axis: Total time (minutes)

N Manager	Allocation (Time)	Provision (Time)	Testing (Time)	Cleanup (Time)	Total (Time)
2**	--	--	--	--	--
6	--	--	--	--	--
10	--	--	--	--	--
13	--	--	--	--	--

Plot M:1 (M manager - 0 agent)
- X-axis: Manager-Agent (number)
- Y-axis: Total time (minutes)

The text was updated successfully, but these errors were encountered:

pro-akim · 2024-04-09T16:45:40Z

Update

Working first on #5125

pro-akim · 2024-04-10T07:53:09Z

Update

1. Analyze results of issue #5125 and provide times in a single comment

Allocation + All the agent tests (install,registration,restart,stop,uninstall) (no clean-up)

VAGRANT (1 Manager + 12 agents)

CASE	Allocation	Testing	Total
Test agent into all supported OS with one manager (RedHat 7)	16min	17min	33min
Test agent into all supported OS with one manager (RedHat 8)	13min	19min	32min
Test agent into all supported OS with one manager (RedHat 9)	16min	10min	26min
Test agent into all supported OS with one manager (Centos 7)	20min	18min	38min
Test agent into all supported OS with one manager (Centos 8)	14min	18min	32min
Test agent into all supported OS with one manager (Debian 11)	13min	18min	31min
Test agent into all supported OS with one manager (Debian 12)	13min	18min	31min
Test agent into all supported OS with one manager (Ubuntu 20.04)	18min	18min	36min
Test agent into all supported OS with one manager (Ubuntu 22.04)	13min	19min	32min
Test agent into all supported OS with one manager (Oracle 9)	13min	20min	23min
Test agent into all supported OS with one manager (Amazon Linux 2)	13min	20min	33min

AWS (1 Manager + 13 agents)

CASE	Allocation	Testing	Total
Test agent into all supported OS with one manager (RedHat 7)	5min	41min	46min
Test agent into all supported OS with one manager (RedHat 8)	5min	46min	51min
Test agent into all supported OS with one manager (RedHat 9)	8min	44min	56min
Test agent into all supported OS with one manager (Centos 7)	5min	43min	48min
Test agent into all supported OS with one manager (Centos 8)	5min	45min	50min
Test agent into all supported OS with one manager (Debian 10)	5min	41min	46min
Test agent into all supported OS with one manager (Debian 11)	6min	36min	42min
Test agent into all supported OS with one manager (Debian 12)	5min	48min	53min
Test agent into all supported OS with one manager (Ubuntu 20.04)	None	44min	None
Test agent into all supported OS with one manager (Ubuntu 22.04)	5min	44min	49min
Test agent into all supported OS with one manager (Oracle 9)	6min	41min	47min

2. Launch new battery of test after fixing the bugs #5125 (comment)

VAGRANT

(Antonio localhost)

CASE	Allocation	Provision	Testing	Total
Part 1: install,register,restart,stop,uninstall on all agents with clean up (1 manager + 7 agents)	10min	7min	17min	34min
Part 2: install,register,restart,stop,uninstall on all agents with clean up (1 manager + 6 agents)	7min	6min	17min	30min

(Federico localhost)

CASE	Allocation	Provision	Testing	Total
Part 1: install with provision and stop on all agents with clean up (1 manager + 7 agents)	12min	12min	13min	37min
Part 2: install with provision and stop on all agents with clean up (1 manager + 6 agents)	7min	15min	3min	25min

(Antonio localhost)

CASE	Allocation	Provision	Testing	Total
Part 1: install with provision and restart on all agents with clean up (1 manager + 7 agents)	7min	22min	4min	33min
Part 2: install with provision and restart on all agents with clean up (1 manager + 6 agents)	7min	12min	4min	23min
Part 1: install with provision and uninstall on all agents with clean up (1 manager + 7 agents)	7min	11min	14min	32min
Part 2: install with provision and uninstall on all agents with clean up (1 manager + 6 agents)	7min	13min	5min	25min

AWS

CASE	Allocation	Provision	Testing	Total
Part 1: install, register, restart,stop,uninstall on all agents with clean up (1 manager + 6 agents)	1min?? 1 ec2 only	2min	27min	30min
Part 2: install, register, restart,stop,uninstall on all agents with clean up (1 manager + 7 agents)	25seg ?? 1 ec2 only	2min	38min	40min 25seg
All: install with provision and stop on all agents with clean up (1 manager + 13 agents)	6min	27min	14min	47min
All: install with provision and restart on all agents with clean up (1 manager + 13 agents)	5min	27min	15min	47min
All: install with provision and uninstall on all agents with clean up (1 manager + 13 agents)	6min	27min	23min	56min

3. Central components tests

VAGRANT

(Antonio localhost)

CASE	Allocation	Provision	Testing	Total
12 managers (install,stop,restart,uninstall)	12min	0	56min	1h 8min

AWS

CASE	Allocation	Provision	Testing	Total
11 managers (install,stop,restart,uninstall)	3min	0	58min	1h 2min

Note:

Antonio Localhost

Lenovo ThinkPad T14 Gen 1
48Gib Memory
Intel® Core™ i5-10310U CPU @ 1.70GHz × 8
Ubuntu 22.04.3 LTS 64bit

Federico Localhost

🚧

pro-akim · 2024-04-10T09:16:42Z

Update

Waiting for some definitions to move to the next analysis.

pro-akim · 2024-04-11T10:28:12Z

Update

YAML

version: 0.1
description: This workflow is used to test agents deployment por DDT1 PoC
variables:
  agent-os:
    - linux-ubuntu-18.04-amd64
    - linux-ubuntu-20.04-amd64
    - linux-ubuntu-22.04-amd64
    - linux-debian-10-amd64
    - linux-debian-11-amd64
    - linux-oracle-9-amd64
    - linux-centos-7-amd64
    - linux-centos-8-amd64
    - linux-redhat-7-amd64
    - linux-redhat-8-amd64
    - linux-redhat-9-amd64
    - linux-amazon-2-amd64
    - linux-amazon-2023-amd64
  manager-os: linux-debian-12-amd64
  infra-provider: aws
  working-dir: /tmp/dtt1-poc

tasks:
  # Unique manager allocate task
  - task: "allocate-manager-{manager-os}"
    description: "Allocate resources for the manager."
    do:
      this: process
      with:
        path: python3
        args:
          - modules/allocation/main.py
          - action: create
          - provider: "{infra-provider}"
          - size: large
          - composite-name: "{manager-os}"
          - inventory-output: "{working-dir}/manager-{manager-os}/inventory.yaml"
          - track-output: "{working-dir}/manager-{manager-os}/track.yaml"
          - label-termination-date: "1d"
          - label-team: "qa"
    on-error: "abort-all"
    cleanup:
      this: process
      with:
        path: python3
        args:
          - modules/allocation/main.py
          - action: delete
          - track-output: "{working-dir}/manager-{manager-os}/track.yaml"

  # Unique agent allocate task
  - task: "allocate-agent-{agent}"
    description: "Allocate resources for the agent."
    do:
      this: process
      with:
        path: python3
        args:
          - modules/allocation/main.py
          - action: create
          - provider: "{infra-provider}"
          - size: small
          - composite-name: "{agent}"
          - inventory-output: "{working-dir}/agent-{agent}/inventory.yaml"
          - track-output: "{working-dir}/agent-{agent}/track.yaml"
          - label-termination-date: "1d"
          - label-team: "qa"
    foreach:
      - variable: agent-os
        as: agent
    cleanup:
      this: process
      with:
        path: python3
        args:
          - modules/allocation/main.py
          - action: delete
          - track-output: "{working-dir}/agent-{agent}/track.yaml"

  # Unique manager provision task
  - task: "provision-manager-{manager-os}"
    description: "Provision the manager."
    do:
      this: process
      with:
        path: python3
        args:
          - modules/provision/main.py
          - inventory: "{working-dir}/manager-{manager-os}/inventory.yaml"
          - install:
            - component: wazuh-manager
              type: assistant
              version: 4.7.3
              live: True
    depends-on:
      - "allocate-manager-{manager-os}"
    on-error: "abort-all"

  # Generic agent provision task
  - task: "provision-install-{agent}"
    description: "Provision resources for the {agent} agent."
    do:
      this: process
      with:
        path: python3
        args:
          - modules/provision/main.py
          - inventory: "{working-dir}/agent-{agent}/inventory.yaml"
          - dependencies:
            - manager: "{working-dir}/manager-{manager-os}/inventory.yaml"
          - install:
            - component: wazuh-agent
              type: package
              version: 4.7.3
              live: True
    depends-on:
      - "allocate-agent-{agent}"
      - "provision-manager-{manager-os}"
    foreach:
      - variable: agent-os
        as: agent

  # Generic agent test task
  - task: "run-agent-{agent}-tests"
    description: "Run tests install for the agent {agent}."
    do:
      this: process
      with:
        path: python3
        args:
          - modules/testing/main.py
          - targets:
            - wazuh-1: "{working-dir}/manager-{manager-os}/inventory.yaml"
            - agent: "{working-dir}/agent-{agent}/inventory.yaml"
          - tests: "stop,restart,uninstall"
          - component: "agent"
          - wazuh-version: "4.7.3"
          - wazuh-revision: "40714"
          - live: "True"
    foreach:
      - variable: agent-os
        as: agent
    depends-on:
      - "provision-install-{agent}"

1 Manager	N Agent	Allocation (Time)	Provision (Time)	Testing (Time)	Clean-up (Time)	Total (Time)
1	1	1m	5m	6m	3m	17m
1	6	3m	16m	18m	7m	44m
1	10	5m	22m	35m	12m	1h14m
1	13	5m	30m	41m	13m	1h29m

1 Manager 1 Agents


[2024-04-11 14:50:30] [INFO] [540577] [MainThread] [workflow_engine]: Executing DAG tasks.
[2024-04-11 14:50:30] [INFO] [540577] [MainThread] [workflow_engine]: Executing tasks in parallel.
[2024-04-11 14:50:30] [INFO] [540577] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-manager-linux-debian-12-amd64] Starting task.
[2024-04-11 14:50:53] [INFO] [540577] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-manager-linux-debian-12-amd64] Finished task in 23.20 seconds.
[2024-04-11 14:50:54] [INFO] [540577] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-ubuntu-22.04-amd64] Starting task.
[2024-04-11 14:51:16] [INFO] [540577] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-ubuntu-22.04-amd64] Finished task in 22.13 seconds.
[2024-04-11 14:51:16] [INFO] [540577] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-manager-linux-debian-12-amd64] Starting task.
[2024-04-11 14:56:01] [INFO] [540577] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-manager-linux-debian-12-amd64] Finished task in 285.44 seconds.
[2024-04-11 14:56:01] [INFO] [540577] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-ubuntu-22.04-amd64] Starting task.
[2024-04-11 14:57:35] [INFO] [540577] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-ubuntu-22.04-amd64] Finished task in 93.75 seconds.
[2024-04-11 14:57:35] [INFO] [540577] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-ubuntu-22.04-amd64-tests] Starting task.
[2024-04-11 15:00:20] [INFO] [540577] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-ubuntu-22.04-amd64-tests] Finished task in 165.31 seconds.
[2024-04-11 15:00:20] [INFO] [540577] [MainThread] [workflow_engine]: Executing Reverse DAG tasks.
[2024-04-11 15:00:20] [INFO] [540577] [MainThread] [workflow_engine]: Executing tasks in parallel.
[2024-04-11 15:00:20] [INFO] [540577] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-ubuntu-22.04-amd64] Starting task.
[2024-04-11 15:01:55] [INFO] [540577] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-ubuntu-22.04-amd64] Finished task in 94.53 seconds.
[2024-04-11 15:01:55] [INFO] [540577] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-manager-linux-debian-12-amd64] Starting task.
[2024-04-11 15:02:43] [INFO] [540577] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-manager-linux-debian-12-amd64] Finished task in 48.57 seconds.

1 Manager 6 Agents

[2024-04-11 14:00:58] [INFO] [522742] [MainThread] [workflow_engine]: Executing DAG tasks.
[2024-04-11 14:00:58] [INFO] [522742] [MainThread] [workflow_engine]: Executing tasks in parallel.
[2024-04-11 14:00:58] [INFO] [522742] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-manager-linux-debian-12-amd64] Starting task.
[2024-04-11 14:01:20] [INFO] [522742] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-manager-linux-debian-12-amd64] Finished task in 21.84 seconds.
[2024-04-11 14:01:20] [INFO] [522742] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-ubuntu-18.04-amd64] Starting task.
[2024-04-11 14:01:42] [INFO] [522742] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-ubuntu-18.04-amd64] Finished task in 21.98 seconds.
[2024-04-11 14:01:42] [INFO] [522742] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-ubuntu-20.04-amd64] Starting task.
[2024-04-11 14:02:04] [INFO] [522742] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-ubuntu-20.04-amd64] Finished task in 22.62 seconds.
[2024-04-11 14:02:04] [INFO] [522742] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-ubuntu-22.04-amd64] Starting task.
[2024-04-11 14:02:26] [INFO] [522742] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-ubuntu-22.04-amd64] Finished task in 21.50 seconds.
[2024-04-11 14:02:26] [INFO] [522742] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-debian-10-amd64] Starting task.
[2024-04-11 14:02:48] [INFO] [522742] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-debian-10-amd64] Finished task in 21.85 seconds.
[2024-04-11 14:02:48] [INFO] [522742] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-debian-11-amd64] Starting task.
[2024-04-11 14:03:10] [INFO] [522742] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-debian-11-amd64] Finished task in 21.79 seconds.
[2024-04-11 14:03:10] [INFO] [522742] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-centos-8-amd64] Starting task.
[2024-04-11 14:03:32] [INFO] [522742] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-centos-8-amd64] Finished task in 22.79 seconds.
[2024-04-11 14:03:32] [INFO] [522742] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-manager-linux-debian-12-amd64] Starting task.
[2024-04-11 14:07:31] [INFO] [522742] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-manager-linux-debian-12-amd64] Finished task in 238.49 seconds.
[2024-04-11 14:07:31] [INFO] [522742] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-ubuntu-18.04-amd64] Starting task.
[2024-04-11 14:09:16] [INFO] [522742] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-ubuntu-18.04-amd64] Finished task in 105.35 seconds.
[2024-04-11 14:09:16] [INFO] [522742] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-ubuntu-20.04-amd64] Starting task.
[2024-04-11 14:11:01] [INFO] [522742] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-ubuntu-20.04-amd64] Finished task in 104.45 seconds.
[2024-04-11 14:11:01] [INFO] [522742] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-ubuntu-22.04-amd64] Starting task.
[2024-04-11 14:13:52] [INFO] [522742] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-ubuntu-22.04-amd64] Finished task in 170.92 seconds.
[2024-04-11 14:13:52] [INFO] [522742] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-debian-10-amd64] Starting task.
[2024-04-11 14:15:31] [INFO] [522742] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-debian-10-amd64] Finished task in 99.61 seconds.
[2024-04-11 14:15:31] [INFO] [522742] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-debian-11-amd64] Starting task.
[2024-04-11 14:17:10] [INFO] [522742] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-debian-11-amd64] Finished task in 98.37 seconds.
[2024-04-11 14:17:10] [INFO] [522742] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-centos-8-amd64] Starting task.
[2024-04-11 14:19:45] [INFO] [522742] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-centos-8-amd64] Finished task in 155.14 seconds.
[2024-04-11 14:19:45] [INFO] [522742] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-ubuntu-18.04-amd64-tests] Starting task.
[2024-04-11 14:23:06] [INFO] [522742] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-ubuntu-18.04-amd64-tests] Finished task in 201.39 seconds.
[2024-04-11 14:23:06] [INFO] [522742] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-ubuntu-20.04-amd64-tests] Starting task.
[2024-04-11 14:26:06] [INFO] [522742] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-ubuntu-20.04-amd64-tests] Finished task in 179.92 seconds.
[2024-04-11 14:26:06] [INFO] [522742] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-ubuntu-22.04-amd64-tests] Starting task.
[2024-04-11 14:28:53] [INFO] [522742] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-ubuntu-22.04-amd64-tests] Finished task in 166.76 seconds.
[2024-04-11 14:28:53] [INFO] [522742] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-debian-10-amd64-tests] Starting task.
[2024-04-11 14:31:23] [INFO] [522742] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-debian-10-amd64-tests] Finished task in 150.42 seconds.
[2024-04-11 14:31:23] [INFO] [522742] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-debian-11-amd64-tests] Starting task.
[2024-04-11 14:33:58] [INFO] [522742] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-debian-11-amd64-tests] Finished task in 154.51 seconds.
[2024-04-11 14:33:58] [INFO] [522742] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-centos-8-amd64-tests] Starting task.
[2024-04-11 14:37:01] [INFO] [522742] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-centos-8-amd64-tests] Finished task in 183.30 seconds.
[2024-04-11 14:37:01] [INFO] [522742] [MainThread] [workflow_engine]: Executing Reverse DAG tasks.
[2024-04-11 14:37:01] [INFO] [522742] [MainThread] [workflow_engine]: Executing tasks in parallel.
[2024-04-11 14:37:01] [INFO] [522742] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-ubuntu-18.04-amd64] Starting task.
[2024-04-11 14:38:36] [INFO] [522742] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-ubuntu-18.04-amd64] Finished task in 94.57 seconds.
[2024-04-11 14:38:36] [INFO] [522742] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-ubuntu-20.04-amd64] Starting task.
[2024-04-11 14:39:40] [INFO] [522742] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-ubuntu-20.04-amd64] Finished task in 63.96 seconds.
[2024-04-11 14:39:40] [INFO] [522742] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-ubuntu-22.04-amd64] Starting task.
[2024-04-11 14:40:28] [INFO] [522742] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-ubuntu-22.04-amd64] Finished task in 48.70 seconds.
[2024-04-11 14:40:28] [INFO] [522742] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-debian-10-amd64] Starting task.
[2024-04-11 14:41:02] [INFO] [522742] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-debian-10-amd64] Finished task in 33.76 seconds.
[2024-04-11 14:41:02] [INFO] [522742] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-debian-11-amd64] Starting task.
[2024-04-11 14:42:22] [INFO] [522742] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-debian-11-amd64] Finished task in 79.33 seconds.
[2024-04-11 14:42:22] [INFO] [522742] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-centos-8-amd64] Starting task.
[2024-04-11 14:43:10] [INFO] [522742] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-centos-8-amd64] Finished task in 48.70 seconds.
[2024-04-11 14:43:10] [INFO] [522742] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-manager-linux-debian-12-amd64] Starting task.
[2024-04-11 14:44:14] [INFO] [522742] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-manager-linux-debian-12-amd64] Finished task in 64.04 seconds.
(deplo_test) akim@akim-PC:~/Desktop/test/wazuh-qa/deployability$

1 Manager 10 Agents

[2024-04-11 12:31:50] [INFO] [479394] [MainThread] [workflow_engine]: Executing DAG tasks.
[2024-04-11 12:31:50] [INFO] [479394] [MainThread] [workflow_engine]: Executing tasks in parallel.
[2024-04-11 12:31:50] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-manager-linux-debian-12-amd64] Starting task.
[2024-04-11 12:32:12] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-manager-linux-debian-12-amd64] Finished task in 21.66 seconds.
[2024-04-11 12:32:12] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-ubuntu-18.04-amd64] Starting task.
[2024-04-11 12:32:34] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-ubuntu-18.04-amd64] Finished task in 21.91 seconds.
[2024-04-11 12:32:34] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-ubuntu-20.04-amd64] Starting task.
[2024-04-11 12:32:56] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-ubuntu-20.04-amd64] Finished task in 21.71 seconds.
[2024-04-11 12:32:56] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-ubuntu-22.04-amd64] Starting task.
[2024-04-11 12:33:17] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-ubuntu-22.04-amd64] Finished task in 21.45 seconds.
[2024-04-11 12:33:17] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-debian-10-amd64] Starting task.
[2024-04-11 12:33:39] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-debian-10-amd64] Finished task in 22.11 seconds.
[2024-04-11 12:33:39] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-debian-11-amd64] Starting task.
[2024-04-11 12:34:02] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-debian-11-amd64] Finished task in 22.56 seconds.
[2024-04-11 12:34:02] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-oracle-9-amd64] Starting task.
[2024-04-11 12:34:27] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-oracle-9-amd64] Finished task in 24.62 seconds.
[2024-04-11 12:34:27] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-centos-7-amd64] Starting task.
[2024-04-11 12:34:49] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-centos-7-amd64] Finished task in 22.24 seconds.
[2024-04-11 12:34:49] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-centos-8-amd64] Starting task.
[2024-04-11 12:35:14] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-centos-8-amd64] Finished task in 25.02 seconds.
[2024-04-11 12:35:14] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-redhat-8-amd64] Starting task.
[2024-04-11 12:35:38] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-redhat-8-amd64] Finished task in 24.15 seconds.
[2024-04-11 12:35:38] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-redhat-9-amd64] Starting task.
[2024-04-11 12:36:01] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-redhat-9-amd64] Finished task in 22.54 seconds.
[2024-04-11 12:36:01] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-manager-linux-debian-12-amd64] Starting task.
[2024-04-11 12:40:00] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-manager-linux-debian-12-amd64] Finished task in 239.28 seconds.
[2024-04-11 12:40:00] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-ubuntu-18.04-amd64] Starting task.
[2024-04-11 12:41:57] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-ubuntu-18.04-amd64] Finished task in 117.26 seconds.
[2024-04-11 12:41:57] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-ubuntu-20.04-amd64] Starting task.
[2024-04-11 12:43:47] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-ubuntu-20.04-amd64] Finished task in 109.38 seconds.
[2024-04-11 12:43:47] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-ubuntu-22.04-amd64] Starting task.
[2024-04-11 12:45:22] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-ubuntu-22.04-amd64] Finished task in 95.58 seconds.
[2024-04-11 12:45:22] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-debian-10-amd64] Starting task.
[2024-04-11 12:46:53] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-debian-10-amd64] Finished task in 90.92 seconds.
[2024-04-11 12:46:53] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-debian-11-amd64] Starting task.
[2024-04-11 12:48:29] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-debian-11-amd64] Finished task in 96.19 seconds.
[2024-04-11 12:48:29] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-oracle-9-amd64] Starting task.
[2024-04-11 12:50:10] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-oracle-9-amd64] Finished task in 100.89 seconds.
[2024-04-11 12:50:10] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-centos-7-amd64] Starting task.
[2024-04-11 12:52:05] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-centos-7-amd64] Finished task in 115.00 seconds.
[2024-04-11 12:52:05] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-centos-8-amd64] Starting task.
[2024-04-11 12:54:47] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-centos-8-amd64] Finished task in 162.15 seconds.
[2024-04-11 12:54:47] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-redhat-8-amd64] Starting task.
[2024-04-11 12:57:21] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-redhat-8-amd64] Finished task in 153.83 seconds.
[2024-04-11 12:57:21] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-redhat-9-amd64] Starting task.
[2024-04-11 12:58:58] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-redhat-9-amd64] Finished task in 96.60 seconds.
[2024-04-11 12:58:58] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-ubuntu-18.04-amd64-tests] Starting task.
[2024-04-11 13:02:22] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-ubuntu-18.04-amd64-tests] Finished task in 204.61 seconds.
[2024-04-11 13:02:22] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-ubuntu-20.04-amd64-tests] Starting task.
[2024-04-11 13:07:21] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-ubuntu-20.04-amd64-tests] Finished task in 298.50 seconds.
[2024-04-11 13:07:21] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-ubuntu-22.04-amd64-tests] Starting task.
[2024-04-11 13:10:20] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-ubuntu-22.04-amd64-tests] Finished task in 178.85 seconds.
[2024-04-11 13:10:20] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-debian-10-amd64-tests] Starting task.
[2024-04-11 13:13:13] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-debian-10-amd64-tests] Finished task in 172.73 seconds.
[2024-04-11 13:13:13] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-debian-11-amd64-tests] Starting task.
[2024-04-11 13:16:11] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-debian-11-amd64-tests] Finished task in 178.61 seconds.
[2024-04-11 13:16:11] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-oracle-9-amd64-tests] Starting task.
[2024-04-11 13:19:37] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-oracle-9-amd64-tests] Finished task in 206.21 seconds.
[2024-04-11 13:19:37] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-centos-7-amd64-tests] Starting task.
[2024-04-11 13:22:57] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-centos-7-amd64-tests] Finished task in 199.66 seconds.
[2024-04-11 13:22:57] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-centos-8-amd64-tests] Starting task.
[2024-04-11 13:26:42] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-centos-8-amd64-tests] Finished task in 224.96 seconds.
[2024-04-11 13:26:42] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-redhat-8-amd64-tests] Starting task.
[2024-04-11 13:30:29] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-redhat-8-amd64-tests] Finished task in 226.78 seconds.
[2024-04-11 13:30:29] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-redhat-9-amd64-tests] Starting task.
[2024-04-11 13:33:47] [INFO] [479394] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-redhat-9-amd64-tests] Finished task in 198.50 seconds.
[2024-04-11 13:33:47] [INFO] [479394] [MainThread] [workflow_engine]: Executing Reverse DAG tasks.
[2024-04-11 13:33:47] [INFO] [479394] [MainThread] [workflow_engine]: Executing tasks in parallel.
[2024-04-11 13:33:47] [INFO] [479394] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-ubuntu-18.04-amd64] Starting task.
[2024-04-11 13:34:51] [INFO] [479394] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-ubuntu-18.04-amd64] Finished task in 63.80 seconds.
[2024-04-11 13:34:51] [INFO] [479394] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-ubuntu-20.04-amd64] Starting task.
[2024-04-11 13:36:11] [INFO] [479394] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-ubuntu-20.04-amd64] Finished task in 79.64 seconds.
[2024-04-11 13:36:11] [INFO] [479394] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-ubuntu-22.04-amd64] Starting task.
[2024-04-11 13:37:15] [INFO] [479394] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-ubuntu-22.04-amd64] Finished task in 64.11 seconds.
[2024-04-11 13:37:15] [INFO] [479394] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-debian-10-amd64] Starting task.
[2024-04-11 13:38:19] [INFO] [479394] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-debian-10-amd64] Finished task in 64.01 seconds.
[2024-04-11 13:38:19] [INFO] [479394] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-debian-11-amd64] Starting task.
[2024-04-11 13:39:23] [INFO] [479394] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-debian-11-amd64] Finished task in 64.42 seconds.
[2024-04-11 13:39:23] [INFO] [479394] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-oracle-9-amd64] Starting task.
[2024-04-11 13:40:27] [INFO] [479394] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-oracle-9-amd64] Finished task in 63.75 seconds.
[2024-04-11 13:40:27] [INFO] [479394] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-centos-7-amd64] Starting task.
[2024-04-11 13:41:32] [INFO] [479394] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-centos-7-amd64] Finished task in 64.58 seconds.
[2024-04-11 13:41:32] [INFO] [479394] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-centos-8-amd64] Starting task.
[2024-04-11 13:42:05] [INFO] [479394] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-centos-8-amd64] Finished task in 33.33 seconds.
[2024-04-11 13:42:05] [INFO] [479394] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-redhat-8-amd64] Starting task.
[2024-04-11 13:43:09] [INFO] [479394] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-redhat-8-amd64] Finished task in 63.68 seconds.
[2024-04-11 13:43:09] [INFO] [479394] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-redhat-9-amd64] Starting task.
[2024-04-11 13:44:28] [INFO] [479394] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-redhat-9-amd64] Finished task in 79.46 seconds.
[2024-04-11 13:44:28] [INFO] [479394] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-manager-linux-debian-12-amd64] Starting task.
[2024-04-11 13:45:02] [INFO] [479394] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-manager-linux-debian-12-amd64] Finished task in 33.40 seconds.

1 Manager 13 Agents

[2024-04-11 10:56:48] [INFO] [458665] [MainThread] [workflow_engine]: Executing DAG tasks.
[2024-04-11 10:56:48] [INFO] [458665] [MainThread] [workflow_engine]: Executing tasks in parallel.
[2024-04-11 10:56:48] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-manager-linux-debian-12-amd64] Starting task.
[2024-04-11 10:57:10] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-manager-linux-debian-12-amd64] Finished task in 21.66 seconds.
[2024-04-11 10:57:10] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-ubuntu-18.04-amd64] Starting task.
[2024-04-11 10:57:37] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-ubuntu-18.04-amd64] Finished task in 26.85 seconds.
[2024-04-11 10:57:37] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-ubuntu-20.04-amd64] Starting task.
[2024-04-11 10:57:58] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-ubuntu-20.04-amd64] Finished task in 21.73 seconds.
[2024-04-11 10:57:58] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-ubuntu-22.04-amd64] Starting task.
[2024-04-11 10:58:21] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-ubuntu-22.04-amd64] Finished task in 22.85 seconds.
[2024-04-11 10:58:21] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-debian-10-amd64] Starting task.
[2024-04-11 10:58:43] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-debian-10-amd64] Finished task in 21.59 seconds.
[2024-04-11 10:58:43] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-debian-11-amd64] Starting task.
[2024-04-11 10:59:05] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-debian-11-amd64] Finished task in 21.80 seconds.
[2024-04-11 10:59:05] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-oracle-9-amd64] Starting task.
[2024-04-11 10:59:26] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-oracle-9-amd64] Finished task in 21.12 seconds.
[2024-04-11 10:59:26] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-centos-7-amd64] Starting task.
[2024-04-11 10:59:47] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-centos-7-amd64] Finished task in 21.55 seconds.
[2024-04-11 10:59:47] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-centos-8-amd64] Starting task.
[2024-04-11 11:00:10] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-centos-8-amd64] Finished task in 22.60 seconds.
[2024-04-11 11:00:10] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-redhat-7-amd64] Starting task.
[2024-04-11 11:00:32] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-redhat-7-amd64] Finished task in 22.42 seconds.
[2024-04-11 11:00:32] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-redhat-8-amd64] Starting task.
[2024-04-11 11:00:54] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-redhat-8-amd64] Finished task in 22.00 seconds.
[2024-04-11 11:00:54] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-redhat-9-amd64] Starting task.
[2024-04-11 11:01:16] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-redhat-9-amd64] Finished task in 21.47 seconds.
[2024-04-11 11:01:16] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-amazon-2-amd64] Starting task.
[2024-04-11 11:01:37] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-amazon-2-amd64] Finished task in 21.37 seconds.
[2024-04-11 11:01:37] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-amazon-2023-amd64] Starting task.
[2024-04-11 11:01:59] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-agent-linux-amazon-2023-amd64] Finished task in 21.40 seconds.
[2024-04-11 11:01:59] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-manager-linux-debian-12-amd64] Starting task.
[2024-04-11 11:07:45] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-manager-linux-debian-12-amd64] Finished task in 346.34 seconds.
[2024-04-11 11:07:45] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-ubuntu-18.04-amd64] Starting task.
[2024-04-11 11:09:11] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-ubuntu-18.04-amd64] Finished task in 85.77 seconds.
[2024-04-11 11:09:11] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-ubuntu-20.04-amd64] Starting task.
[2024-04-11 11:10:59] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-ubuntu-20.04-amd64] Finished task in 107.60 seconds.
[2024-04-11 11:10:59] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-ubuntu-22.04-amd64] Starting task.
[2024-04-11 11:12:40] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-ubuntu-22.04-amd64] Finished task in 101.13 seconds.
[2024-04-11 11:12:40] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-debian-10-amd64] Starting task.
[2024-04-11 11:14:14] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-debian-10-amd64] Finished task in 94.35 seconds.
[2024-04-11 11:14:14] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-debian-11-amd64] Starting task.
[2024-04-11 11:16:11] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-debian-11-amd64] Finished task in 116.73 seconds.
[2024-04-11 11:16:11] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-oracle-9-amd64] Starting task.
[2024-04-11 11:17:58] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-oracle-9-amd64] Finished task in 107.32 seconds.
[2024-04-11 11:17:58] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-centos-7-amd64] Starting task.
[2024-04-11 11:19:34] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-centos-7-amd64] Finished task in 96.30 seconds.
[2024-04-11 11:19:34] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-centos-8-amd64] Starting task.
[2024-04-11 11:22:02] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-centos-8-amd64] Finished task in 147.98 seconds.
[2024-04-11 11:22:02] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-redhat-7-amd64] Starting task.
[2024-04-11 11:23:55] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-redhat-7-amd64] Finished task in 112.31 seconds.
[2024-04-11 11:23:55] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-redhat-8-amd64] Starting task.
[2024-04-11 11:26:37] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-redhat-8-amd64] Finished task in 162.27 seconds.
[2024-04-11 11:26:37] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-redhat-9-amd64] Starting task.
[2024-04-11 11:28:20] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-redhat-9-amd64] Finished task in 102.70 seconds.
[2024-04-11 11:28:20] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-amazon-2-amd64] Starting task.
[2024-04-11 11:29:26] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-amazon-2-amd64] Finished task in 65.94 seconds.
[2024-04-11 11:29:26] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-amazon-2023-amd64] Starting task.
[2024-04-11 11:31:30] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-install-linux-amazon-2023-amd64] Finished task in 124.61 seconds.
[2024-04-11 11:31:30] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-ubuntu-18.04-amd64-tests] Starting task.
[2024-04-11 11:34:52] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-ubuntu-18.04-amd64-tests] Finished task in 201.97 seconds.
[2024-04-11 11:34:52] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-ubuntu-20.04-amd64-tests] Starting task.
[2024-04-11 11:38:25] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-ubuntu-20.04-amd64-tests] Finished task in 212.99 seconds.
[2024-04-11 11:38:25] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-ubuntu-22.04-amd64-tests] Starting task.
[2024-04-11 11:41:44] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-ubuntu-22.04-amd64-tests] Finished task in 198.73 seconds.
[2024-04-11 11:41:44] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-debian-10-amd64-tests] Starting task.
[2024-04-11 11:44:37] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-debian-10-amd64-tests] Finished task in 172.98 seconds.
[2024-04-11 11:44:37] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-debian-11-amd64-tests] Starting task.
[2024-04-11 11:47:41] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-debian-11-amd64-tests] Finished task in 183.72 seconds.
[2024-04-11 11:47:41] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-oracle-9-amd64-tests] Starting task.
[2024-04-11 11:51:06] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-oracle-9-amd64-tests] Finished task in 204.77 seconds.
[2024-04-11 11:51:06] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-centos-7-amd64-tests] Starting task.
[2024-04-11 11:54:55] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-centos-7-amd64-tests] Finished task in 228.96 seconds.
[2024-04-11 11:54:55] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-centos-8-amd64-tests] Starting task.
[2024-04-11 11:58:31] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-centos-8-amd64-tests] Finished task in 216.39 seconds.
[2024-04-11 11:58:31] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-redhat-7-amd64-tests] Starting task.
[2024-04-11 12:01:58] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-redhat-7-amd64-tests] Finished task in 207.34 seconds.
[2024-04-11 12:01:58] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-redhat-8-amd64-tests] Starting task.
[2024-04-11 12:05:52] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-redhat-8-amd64-tests] Finished task in 233.61 seconds.
[2024-04-11 12:05:52] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-amazon-2-amd64-tests] Starting task.
[2024-04-11 12:08:55] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-amazon-2-amd64-tests] Finished task in 182.96 seconds.
[2024-04-11 12:08:55] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-amazon-2023-amd64-tests] Starting task.
[2024-04-11 12:12:20] [INFO] [458665] [ThreadPoolExecutor-0_0] [workflow_engine]: [run-agent-linux-amazon-2023-amd64-tests] Finished task in 205.54 seconds.
[2024-04-11 12:12:21] [INFO] [458665] [MainThread] [workflow_engine]: Executing Reverse DAG tasks.
[2024-04-11 12:12:21] [INFO] [458665] [MainThread] [workflow_engine]: Executing tasks in parallel.
[2024-04-11 12:12:21] [INFO] [458665] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-ubuntu-18.04-amd64] Starting task.
[2024-04-11 12:13:09] [INFO] [458665] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-ubuntu-18.04-amd64] Finished task in 48.37 seconds.
[2024-04-11 12:13:09] [INFO] [458665] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-ubuntu-20.04-amd64] Starting task.
[2024-04-11 12:13:42] [INFO] [458665] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-ubuntu-20.04-amd64] Finished task in 33.56 seconds.
[2024-04-11 12:13:42] [INFO] [458665] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-ubuntu-22.04-amd64] Starting task.
[2024-04-11 12:15:17] [INFO] [458665] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-ubuntu-22.04-amd64] Finished task in 94.64 seconds.
[2024-04-11 12:15:17] [INFO] [458665] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-debian-10-amd64] Starting task.
[2024-04-11 12:16:06] [INFO] [458665] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-debian-10-amd64] Finished task in 49.34 seconds.
[2024-04-11 12:16:06] [INFO] [458665] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-debian-11-amd64] Starting task.
[2024-04-11 12:17:25] [INFO] [458665] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-debian-11-amd64] Finished task in 78.94 seconds.
[2024-04-11 12:17:25] [INFO] [458665] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-oracle-9-amd64] Starting task.
[2024-04-11 12:18:30] [INFO] [458665] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-oracle-9-amd64] Finished task in 64.07 seconds.
[2024-04-11 12:18:30] [INFO] [458665] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-centos-7-amd64] Starting task.
[2024-04-11 12:20:04] [INFO] [458665] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-centos-7-amd64] Finished task in 94.73 seconds.
[2024-04-11 12:20:04] [INFO] [458665] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-centos-8-amd64] Starting task.
[2024-04-11 12:21:09] [INFO] [458665] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-centos-8-amd64] Finished task in 64.25 seconds.
[2024-04-11 12:21:09] [INFO] [458665] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-redhat-7-amd64] Starting task.
[2024-04-11 12:22:28] [INFO] [458665] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-redhat-7-amd64] Finished task in 79.54 seconds.
[2024-04-11 12:22:28] [INFO] [458665] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-redhat-8-amd64] Starting task.
[2024-04-11 12:23:32] [INFO] [458665] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-redhat-8-amd64] Finished task in 63.95 seconds.
[2024-04-11 12:23:32] [INFO] [458665] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-redhat-9-amd64] Starting task.
[2024-04-11 12:24:06] [INFO] [458665] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-redhat-9-amd64] Finished task in 33.86 seconds.
[2024-04-11 12:24:06] [INFO] [458665] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-amazon-2-amd64] Starting task.
[2024-04-11 12:24:39] [INFO] [458665] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-amazon-2-amd64] Finished task in 33.21 seconds.
[2024-04-11 12:24:39] [INFO] [458665] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-amazon-2023-amd64] Starting task.
[2024-04-11 12:25:28] [INFO] [458665] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-agent-linux-amazon-2023-amd64] Finished task in 48.52 seconds.
[2024-04-11 12:25:28] [INFO] [458665] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-manager-linux-debian-12-amd64] Starting task.
[2024-04-11 12:26:01] [INFO] [458665] [ThreadPoolExecutor-1_0] [workflow_engine]: [allocate-manager-linux-debian-12-amd64] Finished task in 33.71 seconds.

pro-akim · 2024-04-11T11:03:56Z

Update

YAML

version: 0.1
description: This workflow is used to test agents' deployment for DDT1 PoC
variables:
  manager-os:
    - linux-ubuntu-22.04-amd64
    - linux-ubuntu-18.04-amd64
    - linux-ubuntu-20.04-amd64
    - linux-amazon-2-amd64
    - linux-redhat-7-amd64
    - linux-redhat-8-amd64
    - linux-redhat-9-amd64
    - linux-centos-7-amd64
    - linux-centos-8-amd64
    - linux-oracle-9-amd64
    - linux-debian-10-amd64
    - linux-debian-11-amd64
    - linux-debian-12-amd64
  infra-provider: aws
  working-dir: /tmp/dtt1-poc

tasks:
  # Unique manager allocate task
  - task: "allocate-manager-linux-ubuntu-22.04-amd64"
    description: "Allocate resources for the manager."
    do:
      this: process
      with:
        path: python3
        args:
          - modules/allocation/main.py
          - action: create
          - provider: "{infra-provider}"
          - size: large
          - composite-name: "linux-ubuntu-22.04-amd64"
          - inventory-output: "{working-dir}/manager-linux-ubuntu-22.04-amd64/inventory.yaml"
          - track-output: "{working-dir}/manager-linux-ubuntu-22.04-amd64/track.yaml"
          - ssh-key: "<path_to_Ephemeral_key>"
          - label-termination-date: "1d"
          - label-team: "qa"
    cleanup:
      this: process
      with:
        path: python3
        args:
          - modules/allocation/main.py
          - action: delete
          - track-output: "{working-dir}/manager-linux-ubuntu-22.04-amd64/track.yaml"

  # Unique manager allocate task
  - task: "allocate-manager-linux-ubuntu-18.04-amd64"
    description: "Allocate resources for the manager."
    do:
      this: process
      with:
        path: python3
        args:
          - modules/allocation/main.py
          - action: create
          - provider: "{infra-provider}"
          - size: large
          - composite-name: "linux-ubuntu-18.04-amd64"
          - inventory-output: "{working-dir}/manager-linux-ubuntu-18.04-amd64/inventory.yaml"
          - track-output: "{working-dir}/manager-linux-ubuntu-18.04-amd64/track.yaml"
          - ssh-key: "<path_to_Ephemeral_key>"
          - label-termination-date: "1d"
          - label-team: "qa"
    cleanup:
      this: process
      with:
        path: python3
        args:
          - modules/allocation/main.py
          - action: delete
          - track-output: "{working-dir}/manager-linux-ubuntu-18.04-amd64/track.yaml"

  # Unique manager allocate task
  - task: "allocate-manager-linux-ubuntu-20.04-amd64"
    description: "Allocate resources for the manager."
    do:
      this: process
      with:
        path: python3
        args:
          - modules/allocation/main.py
          - action: create
          - provider: "{infra-provider}"
          - size: large
          - composite-name: "linux-ubuntu-20.04-amd64"
          - inventory-output: "{working-dir}/manager-linux-ubuntu-20.04-amd64/inventory.yaml"
          - track-output: "{working-dir}/manager-linux-ubuntu-20.04-amd64/track.yaml"
          - ssh-key: "<path_to_Ephemeral_key>"
          - label-termination-date: "1d"
          - label-team: "qa"
    cleanup:
      this: process
      with:
        path: python3
        args:
          - modules/allocation/main.py
          - action: delete
          - track-output: "{working-dir}/manager-linux-ubuntu-20.04-amd64/track.yaml"

  # Unique manager allocate task
  - task: "allocate-manager-linux-amazon-2-amd64"
    description: "Allocate resources for the manager."
    do:
      this: process
      with:
        path: python3
        args:
          - modules/allocation/main.py
          - action: create
          - provider: "{infra-provider}"
          - size: large
          - composite-name: "linux-amazon-2-amd64"
          - inventory-output: "{working-dir}/manager-linux-amazon-2-amd64/inventory.yaml"
          - track-output: "{working-dir}/manager-linux-amazon-2-amd64/track.yaml"
          - ssh-key: "<path_to_Ephemeral_key>"
          - label-termination-date: "1d"
          - label-team: "qa"
    cleanup:
      this: process
      with:
        path: python3
        args:
          - modules/allocation/main.py
          - action: delete
          - track-output: "{working-dir}/manager-linux-amazon-2-amd64/track.yaml"

  # Unique manager allocate task
  - task: "allocate-manager-linux-redhat-7-amd64"
    description: "Allocate resources for the manager."
    do:
      this: process
      with:
        path: python3
        args:
          - modules/allocation/main.py
          - action: create
          - provider: "{infra-provider}"
          - size: large
          - composite-name: "linux-redhat-7-amd64"
          - inventory-output: "{working-dir}/manager-linux-redhat-7-amd64/inventory.yaml"
          - track-output: "{working-dir}/manager-linux-redhat-7-amd64/track.yaml"
          - ssh-key: "<path_to_Ephemeral_key>"
          - label-termination-date: "1d"
          - label-team: "qa"
    cleanup:
      this: process
      with:
        path: python3
        args:
          - modules/allocation/main.py
          - action: delete
          - track-output: "{working-dir}/manager-linux-redhat-7-amd64/track.yaml"

  # Unique manager allocate task
  - task: "allocate-manager-linux-redhat-8-amd64"
    description: "Allocate resources for the manager."
    do:
      this: process
      with:
        path: python3
        args:
          - modules/allocation/main.py
          - action: create
          - provider: "{infra-provider}"
          - size: large
          - composite-name: "linux-redhat-8-amd64"
          - inventory-output: "{working-dir}/manager-linux-redhat-8-amd64/inventory.yaml"
          - track-output: "{working-dir}/manager-linux-redhat-8-amd64/track.yaml"
          - ssh-key: "<path_to_Ephemeral_key>"
          - label-termination-date: "1d"
          - label-team: "qa"
    cleanup:
      this: process
      with:
        path: python3
        args:
          - modules/allocation/main.py
          - action: delete
          - track-output: "{working-dir}/manager-linux-redhat-8-amd64/track.yaml"

  # Unique manager allocate task
  - task: "allocate-manager-linux-redhat-9-amd64"
    description: "Allocate resources for the manager."
    do:
      this: process
      with:
        path: python3
        args:
          - modules/allocation/main.py
          - action: create
          - provider: "{infra-provider}"
          - size: large
          - composite-name: "linux-redhat-9-amd64"
          - inventory-output: "{working-dir}/manager-linux-redhat-9-amd64/inventory.yaml"
          - track-output: "{working-dir}/manager-linux-redhat-9-amd64/track.yaml"
          - ssh-key: "<path_to_Ephemeral_key>"
          - label-termination-date: "1d"
          - label-team: "qa"
    cleanup:
      this: process
      with:
        path: python3
        args:
          - modules/allocation/main.py
          - action: delete
          - track-output: "{working-dir}/manager-linux-redhat-9-amd64/track.yaml"

  # Unique manager allocate task
  - task: "allocate-manager-linux-centos-7-amd64"
    description: "Allocate resources for the manager."
    do:
      this: process
      with:
        path: python3
        args:
          - modules/allocation/main.py
          - action: create
          - provider: "{infra-provider}"
          - size: large
          - composite-name: "linux-centos-7-amd64"
          - inventory-output: "{working-dir}/manager-linux-centos-7-amd64/inventory.yaml"
          - track-output: "{working-dir}/manager-linux-centos-7-amd64/track.yaml"
          - ssh-key: "<path_to_Ephemeral_key>"
          - label-termination-date: "1d"
          - label-team: "qa"
    cleanup:
      this: process
      with:
        path: python3
        args:
          - modules/allocation/main.py
          - action: delete
          - track-output: "{working-dir}/manager-linux-centos-7-amd64/track.yaml"

  # Unique manager allocate task
  - task: "allocate-manager-linux-centos-8-amd64"
    description: "Allocate resources for the manager."
    do:
      this: process
      with:
        path: python3
        args:
          - modules/allocation/main.py
          - action: create
          - provider: "{infra-provider}"
          - size: large
          - composite-name: "linux-centos-8-amd64"
          - inventory-output: "{working-dir}/manager-linux-centos-8-amd64/inventory.yaml"
          - track-output: "{working-dir}/manager-linux-centos-8-amd64/track.yaml"
          - ssh-key: "<path_to_Ephemeral_key>"
          - label-termination-date: "1d"
          - label-team: "qa"
    cleanup:
      this: process
      with:
        path: python3
        args:
          - modules/allocation/main.py
          - action: delete
          - track-output: "{working-dir}/manager-linux-centos-8-amd64/track.yaml"

  # Unique manager allocate task
  - task: "allocate-manager-linux-debian-10-amd64"
    description: "Allocate resources for the manager."
    do:
      this: process
      with:
        path: python3
        args:
          - modules/allocation/main.py
          - action: create
          - provider: "{infra-provider}"
          - size: large
          - composite-name: "linux-debian-10-amd64"
          - inventory-output: "{working-dir}/manager-linux-debian-10-amd64/inventory.yaml"
          - track-output: "{working-dir}/manager-linux-debian-10-amd64/track.yaml"
          - ssh-key: "<path_to_Ephemeral_key>"
          - label-termination-date: "1d"
          - label-team: "qa"
    cleanup:
      this: process
      with:
        path: python3
        args:
          - modules/allocation/main.py
          - action: delete
          - track-output: "{working-dir}/manager-linux-debian-10-amd64/track.yaml"

  # Unique manager allocate task
  - task: "allocate-manager-linux-debian-11-amd64"
    description: "Allocate resources for the manager."
    do:
      this: process
      with:
        path: python3
        args:
          - modules/allocation/main.py
          - action: create
          - provider: "{infra-provider}"
          - size: large
          - composite-name: "linux-debian-11-amd64"
          - inventory-output: "{working-dir}/manager-linux-debian-11-amd64/inventory.yaml"
          - track-output: "{working-dir}/manager-linux-debian-11-amd64/track.yaml"
          - ssh-key: "<path_to_Ephemeral_key>"
          - label-termination-date: "1d"
          - label-team: "qa"
    cleanup:
      this: process
      with:
        path: python3
        args:
          - modules/allocation/main.py
          - action: delete
          - track-output: "{working-dir}/manager-linux-debian-11-amd64/track.yaml"

  # Unique manager allocate task
  - task: "allocate-manager-linux-debian-12-amd64"
    description: "Allocate resources for the manager."
    do:
      this: process
      with:
        path: python3
        args:
          - modules/allocation/main.py
          - action: create
          - provider: "{infra-provider}"
          - size: large
          - composite-name: "linux-debian-12-amd64"
          - inventory-output: "{working-dir}/manager-linux-debian-12-amd64/inventory.yaml"
          - track-output: "{working-dir}/manager-linux-debian-12-amd64/track.yaml"
          - ssh-key: "<path_to_Ephemeral_key>"
          - label-termination-date: "1d"
          - label-team: "qa"
    cleanup:
      this: process
      with:
        path: python3
        args:
          - modules/allocation/main.py
          - action: delete
          - track-output: "{working-dir}/manager-linux-debian-12-amd64/track.yaml"

  # Unique manager allocate task
  - task: "allocate-manager-linux-oracle-9-amd64"
    description: "Allocate resources for the manager."
    do:
      this: process
      with:
        path: python3
        args:
          - modules/allocation/main.py
          - action: create
          - provider: "{infra-provider}"
          - size: large
          - composite-name: "linux-oracle-9-amd64"
          - inventory-output: "{working-dir}/manager-linux-oracle-9-amd64/inventory.yaml"
          - track-output: "{working-dir}/manager-linux-oracle-9-amd64/track.yaml"
          - ssh-key: "<path_to_Ephemeral_key>"
          - label-termination-date: "1d"
          - label-team: "qa"
    cleanup:
      this: process
      with:
        path: python3
        args:
          - modules/allocation/main.py
          - action: delete
          - track-output: "{working-dir}/manager-linux-oracle-9-amd64/track.yaml"

  # Unique manager provision task
  - task: "provision-manager-linux-oracle-9-amd64"
    description: "Provision the manager."
    do:
      this: process
      with:
        path: python3
        args:
          - modules/provision/main.py
          - inventory: "{working-dir}/manager-linux-oracle-9-amd64/inventory.yaml"
          - install:
            - component: tar
    depends-on:
      - "allocate-manager-linux-oracle-9-amd64"
    on-error: "abort-all"

  # Generic manager test task
  - task: "run-manager-tests"
    description: "Run tests install for the manager."
    do:
      this: process
      with:
        path: python3
        args:
          - modules/testing/main.py
          - targets:
            - wazuh-1: "{working-dir}/manager-linux-ubuntu-22.04-amd64/inventory.yaml"
            - wazuh-2: "{working-dir}/manager-linux-ubuntu-18.04-amd64/inventory.yaml"
            - wazuh-3: "{working-dir}/manager-linux-ubuntu-20.04-amd64/inventory.yaml"
            - wazuh-4: "{working-dir}/manager-linux-amazon-2-amd64/inventory.yaml"
            - wazuh-5: "{working-dir}/manager-linux-redhat-7-amd64/inventory.yaml"
            - wazuh-6: "{working-dir}/manager-linux-redhat-8-amd64/inventory.yaml"
            - wazuh-7: "{working-dir}/manager-linux-redhat-9-amd64/inventory.yaml"
            - wazuh-8: "{working-dir}/manager-linux-centos-7-amd64/inventory.yaml"
            - wazuh-9: "{working-dir}/manager-linux-centos-8-amd64/inventory.yaml"
            - wazuh-10: "{working-dir}/manager-linux-oracle-9-amd64/inventory.yaml"
            - wazuh-11: "{working-dir}/manager-linux-debian-10-amd64/inventory.yaml"
            - wazuh-12: "{working-dir}/manager-linux-debian-11-amd64/inventory.yaml"
            - wazuh-13: "{working-dir}/manager-linux-debian-12-amd64/inventory.yaml"
          - tests: "install,restart,stop,uninstall"
          - component: "manager"
          - wazuh-version: "4.7.3"
          - wazuh-revision: "40714"
          - live: "True"
    depends-on:
      - "provision-manager-linux-oracle-9-amd64"
      - "allocate-manager-linux-ubuntu-22.04-amd64"
      - "allocate-manager-linux-ubuntu-18.04-amd64"
      - "allocate-manager-linux-ubuntu-20.04-amd64"
      - "allocate-manager-linux-amazon-2-amd64"
      - "allocate-manager-linux-redhat-7-amd64"
      - "allocate-manager-linux-redhat-8-amd64"
      - "allocate-manager-linux-redhat-9-amd64"
      - "allocate-manager-linux-centos-7-amd64"
      - "allocate-manager-linux-centos-8-amd64"
      - "allocate-manager-linux-oracle-9-amd64"
      - "allocate-manager-linux-debian-10-amd64"
      - "allocate-manager-linux-debian-11-amd64"
      - "allocate-manager-linux-debian-12-amd64"
      - "allocate-manager-linux-oracle-9-amd64"

N Manager	Allocation (Time)	Provision (Time)	Testing (Time)	Clean-up (Time)	Total (Time)
1	--	--	--	--	--
6	--	--	--	--	--
10	--	--	--	--	--
13	--	--	--	--	--

1 Managers 0 Agents

6 Managers 0 Agents

10 Managers 0 Agents

13 Managers 0 Agents

@pro-akim I modified the YAML input to fix some errors, I updated it, and here are the results:

workflow.log
Total time: 16:18 -> 17:52 => 1h 34'

pro-akim · 2024-04-11T15:48:13Z

Update

Some error found in the provision:

[2024-04-11 17:44:00] [DEBUG] [585081] [ThreadPoolExecutor-0_0] [workflow_engine]: Running task "provision-manager-linux-oracle-9-amd64" with arguments: ['modules/provision/main.py', '--inventory=/tmp/dtt1-poc/manager-linux-oracle-9-amd64/inventory.yaml', "--install={'component': 'tar'}"]
[2024-04-11 17:44:03] [DEBUG] [585081] [ThreadPoolExecutor-0_0] [workflow_engine]: Finished task "provision-manager-linux-oracle-9-amd64" execution with result:
[2024-04-11 17:44:00] [DEBUG] ANSIBLE: Using inventory: {'all': {'hosts': {'ec2-52-23-206-224.compute-1.amazonaws.com': {'ansible_port': 2200, 'ansible_user': 'ec2-user', 'ansible_ssh_private_key_file': '/home/akim/Desktop/personal/Ephemeral'}}}}
[2024-04-11 17:44:00] [DEBUG] ANSIBLE: Running playbook: {'hosts': 'ec2-52-23-206-224.compute-1.amazonaws.com', 'become': True, 'gather_facts': True, 'tasks': [{'name': 'Capture ansible_os_family', 'set_fact': {'ansible_os_family': "{{ ansible_facts['distribution_file_variety'] }}", 'cacheable': 'yes'}}]}
No config file found; using defaults

PLAY [ec2-52-23-206-224.compute-1.amazonaws.com] *******************************

TASK [Gathering Facts] *********************************************************
fatal: [ec2-52-23-206-224.compute-1.amazonaws.com]: UNREACHABLE! => changed=false 
  msg: 'Failed to connect to the host via ssh: ssh: connect to host ec2-52-23-206-224.compute-1.amazonaws.com port 2200: No route to host'
  unreachable: true

PLAY RECAP *********************************************************************
ec2-52-23-206-224.compute-1.amazonaws.com : ok=0    changed=0    unreachable=1    failed=0    skipped=0    rescued=0    ignored=0   
[2024-04-11 17:44:02] [DEBUG] ANSIBLE: Playbook {'hosts': 'ec2-52-23-206-224.compute-1.amazonaws.com', 'become': True, 'gather_facts': True, 'tasks': [{'name': 'Capture ansible_os_family', 'set_fact': {'ansible_os_family': "{{ ansible_facts['distribution_file_variety'] }}", 'cacheable': 'yes'}}]} finished with status {'skipped': {}, 'ok': {}, 'dark': {'ec2-52-23-206-224.compute-1.amazonaws.com': 1}, 'failures': {}, 'ignored': {}, 'rescued': {}, 'processed': {'ec2-52-23-206-224.compute-1.amazonaws.com': 1}, 'changed': {}}
[2024-04-11 17:44:02] [DEBUG] ANSIBLE: Templates found: ['install.j2']
[2024-04-11 17:44:02] [DEBUG] ANSIBLE: Rendering template install.j2
[2024-04-11 17:44:02] [WARNING] ANSIBLE: Template install.j2 not rendered
[2024-04-11 17:44:02] [DEBUG] ANSIBLE: []
[2024-04-11 17:44:02] [DEBUG] ANSIBLE: Using inventory: {'all': {'hosts': {'ec2-52-23-206-224.compute-1.amazonaws.com': {'ansible_port': 2200, 'ansible_user': 'ec2-user', 'ansible_ssh_private_key_file': '/home/akim/Desktop/personal/Ephemeral'}}}}
[2024-04-11 17:44:02] [DEBUG] ANSIBLE: Running playbook: {'hosts': 'ec2-52-23-206-224.compute-1.amazonaws.com', 'become': True, 'gather_facts': True, 'tasks': []}
No config file found; using defaults

PLAY [ec2-52-23-206-224.compute-1.amazonaws.com] *******************************

TASK [Gathering Facts] *********************************************************
fatal: [ec2-52-23-206-224.compute-1.amazonaws.com]: UNREACHABLE! => changed=false 
  msg: 'Failed to connect to the host via ssh: ssh: connect to host ec2-52-23-206-224.compute-1.amazonaws.com port 2200: No route to host'
  unreachable: true

PLAY RECAP *********************************************************************
ec2-52-23-206-224.compute-1.amazonaws.com : ok=0    changed=0    unreachable=1    failed=0    skipped=0    rescued=0    ignored=0   
[2024-04-11 17:44:03] [DEBUG] ANSIBLE: Playbook {'hosts': 'ec2-52-23-206-224.compute-1.amazonaws.com', 'become': True, 'gather_facts': True, 'tasks': []} finished with status {'skipped': {}, 'ok': {}, 'dark': {'ec2-52-23-206-224.compute-1.amazonaws.com': 1}, 'failures': {}, 'ignored': {}, 'rescued': {}, 'processed': {'ec2-52-23-206-224.compute-1.amazonaws.com': 1}, 'changed': {}}

[2024-04-11 17:44:03] [INFO] [585081] [ThreadPoolExecutor-0_0] [workflow_engine]: [provision-manager-linux-oracle-9-amd64] Finished task in 2.86 seconds.

[root@ip-172-31-90-222 ec2-user]# uname -a
Linux ip-172-31-90-222.ec2.internal 5.15.0-101.103.2.1.el9uek.x86_64 #2 SMP Tue May 2 01:10:45 PDT 2023 x86_64 x86_64 x86_64 GNU/Linux
[root@ip-172-31-90-222 ec2-user]# tar --version
bash: tar: command not found

Note

The EC2 is being provisioned but it is still not approachable by the provision, this error is related to that unavailability.

The Allocation module will have to check the EC2 before moving to the next step, then this issue will be blocked until that update.

Related issue: #5198

rauldpm · 2024-04-15T13:00:57Z

After the daily meeting today, it has been discussed the time results, these comment result times include the stack (Wazuh manager, Filebeat, Wazuh dashboard, and Wazuh indexer) but the Testing module is only testing the Wazuh manager

The original idea of this issue was to check the flow using the Wazuh manager and Wazuh agent, we need to review this behavior as the flow should allow provisioning of the stack components individually, for example:

Use only a Wazuh manager and 1 Wazuh agent
Use only a Wazuh manager and N Wazuh agent
Use a Wazuh manager cluster (N nodes) with 1 agent
Use a Wazuh manager cluster (N nodes) with M agent

Those cases should contemplate installing the Filebeat, Wazuh dashboard, and Wazuh indexer (single or cluster), so the provision should generate certificates and copy them between nodes, also, the AIO deployment should be considered

If any of those cases are not contemplated in the current development, we should create an issue and discuss if it should be added for DTT1 or DTT2

rauldpm · 2024-04-16T22:12:30Z

Moved this issue to DTT1 as we need to test the threading fix

rauldpm · 2024-04-17T20:47:01Z

Updated #5191 (comment) with AWS results

Due to the thread bug issue, this issue will be edited so it reports the following data:

Vagrant concurrent tests
AWS concurrent tests
Vagrant parallel tests
AWS parallels tests

The parallel tests will have to report the same test cases (1-N, M-1) but for multiple threads {2, 4, 6, 8, 10), with this data, we will be able to determine which AWS instance type we need for the Jenkins CI

Blocked as internal due to We need to finish all other issues to have stable data

rauldpm · 2024-04-23T12:08:11Z

Moved ETA to 30/04/2024 as we need all DTT1 changes to test it correctly (main ETA 29/04/2024)

rauldpm · 2024-04-25T13:11:26Z

Moved ETA to 02/05/2024 as we need time to test the main branch: #4495 (comment)

mhamra · 2024-05-08T13:50:18Z

TEST PLAN - ETA May 15

Testing Plan Guideline

Tests to execute
- Agents: We will test one Wazuh manager with all available agent combinations of architectures (AMD, ARM)/distributions (DEB, RPM)/OS (Linux, Windows, MacOS). We will execute all available test sets for the agent.
- Managers: We will test all available manager architectures/OS combinations.
- AIO: We will test all available manager architectures/OS combinations.
No ARM MacOS test will be executed.
The workflow files of the tests will use the AWS provider only; the workflows will not use the Vagrant provider.
The tests will execute separate runs setting the workflow's --thread parameter with values 2, 3, 4 (parallel threads)

Test mix table and time metrics

Test	Threads	Allocation (Time)	Provision (Time)	Testing (Time)	Clean-up (Time)	Total (Time)
Agents	2	--	--	--	--	--
	3	--	--	--	--	--
	4	--	--	--	--	--
Managers	2	--	--	--	--
	3	--	--	--	--
	4	--	--	--	--
AIO	2	--	--	--	--	--
	3	--	--	--	--	--
	4	--	--	--	--	--

Workflow files

AGENTS

Test Agents workflow file

TBD

PLATFORMS:

linux-redhat-7-amd64
linux-redhat-7-arm64
linux-redhat-8-amd64
linux-redhat-8-arm64
linux-redhat-9-amd64
linux-redhat-9-arm64
linux-centos-7-amd64
linux-centos-7-arm64
linux-centos-8-amd64
linux-centos-8-arm64
linux-debian-10-amd64
linux-debian-10-arm64
linux-debian-11-amd64
linux-debian-11-arm64
linux-debian-12-amd64
linux-debian-12-arm64
linux-ubuntu-22.04-amd64
linux-ubuntu-22.04-arm64
linux-ubuntu-18.04-amd64
linux-ubuntu-18.04-arm64
linux-ubuntu-20.04-amd64
linux-ubuntu-20.04-arm64
linux-oracle-9-amd64
linux-opensuse-15-amd64
linux-amazon-2-amd64
linux-amazon-2-arm64
linux-amazon-2023-amd64
linux-amazon-2023-arm64
windows-desktop-10-amd64
windows-server-2012r2-amd64
windows-server-2016-amd64
windows-server-2019-amd64
windows-server-2022-amd64
macos-ventura-13.4.1-amd64
macos-sonoma-14.4.1-amd64

Agent platform not included in the tests

Windows ARM
MacOS ARM.
Oracle Linux 9 arm
Opensuse 15 arm

MANAGERS

Test Managers workflow file

TBD

PLATFORMS: (TBD)

PLAT 1
PLAT 2
ETC

AIO

Test AIO workflow file

TBD

PLATFORMS: (TBD)

PLAT 1
PLAT 2
ETC

rauldpm added level/task Task issue type/research labels Apr 9, 2024

rauldpm assigned mhamra, fcaffieri and pro-akim Apr 9, 2024

rauldpm mentioned this issue Apr 11, 2024

DTT1 - Testing - Allow single Wazuh manager test #5202

Closed

rauldpm changed the title ~~Post DTT1 - Execution time analysis~~ DTT1 - Execution time analysis Apr 16, 2024

rauldpm mentioned this issue Apr 22, 2024

DTT1 - Test Module - Improve test module execution time #5221

Closed

1 task

rauldpm mentioned this issue Apr 25, 2024

Deployability testing tier 1 #4495

Open

11 tasks

This was referenced May 8, 2024

DTT1-Testing Module. Fix arm installation test #5352

Closed

DTT1 - Testing module. Agent's OS version check procedure. #5359

Closed

rauldpm self-assigned this May 9, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DTT1 - Execution time analysis #5191

DTT1 - Execution time analysis #5191

rauldpm commented Apr 9, 2024 •

edited

pro-akim commented Apr 9, 2024 •

edited

pro-akim commented Apr 10, 2024 •

edited

pro-akim commented Apr 10, 2024

pro-akim commented Apr 11, 2024 •

edited

pro-akim commented Apr 11, 2024 •

edited by rauldpm

pro-akim commented Apr 11, 2024 •

edited by rauldpm

rauldpm commented Apr 15, 2024

rauldpm commented Apr 16, 2024

rauldpm commented Apr 17, 2024 •

edited

rauldpm commented Apr 23, 2024

rauldpm commented Apr 25, 2024

mhamra commented May 8, 2024 •

edited

DTT1 - Execution time analysis #5191

DTT1 - Execution time analysis #5191

Comments

rauldpm commented Apr 9, 2024 • edited

Description

Related:

Tasks

Deliverable 1 - Concurrence

Deliverable 2 - Multithreading

2 threads

4 threads

6 threads

8 threads

10 threads

pro-akim commented Apr 9, 2024 • edited

Update

pro-akim commented Apr 10, 2024 • edited

Update

1. Analyze results of issue #5125 and provide times in a single comment

2. Launch new battery of test after fixing the bugs #5125 (comment)

3. Central components tests

Note:

Antonio Localhost

Federico Localhost

pro-akim commented Apr 10, 2024

Update

pro-akim commented Apr 11, 2024 • edited

Update

pro-akim commented Apr 11, 2024 • edited by rauldpm

Update

pro-akim commented Apr 11, 2024 • edited by rauldpm

Update

Note

rauldpm commented Apr 15, 2024

rauldpm commented Apr 16, 2024

rauldpm commented Apr 17, 2024 • edited

rauldpm commented Apr 23, 2024

rauldpm commented Apr 25, 2024

mhamra commented May 8, 2024 • edited

TEST PLAN - ETA May 15

Testing Plan Guideline

Test mix table and time metrics

Workflow files

AGENTS

MANAGERS

AIO

rauldpm commented Apr 9, 2024 •

edited

pro-akim commented Apr 9, 2024 •

edited

pro-akim commented Apr 10, 2024 •

edited

pro-akim commented Apr 11, 2024 •

edited

pro-akim commented Apr 11, 2024 •

edited by rauldpm

pro-akim commented Apr 11, 2024 •

edited by rauldpm

rauldpm commented Apr 17, 2024 •

edited

mhamra commented May 8, 2024 •

edited