Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DTT1 - Iteration 3 - Allocation module - Implement name and type labels on VMs creation #4866

Closed
2 tasks done
Tracked by #4855
QU3B1M opened this issue Jan 16, 2024 · 5 comments · Fixed by #4967
Closed
2 tasks done
Tracked by #4855

DTT1 - Iteration 3 - Allocation module - Implement name and type labels on VMs creation #4866

QU3B1M opened this issue Jan 16, 2024 · 5 comments · Fixed by #4967
Assignees
@c-bordon
Labels
level/task Task issue type/enhancement

Comments

@QU3B1M
Copy link
Member

QU3B1M commented Jan 16, 2024 •
edited by c-bordon
Loading

Epic #4495

Description

This issue aims to add name and type label on the created VM to improve the tracking (Vagrant will only include the name as it doesnt have labels).

Tasks

  • Add name, team, termination_date, and issue labels on AWS VMs
  • Add name on Vagrant VMs
@QU3B1M QU3B1M mentioned this issue Jan 16, 2024
Closed
19 tasks
@QU3B1M QU3B1M mentioned this issue Feb 1, 2024
Merged
@c-bordon c-bordon self-assigned this Feb 15, 2024
@c-bordon
Copy link
Member

c-bordon commented Feb 15, 2024 •
edited
Loading

Vagrant

In this PR, the name assigned in VirtualBox was modified so that this is the ID of the instance, and in this way, we can track the created VM in the same way:

cbordon@cbordon-MS-7C88:~/Documents/wazuh/repositorios/wazuh-qa/deployability$ python3 launchers/allocation.py --composite-name linux-ubuntu-22.04-amd64 --provider vagrant --size micro --ssh-key ~/.ssh/allocation_test.pub
[2024-02-15 12:26:04] [INFO] ALLOCATOR: Creating instance at /tmp/wazuh-qa
[2024-02-15 12:26:04] [DEBUG] ALLOCATOR: No config provided. Generating from payload
[2024-02-15 12:26:04] [DEBUG] ALLOCATOR: Using provided key pair
[2024-02-15 12:26:07] [DEBUG] ALLOCATOR: Vagrantfile created. Creating instance.
[2024-02-15 12:26:07] [INFO] ALLOCATOR: Instance VAGRANT-C0B4F997-4B42-48D4-B09C-079FCFA80925 created.
[2024-02-15 12:26:48] [INFO] ALLOCATOR: Instance VAGRANT-C0B4F997-4B42-48D4-B09C-079FCFA80925 started.
[2024-02-15 12:26:50] [DEBUG] ALLOCATOR: Using provided credentials
[2024-02-15 12:26:50] [INFO] ALLOCATOR: Inventory file generated at /tmp/wazuh-qa/inventory.yml
[2024-02-15 12:26:50] [INFO] ALLOCATOR: Track file generated at /tmp/wazuh-qa/track.yml
cbordon@cbordon-MS-7C88:~/Documents/wazuh/repositorios/wazuh-qa/deployability$ vagrant global-status
id       name    provider   state   directory                                                  
-----------------------------------------------------------------------------------------------
71dd6b2  default virtualbox running /tmp/wazuh-qa/VAGRANT-73AFD5D8-20E2-450F-88D1-8FE850548050 
4a78764  default virtualbox running /tmp/wazuh-qa/VAGRANT-C0B4F997-4B42-48D4-B09C-079FCFA80925

Screenshot_20240215_123751

@c-bordon
Copy link
Member

AWS

Error messages for labels:

without team label
cbordon@cbordon-MS-7C88:~/Documents/wazuh/repositorios/wazuh-qa/deployability$ python3 launchers/allocation.py --composite-name linux-ubuntu-22.04-amd64 --provider aws --size micro --ssh-key ~/.ssh/allocation_test.pub --label https://github.com/wazuh/internal-devel-requests/issues/775 --label "2024-02-25 12:00:00"
[2024-02-15 16:03:07] [INFO] ALLOCATOR: Creating instance at /tmp/wazuh-qa
[2024-02-15 16:03:07] [DEBUG] ALLOCATOR: No config provided. Generating from payload
[2024-02-15 16:03:07] [ERROR] ALLOCATOR: No team label provided.
without termination date label
cbordon@cbordon-MS-7C88:~/Documents/wazuh/repositorios/wazuh-qa/deployability$ python3 launchers/allocation.py --composite-name linux-ubuntu-22.04-amd64 --provider aws --size micro --ssh-key ~/.ssh/allocation_test.pub --label https://github.com/wazuh/internal-devel-requests/issues/775 --label core
[2024-02-15 16:16:30] [INFO] ALLOCATOR: Creating instance at /tmp/wazuh-qa
[2024-02-15 16:16:31] [DEBUG] ALLOCATOR: No config provided. Generating from payload
[2024-02-15 16:16:31] [ERROR] ALLOCATOR: The termination_date label was not provided or is of incorrect format, example: 2024-02-25 12:00:00.
invalid date format
cbordon@cbordon-MS-7C88:~/Documents/wazuh/repositorios/wazuh-qa/deployability$ python3 launchers/allocation.py --composite-name linux-ubuntu-22.04-amd64 --provider aws --size micro --ssh-key ~/.ssh/allocation_test.pub --label https://github.com/wazuh/internal-devel-requests/issues/775 --label core --label "2024-02-25"
[2024-02-15 16:14:14] [INFO] ALLOCATOR: Creating instance at /tmp/wazuh-qa
[2024-02-15 16:14:14] [DEBUG] ALLOCATOR: No config provided. Generating from payload
[2024-02-15 16:14:14] [ERROR] ALLOCATOR: The termination_date label was not provided or is of incorrect format, example: 2024-02-25 12:00:00.
cbordon@cbordon-MS-7C88:~/Documents/wazuh/repositorios/wazuh-qa/deployability$ python3 launchers/allocation.py --composite-name linux-ubuntu-22.04-amd64 --provider aws --size micro --ssh-key ~/.ssh/allocation_test.pub --label https://github.com/wazuh/internal-devel-requests/issues/775 --label core --label "24/02/2024"
[2024-02-15 16:14:21] [INFO] ALLOCATOR: Creating instance at /tmp/wazuh-qa
[2024-02-15 16:14:21] [DEBUG] ALLOCATOR: No config provided. Generating from payload
[2024-02-15 16:14:21] [ERROR] ALLOCATOR: The termination_date label was not provided or is of incorrect format, example: 2024-02-25 12:00:00.
cbordon@cbordon-MS-7C88:~/Documents/wazuh/repositorios/wazuh-qa/deployability$ python3 launchers/allocation.py --composite-name linux-ubuntu-22.04-amd64 --provider aws --size micro --ssh-key ~/.ssh/allocation_test.pub --label https://github.com/wazuh/internal-devel-requests/issues/775 --label core --label "24-02-2024"
[2024-02-15 16:14:25] [INFO] ALLOCATOR: Creating instance at /tmp/wazuh-qa
[2024-02-15 16:14:25] [DEBUG] ALLOCATOR: No config provided. Generating from payload
[2024-02-15 16:14:25] [ERROR] ALLOCATOR: The termination_date label was not provided or is of incorrect format, example: 2024-02-25 12:00:00.
cbordon@cbordon-MS-7C88:~/Documents/wazuh/repositorios/wazuh-qa/deployability$ python3 launchers/allocation.py --composite-name linux-ubuntu-22.04-amd64 --provider aws --size micro --ssh-key ~/.ssh/allocation_test.pub --label https://github.com/wazuh/internal-devel-requests/issues/775 --label core --label "24-02-2024 11:00:00"
[2024-02-15 16:14:31] [INFO] ALLOCATOR: Creating instance at /tmp/wazuh-qa
[2024-02-15 16:14:31] [DEBUG] ALLOCATOR: No config provided. Generating from payload
[2024-02-15 16:14:31] [ERROR] ALLOCATOR: The termination_date label was not provided or is of incorrect format, example: 2024-02-25 12:00:00.
without issue label
cbordon@cbordon-MS-7C88:~/Documents/wazuh/repositorios/wazuh-qa/deployability$ python3 launchers/allocation.py --composite-name linux-ubuntu-22.04-amd64 --provider aws --size micro --ssh-key ~/.ssh/allocation_test.pub  --label core --label "2024-02-25 12:00:00"
[2024-02-15 16:04:32] [INFO] ALLOCATOR: Creating instance at /tmp/wazuh-qa
[2024-02-15 16:04:32] [DEBUG] ALLOCATOR: No config provided. Generating from payload
[2024-02-15 16:04:32] [ERROR] ALLOCATOR: No issue label provided.

@c-bordon
Copy link
Member

AWS

Creation with labels:

cbordon@cbordon-MS-7C88:~/Documents/wazuh/repositorios/wazuh-qa/deployability$ python3 launchers/allocation.py --composite-name linux-ubuntu-22.04-amd64 --provider aws --size micro --ssh-key ~/.ssh/allocation_test.pub --label https://github.com/wazuh/internal-devel-requests/issues/775 --label core --label "2024-02-25 12:00:00"
[2024-02-15 16:48:59] [INFO] ALLOCATOR: Creating instance at /tmp/wazuh-qa
[2024-02-15 16:48:59] [DEBUG] ALLOCATOR: No config provided. Generating from payload
[2024-02-15 16:48:59] [DEBUG] ALLOCATOR: Using provided key pair
[2024-02-15 16:49:00] [DEBUG] ALLOCATOR: Creating temp directory: /tmp/wazuh-qa/AWS-31749B51-8013-48BC-899A-3BD921DA95F2
[2024-02-15 16:49:18] [DEBUG] ALLOCATOR: Renaming temp /tmp/wazuh-qa/AWS-31749B51-8013-48BC-899A-3BD921DA95F2 directory to /tmp/wazuh-qa/i-023dd5e5f6583685e
[2024-02-15 16:49:18] [INFO] ALLOCATOR: Instance i-023dd5e5f6583685e created.
[2024-02-15 16:49:20] [INFO] ALLOCATOR: Instance i-023dd5e5f6583685e started.
[2024-02-15 16:49:20] [INFO] ALLOCATOR: Inventory file generated at /tmp/wazuh-qa/inventory.yml
[2024-02-15 16:49:20] [INFO] ALLOCATOR: Track file generated at /tmp/wazuh-qa/track.yml

Screenshot_20240215_165054

@c-bordon
Copy link
Member

Vagrant

test without labels:

cbordon@cbordon-MS-7C88:~/Documents/wazuh/repositorios/wazuh-qa/deployability$ python3 launchers/allocation.py --composite-name linux-ubuntu-22.04-amd64 --provider vagrant --size micro
[2024-02-15 16:59:28] [INFO] ALLOCATOR: Creating instance at /tmp/wazuh-qa
[2024-02-15 16:59:28] [DEBUG] ALLOCATOR: No config provided. Generating from payload
[2024-02-15 16:59:28] [DEBUG] ALLOCATOR: Generating new key pair
[2024-02-15 16:59:33] [DEBUG] ALLOCATOR: Vagrantfile created. Creating instance.
[2024-02-15 16:59:33] [INFO] ALLOCATOR: Instance VAGRANT-20141D96-AFC4-405B-A029-1A6090FF526D created.
[2024-02-15 17:00:12] [INFO] ALLOCATOR: Instance VAGRANT-20141D96-AFC4-405B-A029-1A6090FF526D started.
[2024-02-15 17:00:15] [DEBUG] ALLOCATOR: Using provided credentials
[2024-02-15 17:00:15] [INFO] ALLOCATOR: Inventory file generated at /tmp/wazuh-qa/inventory.yml
[2024-02-15 17:00:15] [INFO] ALLOCATOR: Track file generated at /tmp/wazuh-qa/track.yml
cbordon@cbordon-MS-7C88:~/Documents/wazuh/repositorios/wazuh-qa/deployability$ python3 launchers/allocation.py --provider aws --action delete --track-output /tmp/wazuh-qa/track.yml 
[2024-02-15 17:00:36] [INFO] ALLOCATOR: Deleting instance from trackfile /tmp/wazuh-qa/track.yml
[2024-02-15 17:00:36] [DEBUG] ALLOCATOR: Destroying instance VAGRANT-20141D96-AFC4-405B-A029-1A6090FF526D
[2024-02-15 17:00:41] [INFO] ALLOCATOR: Instance VAGRANT-20141D96-AFC4-405B-A029-1A6090FF526D deleted.

with labels:

cbordon@cbordon-MS-7C88:~/Documents/wazuh/repositorios/wazuh-qa/deployability$ python3 launchers/allocation.py --composite-name linux-ubuntu-22.04-amd64 --provider vagrant --size micro --label https://github.com/wazuh/internal-devel-requests/issues/775 --label core --label "2024-02-25 12:00:00"
[2024-02-15 16:56:06] [INFO] ALLOCATOR: Creating instance at /tmp/wazuh-qa
[2024-02-15 16:56:06] [DEBUG] ALLOCATOR: No config provided. Generating from payload
[2024-02-15 16:56:06] [DEBUG] ALLOCATOR: Generating new key pair
[2024-02-15 16:56:10] [DEBUG] ALLOCATOR: Vagrantfile created. Creating instance.
[2024-02-15 16:56:10] [INFO] ALLOCATOR: Instance VAGRANT-5F8B4E03-320E-4558-9E99-0A2FF37FA536 created.
[2024-02-15 16:56:52] [INFO] ALLOCATOR: Instance VAGRANT-5F8B4E03-320E-4558-9E99-0A2FF37FA536 started.
[2024-02-15 16:56:55] [DEBUG] ALLOCATOR: Using provided credentials
[2024-02-15 16:56:55] [INFO] ALLOCATOR: Inventory file generated at /tmp/wazuh-qa/inventory.yml
[2024-02-15 16:56:55] [INFO] ALLOCATOR: Track file generated at /tmp/wazuh-qa/track.yml
cbordon@cbordon-MS-7C88:~/Documents/wazuh/repositorios/wazuh-qa/deployability$ cat /tmp/wazuh-qa/inventory.yml
ansible_host: 192.168.57.2
ansible_port: 22
ansible_ssh_private_key_file: /tmp/wazuh-qa/VAGRANT-5F8B4E03-320E-4558-9E99-0A2FF37FA536/instance_key
ansible_user: vagrant
cbordon@cbordon-MS-7C88:~/Documents/wazuh/repositorios/wazuh-qa/deployability$ ssh -i /tmp/wazuh-qa/VAGRANT-5F8B4E03-320E-4558-9E99-0A2FF37FA536/instance_key vagrant@192.168.57.2
The authenticity of host '192.168.57.2 (192.168.57.2)' can't be established.
ED25519 key fingerprint is SHA256:iGDG1orbuELliV8WDlb4znS2dAb2p22p8KgLZmHfhzQ.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '192.168.57.2' (ED25519) to the list of known hosts.
Welcome to Ubuntu 22.04.3 LTS (GNU/Linux 5.15.0-92-generic x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/pro

  System information as of Thu Feb 15 19:59:03 UTC 2024

  System load:  0.05615234375     Processes:               93
  Usage of /:   3.7% of 38.70GB   Users logged in:         0
  Memory usage: 21%               IPv4 address for enp0s3: 10.0.2.15
  Swap usage:   0%                IPv4 address for enp0s8: 192.168.57.2


Expanded Security Maintenance for Applications is not enabled.

0 updates can be applied immediately.

Enable ESM Apps to receive additional future security updates.
See https://ubuntu.com/esm or run: sudo pro status


The list of available updates is more than a week old.
To check for new updates run: sudo apt update

vagrant@ubuntu-jammy:~$ exit
logout
Connection to 192.168.57.2 closed.
cbordon@cbordon-MS-7C88:~/Documents/wazuh/repositorios/wazuh-qa/deployability$ python3 launchers/allocation.py --provider aws --action delete --track-output /tmp/wazuh-qa/track.yml [2024-02-15 16:59:12] [INFO] ALLOCATOR: Deleting instance from trackfile /tmp/wazuh-qa/track.yml
[2024-02-15 16:59:12] [DEBUG] ALLOCATOR: Destroying instance VAGRANT-5F8B4E03-320E-4558-9E99-0A2FF37FA536
[2024-02-15 16:59:17] [INFO] ALLOCATOR: Instance VAGRANT-5F8B4E03-320E-4558-9E99-0A2FF37FA536 deleted.

@c-bordon
Copy link
Member

AWS

It was used to update the name of the key that is automatically created for the AWS deployment

cbordon@cbordon-MS-7C88:~/Documents/wazuh/repositorios/wazuh-qa/deployability$ python3 launchers/allocation.py --composite-name linux-ubuntu-22.04-amd64 --provider aws --size micro --label https://github.com/wazuh/internal-devel-requests/issues/775 --label core --label "2024-02-25 12:00:00"
[2024-02-15 17:08:09] [INFO] ALLOCATOR: Creating instance at /tmp/wazuh-qa
[2024-02-15 17:08:09] [DEBUG] ALLOCATOR: No config provided. Generating from payload
[2024-02-15 17:08:09] [DEBUG] ALLOCATOR: Generating new key pair
[2024-02-15 17:08:09] [DEBUG] ALLOCATOR: Creating base directory: /tmp/wazuh-qa/AWS-5BCC0CDC-2892-4B26-98AF-BF788BBCDA40
[2024-02-15 17:08:28] [DEBUG] ALLOCATOR: Renaming temp /tmp/wazuh-qa/AWS-5BCC0CDC-2892-4B26-98AF-BF788BBCDA40 directory to /tmp/wazuh-qa/i-0d3ff204f91a44f59
[2024-02-15 17:08:28] [INFO] ALLOCATOR: Instance i-0d3ff204f91a44f59 created.
[2024-02-15 17:08:30] [INFO] ALLOCATOR: Instance i-0d3ff204f91a44f59 started.
[2024-02-15 17:08:30] [INFO] ALLOCATOR: Inventory file generated at /tmp/wazuh-qa/inventory.yml
[2024-02-15 17:08:30] [INFO] ALLOCATOR: Track file generated at /tmp/wazuh-qa/track.yml
cbordon@cbordon-MS-7C88:~/Documents/wazuh/repositorios/wazuh-qa/deployability$ cat /tmp/wazuh-qa/inventory.yml
ansible_host: ec2-3-94-166-51.compute-1.amazonaws.com
ansible_port: 22
ansible_ssh_private_key_file: /tmp/wazuh-qa/i-0d3ff204f91a44f59/internal-devel-requests-775
ansible_user: ubuntu
cbordon@cbordon-MS-7C88:~/Documents/wazuh/repositorios/wazuh-qa/deployability$ ssh -i /tmp/wazuh-qa/i-0d3ff204f91a44f59/internal-devel-requests-775 ubuntu@ec2-3-94-166-51.compute-1.amazonaws.com
The authenticity of host 'ec2-3-94-166-51.compute-1.amazonaws.com (3.94.166.51)' can't be established.
ED25519 key fingerprint is SHA256:RGl25gUOg8+btdWpuK/TPg8bs9acjY+etAW/kaHf3/U.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added 'ec2-3-94-166-51.compute-1.amazonaws.com' (ED25519) to the list of known hosts.
Welcome to Ubuntu 22.04.2 LTS (GNU/Linux 5.19.0-1025-aws x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

  System information as of Thu Feb 15 20:09:28 UTC 2024

  System load:  0.28759765625     Processes:             107
  Usage of /:   20.6% of 7.57GB   Users logged in:       0
  Memory usage: 23%               IPv4 address for ens5: 172.31.28.40
  Swap usage:   0%

Expanded Security Maintenance for Applications is not enabled.

0 updates can be applied immediately.

Enable ESM Apps to receive additional future security updates.
See https://ubuntu.com/esm or run: sudo pro status


The list of available updates is more than a week old.
To check for new updates run: sudo apt update


The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.

To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.

ubuntu@ip-172-31-28-40:~$ exit
logout
Connection to ec2-3-94-166-51.compute-1.amazonaws.com closed.
cbordon@cbordon-MS-7C88:~/Documents/wazuh/repositorios/wazuh-qa/deployability$ python3 launchers/allocation.py --provider aws --action delete --track-output /tmp/wazuh-qa/track.yml [2024-02-15 17:09:35] [INFO] ALLOCATOR: Deleting instance from trackfile /tmp/wazuh-qa/track.yml
[2024-02-15 17:09:36] [DEBUG] ALLOCATOR: Deleting credentials: internal-devel-requests-775
[2024-02-15 17:10:55] [INFO] ALLOCATOR: Instance i-0d3ff204f91a44f59 deleted.

@c-bordon c-bordon mentioned this issue Feb 15, 2024
Merged
@c-bordon c-bordon linked a pull request Feb 15, 2024 that will close this issue
Added support for labels for AWS EC2 creation #4967
Merged
@fcaffieri fcaffieri added level/task Task issue and removed level/subtask Subtask issue labels Mar 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@c-bordon c-bordon

Labels
level/task Task issue type/enhancement
Projects
Release 4.9.0
Status: Done
Milestone
No milestone
4 participants
@teddytpc1 @QU3B1M @c-bordon @fcaffieri