Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor azure module structure #20624

Merged
merged 8 commits into from
May 16, 2024
Merged

Conversation

nico-stefani
Copy link
Member

Related issue
#19576

Description

This PR refactors the structure of the Azure module. Splits the base code into different files to get a better organization.

Logs/Alerts example

Graph

root@255ad66dbaf2:/var/ossec# wodles/azure/azure-logs --graph --graph_auth_path /var/ossec/wodles/azure/graph.credentials --graph_tenant_domain wazuh.onmicrosoft.com --graph_tag request_829166026 --graph_query 'auditLogs/directoryaudits' --graph_time_offset 30d --debug 2
2023/12/04 19:59:52 azure: INFO: Checking database integrity
2023/12/04 19:59:52 azure: INFO: Database integrity check finished
2023/12/04 19:59:52 azure: INFO: Azure Graph starting.
2023/12/04 19:59:52 azure: INFO: Graph: Getting authentication token.
2023/12/04 19:59:53 azure: INFO: Graph: Building the url.
2023/12/04 19:59:53 azure: INFO: Graph: The search starts for query: 'auditLogs/directoryaudits' using activityDateTime+gt+2023-12-04T14:50:13.477251Z
2023/12/04 19:59:53 azure: INFO: Graph: The URL is 'https://graph.microsoft.com/v1.0/auditLogs/directoryaudits?&$filter=activityDateTime+gt+2023-12-04T14:50:13.477251Z'
2023/12/04 19:59:53 azure: INFO: Graph: Pagination starts
2023/12/04 19:59:56 azure: INFO: Graph: There are no new results
2023/12/04 19:59:56 azure: INFO: Graph: End

Log Analytics

root@255ad66dbaf2:/var/ossec# wodles/azure/azure-logs --log_analytics --la_auth_path /var/ossec/wodles/azure/loganalytics.credentials --la_tenant_domain wazuh.onmicrosoft.com --la_tag request_607367744 --la_query
"AuditLogs" --workspace xxx --la_time_offset 60d --debug 2
2023/12/04 20:04:38 azure: INFO: Checking database integrity
2023/12/04 20:04:38 azure: INFO: Database integrity check finished
2023/12/04 20:04:38 azure: INFO: Azure Log Analytics starting.
2023/12/04 20:04:38 azure: INFO: Log Analytics: Getting authentication token.
2023/12/04 20:04:40 azure: INFO: Log Analytics: The search starts for query: 'AuditLogs | order by TimeGenerated asc | where ( TimeGenerated < datetime(2023-11-04T19:52:53.759000Z) and TimeGenerated >= datetime(2023-10-05T20:04:40.941765Z)) or ( TimeGenerated > datetime(2023-12-04T14:50:13.477251Z)) '
2023/12/04 20:04:40 azure: INFO: Log Analytics: Sending a request to the Log Analytics API.
2023/12/04 20:04:42 azure: INFO: Log Analytics: There are no new results
2023/12/04 20:04:42 azure: INFO: Azure Log Analytics ending.

Storage

root@255ad66dbaf2:/var/ossec# wodles/azure/azure-logs --storage --storage_auth_path /var/ossec/wodles/azure/storage.credentials --container "frameworktestcontainer" --blobs "*" --storage_tag azure-activity --storage_time_offset 30d --debug 2
2023/12/04 20:05:48 azure: INFO: Checking database integrity
2023/12/04 20:05:48 azure: INFO: Database integrity check finished
2023/12/04 20:05:48 azure: INFO: Azure Storage starting.
2023/12/04 20:05:48 azure: INFO: Storage: Authenticating.
2023/12/04 20:05:50 azure: INFO: Storage: Authenticated.
2023/12/04 20:05:50 azure: INFO: Storage: Getting blobs.
2023/12/04 20:05:50 azure: DEBUG: String_to_sign=GET
2023/12/04 20:05:51 azure: INFO: Storage: The search starts from the date: 2023-11-04 20:05:50.573825+00:00 for blobs in container: 'frameworktestcontainer' and prefix: '/'
2023/12/04 20:05:51 azure: INFO: Storage: End

@nico-stefani nico-stefani self-assigned this Dec 4, 2023
@nico-stefani nico-stefani linked an issue Dec 4, 2023 that may be closed by this pull request
8 tasks
@nico-stefani nico-stefani force-pushed the 19576-refactor-azure-structure branch 2 times, most recently from 9581050 to 50843c9 Compare December 5, 2023 17:04
@GGP1 GGP1 self-requested a review December 6, 2023 12:50
wodles/azure/azure_services/analytics.py Outdated Show resolved Hide resolved
GGP1
GGP1 previously approved these changes Dec 7, 2023
Copy link
Member

@GGP1 GGP1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@Selutario Selutario left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good, but I have some questions:

  1. We are changing the location of azure.db from /var/ossec/wodles/azure/azure.db to /var/ossec/wodles/azure/db/azure.db. Does this affect in any way those users who upgrade compared to current behavior?

    For example, is it possible that alerts that had already been generated before upgrading could be duplicated since the DB is in a different location? Will the old db be deleted or moved?

    I would like to see an example where 4.8.0 is installed, the wodle is run to get some events/alerts and then upgrade to this branch and run the wodle again.

  2. I guess it is also needed to update the check_files and other related files in Jenkins, as it was done when refactoring AWS here.

  3. Probably, Solaris package files also need to be updated (AWS example).

wodles/azure/azure-logs.py Outdated Show resolved Hide resolved
wodles/azure/db/azure.db Outdated Show resolved Hide resolved
@nico-stefani
Copy link
Member Author

nico-stefani commented Dec 13, 2023

After checking the behaviour of the AWS module, I moved the db file to the root of the module

root@0aaf5453a213:/var/ossec# ll wodles/azure/
total 84
drwxr-x--- 5 root wazuh  4096 Dec 13 18:54 ./
drwxr-x--- 7 root wazuh  4096 Dec 13 18:52 ../
drwxr-xr-x 2 root root   4096 Dec 13 18:52 __pycache__/
-rwxr-x--- 1 root wazuh  1045 Dec 13 18:42 azure-logs*
-rwxr-x--- 1 root wazuh  1466 Dec 13 18:42 azure-logs.py*
-rw-r--r-- 1 root root  28672 Dec 13 18:54 azure.db
drwxr-x--- 3 root wazuh  4096 Dec 13 18:52 azure_services/
-rwxr-x--- 1 root wazuh 13811 Dec 13 18:42 azure_utils.py*
drwxr-x--- 3 root wazuh  4096 Dec 13 18:52 db/
-rw-r--r-- 1 root root    113 Dec 13 18:52 graph.credentials
-rw-r--r-- 1 root root    112 Dec 13 18:52 loganalytics.credentials
-rw-r--r-- 1 root root    138 Dec 13 18:52 storage.credentials

Also, two new issues were opened to address the changes in wazuh-jenkins and wazuh-packages respectively:

Selutario
Selutario previously approved these changes Dec 14, 2023
@nico-stefani nico-stefani changed the base branch from master to 4.9.0 May 8, 2024 20:37
@nico-stefani nico-stefani force-pushed the 19576-refactor-azure-structure branch from 30b21c1 to fd77e0f Compare May 9, 2024 13:41
@Selutario
Copy link
Contributor

The failed checks are unrelated to these changes:

2024/05/09 14:18:16 wazuh-modulesd:vulnerability-scanner: ERROR: VulnerabilityScannerFacade::initEventDispatcher: [json.exception.type_error.302] type must be string, but is object

@Selutario Selutario merged commit 3125655 into 4.9.0 May 16, 2024
62 of 68 checks passed
@Selutario Selutario deleted the 19576-refactor-azure-structure branch May 16, 2024 07:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Refactor Azure cloud module project structure.
3 participants