Added force deletion capability to log analytics module and updated requirements.txt #273

next-davidduquehernandez · 2020-09-25T18:58:26Z

SUMMARY

Added force delete capability introduced on 05/26/2020 update. Updated requirement library 'azure-mgmt-loganalytics' to 1.0.0 and update its method calls.

ISSUE TYPE

Feature Pull Request

COMPONENT NAME

azure_rm_loganalyticsworkspace

ADDITIONAL INFORMATION

According to azure documentation we can force log analytics deletion. This capability allow us to remove log analytics workspace permanently, cause default deletion has a soft-delete behavior.

As requirement was set to 0.2.0 (and this feature was introduced on 1.0.0), this forced me to update requirements and some sdk call methods.

This is my first pull request to an open source project. Please, tell me if I need to do some additional step or modify something in my code. Regards :)

Usage example:

Create log analytics workspace:

    - azure_rm_loganalyticsworkspace:
        name: loganalyticsex
        resource_group: rgtest01
        state: present

Create output:

ok: [localhost] => {
    "changed": false,
    "customer_id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
    "id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourcegroups/rgtest01/providers/microsoft.operationalinsights/workspaces/loganalyticsex",
    "intelligence_packs": [
        {
            "enabled": false,
            "name": "CapacityPerformance"
        },
        {
            "enabled": false,
            "name": "AzureWebAppsAnalytics"
        },
        {
            "enabled": false,
            "name": "BehaviorAnalyticsInsights"
        },
        {
            "enabled": false,
            "name": "Security"
        },
        {
            "enabled": false,
            "name": "Updates"
        },
        {
            "enabled": false,
            "name": "AntiMalware"
        },
        {
            "enabled": true,
            "name": "LogManagement"
        },
        {
            "enabled": true,
            "name": "AzureResources"
        },
        {
            "enabled": false,
            "name": "ChangeTracking"
        },
        {
            "enabled": false,
            "name": "SQLAssessment"
        },
        {
            "enabled": false,
            "name": "SCOMAssessment"
        },
        {
            "enabled": false,
            "name": "ServiceDesk"
        },
        {
            "enabled": false,
            "name": "SQLAssessmentPlus"
        },
        {
            "enabled": false,
            "name": "ADAssessment"
        },
        {
            "enabled": false,
            "name": "AlertManagement"
        },
        {
            "enabled": false,
            "name": "AzureAutomation"
        },
        {
            "enabled": false,
            "name": "WireData"
        },
        {
            "enabled": false,
            "name": "SiteRecovery"
        },
        {
            "enabled": false,
            "name": "Backup"
        },
        {
            "enabled": false,
            "name": "SurfaceHub"
        },
        {
            "enabled": false,
            "name": "NetworkMonitoring"
        },
        {
            "enabled": false,
            "name": "Containers"
        },
        {
            "enabled": false,
            "name": "ContainerInsights"
        },
        {
            "enabled": false,
            "name": "ServiceMap"
        },
        {
            "enabled": false,
            "name": "AzureNetworking"
        },
        {
            "enabled": false,
            "name": "ADReplication"
        },
        {
            "enabled": false,
            "name": "Office365"
        },
        {
            "enabled": false,
            "name": "CompatibilityAssessment"
        },
        {
            "enabled": false,
            "name": "KeyVault"
        },
        {
            "enabled": false,
            "name": "ServiceFabric"
        },
        {
            "enabled": false,
            "name": "DnsAnalytics"
        },
        {
            "enabled": false,
            "name": "ApplicationInsights"
        },
        {
            "enabled": false,
            "name": "WireData2"
        },
        {
            "enabled": false,
            "name": "WaaSUpdateInsights"
        },
        {
            "enabled": false,
            "name": "AgentHealthAssessment"
        },
        {
            "enabled": false,
            "name": "AzureActivity"
        },
        {
            "enabled": false,
            "name": "HDInsight"
        },
        {
            "enabled": false,
            "name": "HDInsightKafka"
        },
        {
            "enabled": false,
            "name": "HDInsightSpark"
        },
        {
            "enabled": false,
            "name": "HDInsightHadoop"
        },
        {
            "enabled": false,
            "name": "AzureCdnCoreAnalytics"
        },
        {
            "enabled": false,
            "name": "HDInsightStorm"
        },
        {
            "enabled": false,
            "name": "HDInsightInteractiveQuery"
        },
        {
            "enabled": false,
            "name": "VMware"
        },
        {
            "enabled": false,
            "name": "SecurityCenterFree"
        },
        {
            "enabled": false,
            "name": "LogicAppsManagement"
        },
        {
            "enabled": false,
            "name": "LogicAppB2B"
        },
        {
            "enabled": false,
            "name": "AzureSQLAnalytics"
        },
        {
            "enabled": false,
            "name": "KeyVaultAnalytics"
        },
        {
            "enabled": false,
            "name": "AzureNSGAnalytics"
        },
        {
            "enabled": false,
            "name": "AzureAppGatewayAnalytics"
        },
        {
            "enabled": false,
            "name": "DeviceHealthProd"
        },
        {
            "enabled": false,
            "name": "WindowsDefenderATP"
        },
        {
            "enabled": false,
            "name": "WindowsDefenderATPStable"
        },
        {
            "enabled": false,
            "name": "InfrastructureInsights"
        },
        {
            "enabled": false,
            "name": "ProcessInvestigator"
        },
        {
            "enabled": false,
            "name": "Microsoft365Analytics"
        },
        {
            "enabled": false,
            "name": "ASRAnalytics"
        },
        {
            "enabled": false,
            "name": "AzureSentinelPrivatePreview"
        },
        {
            "enabled": false,
            "name": "SecurityInsights"
        },
        {
            "enabled": false,
            "name": "SecurityCenterNetworkTraffic"
        },
        {
            "enabled": false,
            "name": "WindowsFirewall"
        },
        {
            "enabled": false,
            "name": "WindowsEventForwarding"
        },
        {
            "enabled": false,
            "name": "InternalWindowsEvent"
        },
        {
            "enabled": false,
            "name": "CustomizedWindowsEventsFiltering"
        },
        {
            "enabled": false,
            "name": "WEFInternalUat"
        },
        {
            "enabled": false,
            "name": "WEFInternal"
        },
        {
            "enabled": false,
            "name": "WEF_10x"
        },
        {
            "enabled": false,
            "name": "WEF_10xDSRE"
        },
        {
            "enabled": false,
            "name": "DHCPActivity"
        },
        {
            "enabled": false,
            "name": "AzureDataFactoryAnalytics"
        },
        {
            "enabled": false,
            "name": "AzureSecurityOfThings"
        },
        {
            "enabled": false,
            "name": "SQLVulnerabilityAssessment"
        },
        {
            "enabled": false,
            "name": "SQLAdvancedThreatProtection"
        },
        {
            "enabled": false,
            "name": "VMInsights"
        },
        {
            "enabled": false,
            "name": "WinLog"
        },
        {
            "enabled": false,
            "name": "NetFlow"
        }
    ],
    "invocation": {
        "module_args": {
            "ad_user": null,
            "adfs_authority_url": null,
            "api_profile": "latest",
            "append_tags": true,
            "auth_source": null,
            "cert_validation_mode": null,
            "client_id": null,
            "cloud_environment": "AzureCloud",
            "force": null,
            "intelligence_packs": null,
            "location": null,
            "name": "loganalyticsex",
            "password": null,
            "profile": null,
            "resource_group": "rgtest01",
            "retention_in_days": null,
            "secret": null,
            "sku": "per_gb2018",
            "state": "present",
            "subscription_id": null,
            "tags": null,
            "tenant": null
        }
    },
    "location": "westeurope",
    "management_groups": [],
    "name": "loganalyticsex",
    "provisioning_state": "Succeeded",
    "public_network_access_for_ingestion": "Enabled",
    "public_network_access_for_query": "Enabled",
    "retention_in_days": 30,
    "shared_keys": {
        "primary_shared_key": "mh360coZP3tUvaD6RG57ABnuOdkchgTq+UsrMNhjwAjFIljnB+qsrAvfKMSCkBV+D4rpNkbHZG/xDM1QDKBSfQ==",
        "secondary_shared_key": "NziIgGDqsHn7tY+acDxKPG7XTlCCRMI0cS6sUc+fA5M7GCZvcClaaweTY6eaOmIzCapw4nIWfnDLYvg9B1I+qQ=="
    },
    "sku": "pergb2018",
    "tags": {},
    "type": "Microsoft.OperationalInsights/workspaces",
    "usages": [
        {
            "current_value": 0.0,
            "limit": -1.0,
            "name": {
                "localized_value": "Data Analyzed",
                "value": "DataAnalyzed"
            },
            "next_reset_time": "2020-09-25T20:00:00.000Z",
            "quota_period": "P1D",
            "unit": "Bytes"
        }
    ],
    "workspace_capping": {
        "daily_quota_gb": -1.0,
        "data_ingestion_status": "RespectQuota",
        "quota_next_reset_time": "Fri, 25 Sep 2020 20:00:00 GMT"
    }
}

Delete log analytics workspace with force:

    - azure_rm_loganalyticsworkspace:
        name: loganalyticsex
        resource_group: rgtest01
        state: absent
        force: true

Delete output:

changed: [localhost] => {
    "changed": true,
    "invocation": {
        "module_args": {
            "ad_user": null,
            "adfs_authority_url": null,
            "api_profile": "latest",
            "append_tags": true,
            "auth_source": null,
            "cert_validation_mode": null,
            "client_id": null,
            "cloud_environment": "AzureCloud",
            "force": true,
            "intelligence_packs": null,
            "location": null,
            "name": "loganalyticsex",
            "password": null,
            "profile": null,
            "resource_group": "rgtest01",
            "retention_in_days": null,
            "secret": null,
            "sku": "per_gb2018",
            "state": "absent",
            "subscription_id": null,
            "tags": null,
            "tenant": null
        }
    }
}

imjoseangel · 2020-09-25T19:33:44Z

Hey @next-davidduquehernandez as a good startup point, it is important to open an issue explaining what do you want to achieve and then link the PR with the comment Fixes #yourissuenumber. Document the issue with the requirements to understand what we should expect in the PR.

Thank you!

plugins/modules/azure_rm_loganalyticsworkspace.py

imjoseangel

@next-davidduquehernandez I have added some comments to add a default value to the force option. Remember to open an issue with the description of what do you want to fix. Nice Job!

next-davidduquehernandez · 2020-09-26T00:01:37Z

Fixes #274

next-davidduquehernandez · 2020-09-26T00:15:21Z

Thank you for your comments @imjoseangel. I introduced some tests to this PR and fixed 'azure_rm_loganalyticsworkspace_info' too. I has created issue #274 and tried to link with this PR, but I don't know if it's already done.

imjoseangel · 2020-09-28T10:16:53Z

LGTM

Fred-sun · 2020-09-30T01:36:33Z

Sorry, This is my mistake!

Fred-sun · 2020-10-22T03:15:19Z

@next-davidduquehernandez Please add line "azure_rm_loganalyticsworkspace" to pr-pipelines.yml. Thank you very much!

Fred-sun

Currently, python3 in the pipeline lacks the filter json_query. Cancel it temporarily and change it back after updating the environment. Thank you!

Fred-sun · 2020-10-22T05:50:36Z

tests/integration/targets/azure_rm_loganalyticsworkspace/tasks/main.yml

+    that:
+      - output.changed
+      - output.retention_in_days == retention_days
+      - output.intelligence_packs | json_query('[?name == `Backup`].enabled') | first == true


Suggested change

- output.intelligence_packs | json_query('[?name == `Backup`].enabled') | first == true

#- output.intelligence_packs | json_query('[?name == `Backup`].enabled') | first == true

Fred-sun · 2020-10-22T05:50:59Z

tests/integration/targets/azure_rm_loganalyticsworkspace/tasks/main.yml

+      - output.changed
+      - output.retention_in_days == retention_days
+      - output.intelligence_packs | json_query('[?name == `Backup`].enabled') | first == true
+      - output.intelligence_packs | json_query('[?name == `Containers`].enabled') | first == true


Suggested change

- output.intelligence_packs | json_query('[?name == `Containers`].enabled') | first == true

#- output.intelligence_packs | json_query('[?name == `Containers`].enabled') | first == true

Fred-sun · 2020-10-22T05:51:22Z

tests/integration/targets/azure_rm_loganalyticsworkspace/tasks/main.yml

+      - facts.workspaces | length > 0
+      - facts.workspaces[0].retention_in_days == retention_days
+      - facts.workspaces[0].intelligence_packs | length > 0
+      - facts.workspaces[0].intelligence_packs | json_query('[?name == `Backup`].enabled') | first == true


Suggested change

- facts.workspaces[0].intelligence_packs | json_query('[?name == `Backup`].enabled') | first == true

#- facts.workspaces[0].intelligence_packs | json_query('[?name == `Backup`].enabled') | first == true

Fred-sun · 2020-10-22T05:51:37Z

tests/integration/targets/azure_rm_loganalyticsworkspace/tasks/main.yml

+      - facts.workspaces[0].retention_in_days == retention_days
+      - facts.workspaces[0].intelligence_packs | length > 0
+      - facts.workspaces[0].intelligence_packs | json_query('[?name == `Backup`].enabled') | first == true
+      - facts.workspaces[0].intelligence_packs | json_query('[?name == `Containers`].enabled') | first == true


Suggested change

- facts.workspaces[0].intelligence_packs | json_query('[?name == `Containers`].enabled') | first == true

#- facts.workspaces[0].intelligence_packs | json_query('[?name == `Containers`].enabled') | first == true

…rement azure-mgmt-loganalytics to 1.0.0 and update sdk calls

…nalyticsworkspace_info.py

…274

…json_query in test

next-davidduquehernandez · 2020-10-22T17:27:58Z

Added "azure_rm_loganalyticsworkspace" to pr-pipelines.yml and commented lines with json_query. :)

…icsworkspace

imjoseangel reviewed Sep 25, 2020

View reviewed changes

plugins/modules/azure_rm_loganalyticsworkspace.py Show resolved Hide resolved

imjoseangel reviewed Sep 25, 2020

View reviewed changes

plugins/modules/azure_rm_loganalyticsworkspace.py Outdated Show resolved Hide resolved

imjoseangel suggested changes Sep 25, 2020

View reviewed changes

This comment has been minimized.

Sign in to view

Fred-sun added work in In trying to solve, or in working with contributors medium_priority Medium priority labels Sep 30, 2020

Fred-sun reviewed Oct 22, 2020

View reviewed changes

lupocito and others added 5 commits October 22, 2020 19:12

Added force deletion capability to log analytics module, update requi…

ac9b023

…rement azure-mgmt-loganalytics to 1.0.0 and update sdk calls

Added integration tests and update sdk method calls for azure_rm_loga…

c00b710

…nalyticsworkspace_info.py

Add default value to force parameter as @imjoseangel commented. Fixes #…

62cb9ab

…274

Fixes name fact from integrity test

f1422e6

Added azure_rm_loganalyticsworkspace to pr-pipelines.yml and disable …

18df968

…json_query in test

Fred-sun added ready_for_review The PR has been modified and can be reviewed and merged and removed work in In trying to solve, or in working with contributors labels Oct 23, 2020

Merge branch 'dev' into feature/force_deletion_for_azure_rm_loganalyt…

abbb3df

…icsworkspace

haiyuazhang merged commit 0dcdcaa into ansible-collections:dev Oct 26, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added force deletion capability to log analytics module and updated requirements.txt #273

Added force deletion capability to log analytics module and updated requirements.txt #273

next-davidduquehernandez commented Sep 25, 2020

imjoseangel commented Sep 25, 2020

imjoseangel left a comment •

edited

Loading

next-davidduquehernandez commented Sep 26, 2020

next-davidduquehernandez commented Sep 26, 2020

imjoseangel commented Sep 28, 2020

This comment has been minimized.

This comment has been minimized.

Fred-sun commented Sep 30, 2020

Fred-sun commented Oct 22, 2020

Fred-sun left a comment

Fred-sun Oct 22, 2020

Fred-sun Oct 22, 2020

Fred-sun Oct 22, 2020

Fred-sun Oct 22, 2020

next-davidduquehernandez commented Oct 22, 2020

	- output.intelligence_packs \| json_query('[?name == `Backup`].enabled') \| first == true
	#- output.intelligence_packs \| json_query('[?name == `Backup`].enabled') \| first == true

	- facts.workspaces[0].intelligence_packs \| json_query('[?name == `Backup`].enabled') \| first == true
	#- facts.workspaces[0].intelligence_packs \| json_query('[?name == `Backup`].enabled') \| first == true

Added force deletion capability to log analytics module and updated requirements.txt #273

Added force deletion capability to log analytics module and updated requirements.txt #273

Conversation

next-davidduquehernandez commented Sep 25, 2020

SUMMARY

ISSUE TYPE

COMPONENT NAME

ADDITIONAL INFORMATION

imjoseangel commented Sep 25, 2020

imjoseangel left a comment • edited Loading

Choose a reason for hiding this comment

next-davidduquehernandez commented Sep 26, 2020

next-davidduquehernandez commented Sep 26, 2020

imjoseangel commented Sep 28, 2020

This comment has been minimized.

This comment has been minimized.

Fred-sun commented Sep 30, 2020

Fred-sun commented Oct 22, 2020

Fred-sun left a comment

Choose a reason for hiding this comment

Fred-sun Oct 22, 2020

Choose a reason for hiding this comment

Fred-sun Oct 22, 2020

Choose a reason for hiding this comment

Fred-sun Oct 22, 2020

Choose a reason for hiding this comment

Fred-sun Oct 22, 2020

Choose a reason for hiding this comment

next-davidduquehernandez commented Oct 22, 2020

imjoseangel left a comment •

edited

Loading