Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Shared ALZ logs cost - query that runs too long #91

Open
stdistef opened this issue Nov 29, 2021 · 0 comments
Open

Shared ALZ logs cost - query that runs too long #91

stdistef opened this issue Nov 29, 2021 · 0 comments

Comments

@stdistef
Copy link

In a shared Azure Landing Zone infrastructure, we are running a dozen or so SW applications, and we must report the cost of LAW ingestion of all the Azure resources allocated to each application (we call them "Outcomes") in this environment. We built a workbook and in it fashioned some queries, one of them "by outcome" and it takes a loooooong time to complete (I have never seen it complete) based on several TiB of data ingested over the last 30 days.

Screenshot_Outcome_query_edit1
Screenshot_Outcome_query_runs

The Kusto Query

Parameter Query :- (Scoped to Subscription)

ResourceContainers
| where type=='microsoft.resources/subscriptions/resourcegroups'
| extend Tag = todynamic(tags)
| extend TeamName = Tag["TEAM NAME"]
| where isnotempty(TeamName)
| project Owner = strcat("'",name,'#',tostring(TeamName),"'")

Actual Query :- (Scoped to Log Analytics Workspace)

let OutcomeTable = datatable(ResourceGroupOwner:string) {ResourceGroupOwnerList};
find where TimeGenerated {TimeRange:value} project _ResourceId, _BilledSize, _IsBillable, TimeGenerated
| where _IsBillable == true
| extend ResourceGroup = case(isempty(_ResourceId),"Infrastructure",tostring(split(_ResourceId, '/')[4]))
| where isnotempty(ResourceGroup)
| summarize IngestedData = sum(_BilledSize) by ResourceGroup
| join kind=leftouter (OutcomeTable | extend ResourceGroup = tostring(split(ResourceGroupOwner,'#')[0]), Owner = tostring(split(ResourceGroupOwner,'#')[1]) | project ResourceGroup, Owner) on ResourceGroup
| project Owner = case(isempty(Owner),ResourceGroup,Owner), IngestedData
| summarize sum(IngestedData) by Owner

@stdistef stdistef changed the title Shared AKS Cluster container logs cost - query that runs too long Shared ALZ logs cost - query that runs too long Nov 29, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant