Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/bosun: Azure Monitor Datasource #2283

Merged
merged 5 commits into from Aug 1, 2018
Merged

cmd/bosun: Azure Monitor Datasource #2283

merged 5 commits into from Aug 1, 2018

Conversation

kylebrandt
Copy link
Member

No description provided.

@kylebrandt
Copy link
Member Author

kylebrandt commented Jul 31, 2018

For review except the metric metadata function which can be ignored for now. Also will vendor later. cc @mhenderson-so . Note: the azure datasource commit by itself has some cache edits by accident and doesn't build, so I will probably just squash all this into one when time ready for master. (fixed in rebase and force push)

@kylebrandt kylebrandt changed the title cmd/bosun: (WIP) Azure Monitor Datasource cmd/bosun: Azure Monitor Datasource Jul 31, 2018
DebugResponse bool
}

// Valid returns if the configuration for the AzureMonitor
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fix comment

return allClients
}

func azureLogRequest() autorest.PrepareDecorator {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment this function and the following, add comment to attribute source of readme from the go azure sdk

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these even required long term?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think they hurt, and since the documentation is marking the experimental for now they could be handy. I would probably be better if more datasources had this I think

if err != nil {
slog.Warningf("failure to dump azure request: %v", err)
}
dump, _ := httputil.DumpRequestOut(r, true)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

check + log err

if err != nil {
slog.Warningf("failure to dump azure request: %v", err)
}
dump, _ := httputil.DumpResponse(r, true)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

check + log err

},
}

// Tag function for the "az" expression function
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

comment: azTags is

return azureTags(args[2])
}

// Tag function for the "azmulti" expression function
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

comment: azMultiTags is


// azureTags adds tags for the csv argument along with the "name" and "rsg" tags
func azureTags(arg parse.Node) (parse.Tags, error) {
tags := parse.Tags{"name": struct{}{}, "rsg": struct{}{}}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make name and rsg constants and change in all places where referenced


const azTimeFmt = "2006-01-02T15:04:05"

func azResourceURI(subscription, resourceGrp, Namespace, Resource string) string {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

comment func

return
}

// AzureQuery queries an Azure monitor metric for the given resource and returns a series set tagged by
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

finish this func comment

// Verify prefix is a defined resource and fetch the collection of clients
cc, clientFound := e.Backends.AzureMonitor[prefix]
if !clientFound {
return r, fmt.Errorf("azure client with name %v not defined", prefix)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

... "%v" ...

st := e.now.Add(time.Duration(-sd)).Format(azTimeFmt)
en := e.now.Add(time.Duration(-ed)).Format(azTimeFmt)

// Set Dimensions (tag) keys for metrics that support them by building a filter
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tg = azureIntervalToTimegrain(interval)
}

// Set azure aggregation method
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Find various "azure" comments and replace with "Azure" (casing)

if err != nil {
slog.Errorf("failure to parse remaning reads from azure response")
} else {
collect.Sample("azure.remaining_reads", opentsdb.TagSet{"prefix": prefix}, float64(readsRemaining))
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add comment explaining why this is sampled

// collectCache is a helper function for collecting metrics on
// the expression cache
func collectCacheHit(cacheName, qType string, hit bool) {
tags := opentsdb.TagSet{"query_type": qType, "name": cacheName}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add metadata for these metrics

}
series := make(Series)
tags := make(opentsdb.TagSet)
tags["rsg"] = rsg
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ref constant here

}
}
for _, mValue := range *dataContainer.Data {
exValue := azureExtractMetricValue(&mValue, aggLong)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add comment

for res := range resCh {
queryResults = append(queryResults, res)
}
// Merge the query results into a single seriesSet
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

skip merge if length of set is not gt than 1

// or tags associated with that resource
func AzureFilterResources(e *State, T miniprofiler.Timer, resources AzureResources, filter string) (r *Results, err error) {
r = new(Results)
bqf, err := boolq.Parse(filter)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add comment

if len(sp) != 2 {
return false, fmt.Errorf("bad filter, filter must be in k:v format, got %v", filter)
}
key := strings.ToLower(sp[0]) // Make key case insensitive
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note case insensitivity in expression docs

key := strings.ToLower(sp[0]) // Make key case insensitive
value := sp[1]
switch key {
case "name":
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

consts again for name, rsg

if re.MatchString(ar.Name) {
return true, nil
}
case "rsg", "resourcegroup":
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove "resourcegroup" alias, no real point, update expr docs to reflect

case string(insights.Maximum):
v = mv.Maximum
case string(insights.Total):
v = mv.Total
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add missing None aggregation

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nm, None isn't a field in the sdk

return string(insights.Maximum), nil
case "total":
return string(insights.Total), nil
case "count":
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add missing none aggregation here as well

Copy link
Contributor

@captncraig captncraig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems pretty good to me. Just finish out todos and comments.

@@ -84,6 +94,7 @@ func (sc *SystemConf) EnabledBackends() EnabledBackends {
b.Influx = sc.InfluxConf.URL != ""
b.Elastic = len(sc.ElasticConf["default"].Hosts) != 0
b.Annotate = len(sc.AnnotateConf.Hosts) != 0
b.AzureMonitor = sc.AzureMonitorConf["default"].ClientId != ""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feels like a weak thing to check. Are they required to have a "default"? Is ClientID required? (assume thats a credential of some kind?)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed will make something better, this is left from my first pass

clients.MetricsClient = insights.NewMetricsClient(conf.SubscriptionId)
clients.MetricDefinitionsClient = insights.NewMetricDefinitionsClient(conf.SubscriptionId)
clients.ResourcesClient = resources.NewClient(conf.SubscriptionId)
if conf.DebugRequest {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really need these debug things long term? I'd probably prefer not having a million different config options if we can avoid it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just two options. Could help with people debugging issues, in particular since I saw what I think is a bug in the SDK about error being missed.

if err != nil {
// Should not hit this since we check for authorizer errors in Validation
// This is checked before because this method is not called until the an expression is called
slog.Fatal("Azure conf: ", err)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fatal feels wrong here. Does this method get invoked every time an expression runs? If azure is down intermittently, could this crash bosun?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this actually make any API calls and is just a validity thing. So it should be caught by Valid() method, but added this here in case I'm wrong.

return allClients
}

func azureLogRequest() autorest.PrepareDecorator {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these even required long term?

F: AzureMultiQuery,
PrefixEnabled: true,
},
"azmd": { // TODO Finish and document this func
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

todo

F: AzureResourcesByType,
PrefixEnabled: true,
},
"azrf": {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

azrf or azfr? Inner function is "FilterResources", so maybe azfr matches better? I kinda hate short names.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think like azrf because it shares the azr prefix with azrt

}

// AzureMetricDefinitions fetches metric information for a specific resource and metric tuple
// TODO make this return and not fmt.Printf
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

todo

return
}
}
st := e.now.Add(time.Duration(-sd)).Format(azTimeFmt)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"startTime" and "endTime" would not be unreasonable names.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When this is done I want to go back and make a method on the State that does since we use it all the time.

this includes an incomplete azure metadata function with TODOs to be
completed in a future commit.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants