New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/bosun: Azure Monitor Datasource #2283

Merged
merged 5 commits into from Aug 1, 2018

Conversation

Projects
None yet
2 participants
@kylebrandt
Member

kylebrandt commented Jul 20, 2018

No description provided.

@kylebrandt

This comment has been minimized.

Member

kylebrandt commented Jul 31, 2018

For review except the metric metadata function which can be ignored for now. Also will vendor later. cc @mhenderson-so . Note: the azure datasource commit by itself has some cache edits by accident and doesn't build, so I will probably just squash all this into one when time ready for master. (fixed in rebase and force push)

@kylebrandt kylebrandt requested a review from captncraig Jul 31, 2018

@kylebrandt kylebrandt changed the title from cmd/bosun: (WIP) Azure Monitor Datasource to cmd/bosun: Azure Monitor Datasource Jul 31, 2018

DebugResponse bool
}
// Valid returns if the configuration for the AzureMonitor

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018

Member

fix comment

return allClients
}
func azureLogRequest() autorest.PrepareDecorator {

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018

Member

Comment this function and the following, add comment to attribute source of readme from the go azure sdk

This comment has been minimized.

@captncraig

captncraig Jul 31, 2018

Contributor

Are these even required long term?

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018

Member

I don't think they hurt, and since the documentation is marking the experimental for now they could be handy. I would probably be better if more datasources had this I think

if err != nil {
slog.Warningf("failure to dump azure request: %v", err)
}
dump, _ := httputil.DumpRequestOut(r, true)

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018

Member

check + log err

if err != nil {
slog.Warningf("failure to dump azure request: %v", err)
}
dump, _ := httputil.DumpResponse(r, true)

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018

Member

check + log err

},
}
// Tag function for the "az" expression function

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018

Member

comment: azTags is

return azureTags(args[2])
}
// Tag function for the "azmulti" expression function

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018

Member

comment: azMultiTags is

// azureTags adds tags for the csv argument along with the "name" and "rsg" tags
func azureTags(arg parse.Node) (parse.Tags, error) {
tags := parse.Tags{"name": struct{}{}, "rsg": struct{}{}}

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018

Member

make name and rsg constants and change in all places where referenced

const azTimeFmt = "2006-01-02T15:04:05"
func azResourceURI(subscription, resourceGrp, Namespace, Resource string) string {

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018

Member

comment func

return
}
// AzureQuery queries an Azure monitor metric for the given resource and returns a series set tagged by

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018

Member

finish this func comment

// Verify prefix is a defined resource and fetch the collection of clients
cc, clientFound := e.Backends.AzureMonitor[prefix]
if !clientFound {
return r, fmt.Errorf("azure client with name %v not defined", prefix)

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018

Member

... "%v" ...

st := e.now.Add(time.Duration(-sd)).Format(azTimeFmt)
en := e.now.Add(time.Duration(-ed)).Format(azTimeFmt)
// Set Dimensions (tag) keys for metrics that support them by building a filter

This comment has been minimized.

@kylebrandt
tg = azureIntervalToTimegrain(interval)
}
// Set azure aggregation method

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018

Member

Find various "azure" comments and replace with "Azure" (casing)

if err != nil {
slog.Errorf("failure to parse remaning reads from azure response")
} else {
collect.Sample("azure.remaining_reads", opentsdb.TagSet{"prefix": prefix}, float64(readsRemaining))

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018

Member

add comment explaining why this is sampled

// collectCache is a helper function for collecting metrics on
// the expression cache
func collectCacheHit(cacheName, qType string, hit bool) {
tags := opentsdb.TagSet{"query_type": qType, "name": cacheName}

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018

Member

add metadata for these metrics

}
series := make(Series)
tags := make(opentsdb.TagSet)
tags["rsg"] = rsg

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018

Member

ref constant here

}
}
for _, mValue := range *dataContainer.Data {
exValue := azureExtractMetricValue(&mValue, aggLong)

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018

Member

add comment

for res := range resCh {
queryResults = append(queryResults, res)
}
// Merge the query results into a single seriesSet

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018

Member

skip merge if length of set is not gt than 1

// or tags associated with that resource
func AzureFilterResources(e *State, T miniprofiler.Timer, resources AzureResources, filter string) (r *Results, err error) {
r = new(Results)
bqf, err := boolq.Parse(filter)

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018

Member

add comment

if len(sp) != 2 {
return false, fmt.Errorf("bad filter, filter must be in k:v format, got %v", filter)
}
key := strings.ToLower(sp[0]) // Make key case insensitive

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018

Member

note case insensitivity in expression docs

key := strings.ToLower(sp[0]) // Make key case insensitive
value := sp[1]
switch key {
case "name":

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018

Member

consts again for name, rsg

if re.MatchString(ar.Name) {
return true, nil
}
case "rsg", "resourcegroup":

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018

Member

remove "resourcegroup" alias, no real point, update expr docs to reflect

case string(insights.Maximum):
v = mv.Maximum
case string(insights.Total):
v = mv.Total

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018

Member

add missing None aggregation

This comment has been minimized.

@kylebrandt

kylebrandt Aug 1, 2018

Member

nm, None isn't a field in the sdk

return string(insights.Maximum), nil
case "total":
return string(insights.Total), nil
case "count":

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018

Member

add missing none aggregation here as well

@captncraig

Seems pretty good to me. Just finish out todos and comments.

@@ -84,6 +94,7 @@ func (sc *SystemConf) EnabledBackends() EnabledBackends {
b.Influx = sc.InfluxConf.URL != ""
b.Elastic = len(sc.ElasticConf["default"].Hosts) != 0
b.Annotate = len(sc.AnnotateConf.Hosts) != 0
b.AzureMonitor = sc.AzureMonitorConf["default"].ClientId != ""

This comment has been minimized.

@captncraig

captncraig Jul 31, 2018

Contributor

This feels like a weak thing to check. Are they required to have a "default"? Is ClientID required? (assume thats a credential of some kind?)

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018

Member

Agreed will make something better, this is left from my first pass

clients.MetricsClient = insights.NewMetricsClient(conf.SubscriptionId)
clients.MetricDefinitionsClient = insights.NewMetricDefinitionsClient(conf.SubscriptionId)
clients.ResourcesClient = resources.NewClient(conf.SubscriptionId)
if conf.DebugRequest {

This comment has been minimized.

@captncraig

captncraig Jul 31, 2018

Contributor

Do we really need these debug things long term? I'd probably prefer not having a million different config options if we can avoid it.

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018

Member

Just two options. Could help with people debugging issues, in particular since I saw what I think is a bug in the SDK about error being missed.

if err != nil {
// Should not hit this since we check for authorizer errors in Validation
// This is checked before because this method is not called until the an expression is called
slog.Fatal("Azure conf: ", err)

This comment has been minimized.

@captncraig

captncraig Jul 31, 2018

Contributor

Fatal feels wrong here. Does this method get invoked every time an expression runs? If azure is down intermittently, could this crash bosun?

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018

Member

I don't think this actually make any API calls and is just a validity thing. So it should be caught by Valid() method, but added this here in case I'm wrong.

return allClients
}
func azureLogRequest() autorest.PrepareDecorator {

This comment has been minimized.

@captncraig

captncraig Jul 31, 2018

Contributor

Are these even required long term?

F: AzureMultiQuery,
PrefixEnabled: true,
},
"azmd": { // TODO Finish and document this func

This comment has been minimized.

@captncraig

captncraig Jul 31, 2018

Contributor

todo

F: AzureResourcesByType,
PrefixEnabled: true,
},
"azrf": {

This comment has been minimized.

@captncraig

captncraig Jul 31, 2018

Contributor

azrf or azfr? Inner function is "FilterResources", so maybe azfr matches better? I kinda hate short names.

This comment has been minimized.

@kylebrandt

kylebrandt Aug 1, 2018

Member

I think like azrf because it shares the azr prefix with azrt

}
// AzureMetricDefinitions fetches metric information for a specific resource and metric tuple
// TODO make this return and not fmt.Printf

This comment has been minimized.

@captncraig

captncraig Jul 31, 2018

Contributor

todo

return
}
}
st := e.now.Add(time.Duration(-sd)).Format(azTimeFmt)

This comment has been minimized.

@captncraig

captncraig Jul 31, 2018

Contributor

"startTime" and "endTime" would not be unreasonable names.

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018

Member

When this is done I want to go back and make a method on the State that does since we use it all the time.

kylebrandt added some commits Jul 18, 2018

cmd/bosun: azure datasource
this includes an incomplete azure metadata function with TODOs to be
completed in a future commit.

@kylebrandt kylebrandt merged commit bcd2335 into master Aug 1, 2018

3 checks passed

bosun All checks Passed!
continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details

@kylebrandt kylebrandt deleted the ads branch Sep 27, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment