/bosun

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/bosun: Azure Monitor Datasource #2283

Merged
merged 5 commits into from Aug 1, 2018

Conversation

Reviewers

@captncraig captncraig

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
2 participants
@kylebrandt @captncraig
@kylebrandt
Member

kylebrandt commented Jul 20, 2018

No description provided.

@kylebrandt

This comment has been minimized.

Sign in to view
Member

kylebrandt commented Jul 31, 2018 •
edited

For review except the metric metadata function which can be ignored for now. Also will vendor later. cc @mhenderson-so . Note: the azure datasource commit by itself has some cache edits by accident and doesn't build, so I will probably just squash all this into one when time ready for master. (fixed in rebase and force push)

@kylebrandt kylebrandt requested a review from captncraig Jul 31, 2018

@kylebrandt kylebrandt changed the title from cmd/bosun: (WIP) Azure Monitor Datasource to cmd/bosun: Azure Monitor Datasource Jul 31, 2018

cmd/bosun/conf/system.go Outdated
DebugResponse bool
}
// Valid returns if the configuration for the AzureMonitor

This comment has been minimized.

Sign in to view
Copy link
@kylebrandt

kylebrandt Jul 31, 2018

Member

fix comment
cmd/bosun/conf/system.go Outdated
return allClients
}
func azureLogRequest() autorest.PrepareDecorator {

This comment has been minimized.

Sign in to view
Copy link
@kylebrandt

kylebrandt Jul 31, 2018

Member

Comment this function and the following, add comment to attribute source of readme from the go azure sdk

This comment has been minimized.

Sign in to view
Copy link
@captncraig

captncraig Jul 31, 2018

Contributor

Are these even required long term?

This comment has been minimized.

Sign in to view
Copy link
@kylebrandt

kylebrandt Jul 31, 2018

Member

I don't think they hurt, and since the documentation is marking the experimental for now they could be handy. I would probably be better if more datasources had this I think
cmd/bosun/conf/system.go Outdated
if err != nil {
slog.Warningf("failure to dump azure request: %v", err)
}
dump, _ := httputil.DumpRequestOut(r, true)

This comment has been minimized.

Sign in to view
Copy link
@kylebrandt

kylebrandt Jul 31, 2018

Member

check + log err
cmd/bosun/conf/system.go Outdated
if err != nil {
slog.Warningf("failure to dump azure request: %v", err)
}
dump, _ := httputil.DumpResponse(r, true)

This comment has been minimized.

Sign in to view
Copy link
@kylebrandt

kylebrandt Jul 31, 2018

Member

check + log err
cmd/bosun/expr/azure.go Outdated
},
}
// Tag function for the "az" expression function

This comment has been minimized.

Sign in to view
Copy link
@kylebrandt

kylebrandt Jul 31, 2018

Member

comment: azTags is
cmd/bosun/expr/azure.go Outdated
return azureTags(args[2])
}
// Tag function for the "azmulti" expression function

This comment has been minimized.

Sign in to view
Copy link
@kylebrandt

kylebrandt Jul 31, 2018

Member

comment: azMultiTags is
cmd/bosun/expr/azure.go Outdated
// azureTags adds tags for the csv argument along with the "name" and "rsg" tags
func azureTags(arg parse.Node) (parse.Tags, error) {
tags := parse.Tags{"name": struct{}{}, "rsg": struct{}{}}

This comment has been minimized.

Sign in to view
Copy link
@kylebrandt

kylebrandt Jul 31, 2018

Member

make name and rsg constants and change in all places where referenced
cmd/bosun/expr/azure.go Outdated
const azTimeFmt = "2006-01-02T15:04:05"
func azResourceURI(subscription, resourceGrp, Namespace, Resource string) string {

This comment has been minimized.

Sign in to view
Copy link
@kylebrandt

kylebrandt Jul 31, 2018

Member

comment func
cmd/bosun/expr/azure.go
return
}
// AzureQuery queries an Azure monitor metric for the given resource and returns a series set tagged by

This comment has been minimized.

Sign in to view
Copy link
@kylebrandt

kylebrandt Jul 31, 2018

Member

finish this func comment
cmd/bosun/expr/azure.go Outdated
// Verify prefix is a defined resource and fetch the collection of clients
cc, clientFound := e.Backends.AzureMonitor[prefix]
if !clientFound {
return r, fmt.Errorf("azure client with name %v not defined", prefix)

This comment has been minimized.

Sign in to view
Copy link
@kylebrandt

kylebrandt Jul 31, 2018

Member

... "%v" ...
cmd/bosun/expr/azure.go Outdated
st := e.now.Add(time.Duration(-sd)).Format(azTimeFmt)
en := e.now.Add(time.Duration(-ed)).Format(azTimeFmt)
// Set Dimensions (tag) keys for metrics that support them by building a filter

This comment has been minimized.

Sign in to view
Copy link
@kylebrandt

kylebrandt Jul 31, 2018

Member

expand on comment , link to https://docs.microsoft.com/en-us/rest/api/monitor/filter-syntax
cmd/bosun/expr/azure.go Outdated
tg = azureIntervalToTimegrain(interval)
}
// Set azure aggregation method

This comment has been minimized.

Sign in to view
Copy link
@kylebrandt

kylebrandt Jul 31, 2018

Member

Find various "azure" comments and replace with "Azure" (casing)
cmd/bosun/expr/azure.go Outdated
if err != nil {
slog.Errorf("failure to parse remaning reads from azure response")
} else {
collect.Sample("azure.remaining_reads", opentsdb.TagSet{"prefix": prefix}, float64(readsRemaining))

This comment has been minimized.

Sign in to view
Copy link
@kylebrandt

kylebrandt Jul 31, 2018

Member

add comment explaining why this is sampled
cmd/bosun/expr/expr.go Outdated
// collectCache is a helper function for collecting metrics on
// the expression cache
func collectCacheHit(cacheName, qType string, hit bool) {
tags := opentsdb.TagSet{"query_type": qType, "name": cacheName}

This comment has been minimized.

Sign in to view
Copy link
@kylebrandt

kylebrandt Jul 31, 2018

Member

add metadata for these metrics
cmd/bosun/expr/azure.go Outdated
}
series := make(Series)
tags := make(opentsdb.TagSet)
tags["rsg"] = rsg

This comment has been minimized.

Sign in to view
Copy link
@kylebrandt

kylebrandt Jul 31, 2018

Member

ref constant here
cmd/bosun/expr/azure.go Outdated
}
}
for _, mValue := range *dataContainer.Data {
exValue := azureExtractMetricValue(&mValue, aggLong)

This comment has been minimized.

Sign in to view
Copy link
@kylebrandt

kylebrandt Jul 31, 2018

Member

add comment
cmd/bosun/expr/azure.go Outdated
for res := range resCh {
queryResults = append(queryResults, res)
}
// Merge the query results into a single seriesSet

This comment has been minimized.

Sign in to view
Copy link
@kylebrandt

kylebrandt Jul 31, 2018

Member

skip merge if length of set is not gt than 1
cmd/bosun/expr/azure.go Outdated
// or tags associated with that resource
func AzureFilterResources(e *State, T miniprofiler.Timer, resources AzureResources, filter string) (r *Results, err error) {
r = new(Results)
bqf, err := boolq.Parse(filter)

This comment has been minimized.

Sign in to view
Copy link
@kylebrandt

kylebrandt Jul 31, 2018

Member

add comment
cmd/bosun/expr/azure.go
if len(sp) != 2 {
return false, fmt.Errorf("bad filter, filter must be in k:v format, got %v", filter)
}
key := strings.ToLower(sp[0]) // Make key case insensitive

This comment has been minimized.

Sign in to view
Copy link
@kylebrandt

kylebrandt Jul 31, 2018

Member

note case insensitivity in expression docs
cmd/bosun/expr/azure.go Outdated
key := strings.ToLower(sp[0]) // Make key case insensitive
value := sp[1]
switch key {
case "name":

This comment has been minimized.

Sign in to view
Copy link
@kylebrandt

kylebrandt Jul 31, 2018

Member

consts again for name, rsg
cmd/bosun/expr/azure.go Outdated
if re.MatchString(ar.Name) {
return true, nil
}
case "rsg", "resourcegroup":

This comment has been minimized.

Sign in to view
Copy link
@kylebrandt

kylebrandt Jul 31, 2018

Member

remove "resourcegroup" alias, no real point, update expr docs to reflect
cmd/bosun/expr/azure.go
case string(insights.Maximum):
v = mv.Maximum
case string(insights.Total):
v = mv.Total

This comment has been minimized.

Sign in to view
Copy link
@kylebrandt

kylebrandt Jul 31, 2018

Member

add missing None aggregation

This comment has been minimized.

Sign in to view
Copy link
@kylebrandt

kylebrandt Aug 1, 2018

Member

nm, None isn't a field in the sdk
cmd/bosun/expr/azure.go
return string(insights.Maximum), nil
case "total":
return string(insights.Total), nil
case "count":

This comment has been minimized.

Sign in to view
Copy link
@kylebrandt

kylebrandt Jul 31, 2018

Member

add missing none aggregation here as well
@captncraig

captncraig approved these changes Jul 31, 2018
View changes

Seems pretty good to me. Just finish out todos and comments.

cmd/bosun/conf/system.go Outdated
@@ -84,6 +94,7 @@ func (sc *SystemConf) EnabledBackends() EnabledBackends {
b.Influx = sc.InfluxConf.URL != ""
b.Elastic = len(sc.ElasticConf["default"].Hosts) != 0
b.Annotate = len(sc.AnnotateConf.Hosts) != 0
b.AzureMonitor = sc.AzureMonitorConf["default"].ClientId != ""

This comment has been minimized.

Sign in to view
Copy link
@captncraig

captncraig Jul 31, 2018

Contributor

This feels like a weak thing to check. Are they required to have a "default"? Is ClientID required? (assume thats a credential of some kind?)

This comment has been minimized.

Sign in to view
Copy link
@kylebrandt

kylebrandt Jul 31, 2018

Member

Agreed will make something better, this is left from my first pass
cmd/bosun/conf/system.go
clients.MetricsClient = insights.NewMetricsClient(conf.SubscriptionId)
clients.MetricDefinitionsClient = insights.NewMetricDefinitionsClient(conf.SubscriptionId)
clients.ResourcesClient = resources.NewClient(conf.SubscriptionId)
if conf.DebugRequest {

This comment has been minimized.

Sign in to view
Copy link
@captncraig

captncraig Jul 31, 2018

Contributor

Do we really need these debug things long term? I'd probably prefer not having a million different config options if we can avoid it.

This comment has been minimized.

Sign in to view
Copy link
@kylebrandt

kylebrandt Jul 31, 2018

Member

Just two options. Could help with people debugging issues, in particular since I saw what I think is a bug in the SDK about error being missed.
cmd/bosun/conf/system.go Outdated
if err != nil {
// Should not hit this since we check for authorizer errors in Validation
// This is checked before because this method is not called until the an expression is called
slog.Fatal("Azure conf: ", err)

This comment has been minimized.

Sign in to view
Copy link
@captncraig

captncraig Jul 31, 2018

Contributor

Fatal feels wrong here. Does this method get invoked every time an expression runs? If azure is down intermittently, could this crash bosun?

This comment has been minimized.

Sign in to view
Copy link
@kylebrandt

kylebrandt Jul 31, 2018

Member

I don't think this actually make any API calls and is just a validity thing. So it should be caught by Valid() method, but added this here in case I'm wrong.
cmd/bosun/conf/system.go Outdated
return allClients
}
func azureLogRequest() autorest.PrepareDecorator {

This comment has been minimized.

Sign in to view
Copy link
@captncraig

captncraig Jul 31, 2018

Contributor

Are these even required long term?
cmd/bosun/expr/azure.go
F: AzureMultiQuery,
PrefixEnabled: true,
},
"azmd": { // TODO Finish and document this func

This comment has been minimized.

Sign in to view
Copy link
@captncraig

captncraig Jul 31, 2018

Contributor

todo
cmd/bosun/expr/azure.go
F: AzureResourcesByType,
PrefixEnabled: true,
},
"azrf": {

This comment has been minimized.

Sign in to view
Copy link
@captncraig

captncraig Jul 31, 2018

Contributor

azrf or azfr? Inner function is "FilterResources", so maybe azfr matches better? I kinda hate short names.

This comment has been minimized.

Sign in to view
Copy link
@kylebrandt

kylebrandt Aug 1, 2018

Member

I think like azrf because it shares the azr prefix with azrt
cmd/bosun/expr/azure.go
}
// AzureMetricDefinitions fetches metric information for a specific resource and metric tuple
// TODO make this return and not fmt.Printf

This comment has been minimized.

Sign in to view
Copy link
@captncraig

captncraig Jul 31, 2018

Contributor

todo
cmd/bosun/expr/azure.go
return
}
}
st := e.now.Add(time.Duration(-sd)).Format(azTimeFmt)

This comment has been minimized.

Sign in to view
Copy link
@captncraig

captncraig Jul 31, 2018

Contributor

"startTime" and "endTime" would not be unreasonable names.

This comment has been minimized.

Sign in to view
Copy link
@kylebrandt

kylebrandt Jul 31, 2018

Member

When this is done I want to go back and make a method on the State that does since we use it all the time.

kylebrandt added some commits Jul 18, 2018

@kylebrandt
cmd/bosun: azure datasource 
this includes an incomplete azure metadata function with TODOs to be
completed in a future commit.
a83dec8
@kylebrandt
vendor: azure go sdk
7bf4161
@kylebrandt
cmd/bosun: allow comments after last line in web expr ui (keep this c… 
…ommit)
Loading status checks…
07dc248

@kylebrandt kylebrandt merged commit bcd2335 into master Aug 1, 2018
3 checks passed

3 checks passed

bosun All checks Passed!
continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details

@kylebrandt kylebrandt deleted the ads branch Sep 27, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment