Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/bosun: Azure Monitor Datasource #2283

Merged
merged 5 commits into from Aug 1, 2018
Merged

cmd/bosun: Azure Monitor Datasource #2283

merged 5 commits into from Aug 1, 2018

Conversation

@kylebrandt
Copy link
Member

@kylebrandt kylebrandt commented Jul 20, 2018

No description provided.

@kylebrandt kylebrandt force-pushed the ads branch 2 times, most recently from e22f47c to 1fc3f64 Jul 30, 2018
@kylebrandt
Copy link
Member Author

@kylebrandt kylebrandt commented Jul 31, 2018

For review except the metric metadata function which can be ignored for now. Also will vendor later. cc @mhenderson-so . Note: the azure datasource commit by itself has some cache edits by accident and doesn't build, so I will probably just squash all this into one when time ready for master. (fixed in rebase and force push)

@kylebrandt kylebrandt requested a review from captncraig Jul 31, 2018
@kylebrandt kylebrandt changed the title cmd/bosun: (WIP) Azure Monitor Datasource cmd/bosun: Azure Monitor Datasource Jul 31, 2018
DebugResponse bool
}

// Valid returns if the configuration for the AzureMonitor

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018
Author Member

fix comment

return allClients
}

func azureLogRequest() autorest.PrepareDecorator {

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018
Author Member

Comment this function and the following, add comment to attribute source of readme from the go azure sdk

This comment has been minimized.

@captncraig

captncraig Jul 31, 2018
Contributor

Are these even required long term?

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018
Author Member

I don't think they hurt, and since the documentation is marking the experimental for now they could be handy. I would probably be better if more datasources had this I think

if err != nil {
slog.Warningf("failure to dump azure request: %v", err)
}
dump, _ := httputil.DumpRequestOut(r, true)

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018
Author Member

check + log err

if err != nil {
slog.Warningf("failure to dump azure request: %v", err)
}
dump, _ := httputil.DumpResponse(r, true)

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018
Author Member

check + log err

},
}

// Tag function for the "az" expression function

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018
Author Member

comment: azTags is

return azureTags(args[2])
}

// Tag function for the "azmulti" expression function

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018
Author Member

comment: azMultiTags is


// azureTags adds tags for the csv argument along with the "name" and "rsg" tags
func azureTags(arg parse.Node) (parse.Tags, error) {
tags := parse.Tags{"name": struct{}{}, "rsg": struct{}{}}

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018
Author Member

make name and rsg constants and change in all places where referenced


const azTimeFmt = "2006-01-02T15:04:05"

func azResourceURI(subscription, resourceGrp, Namespace, Resource string) string {

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018
Author Member

comment func

return
}

// AzureQuery queries an Azure monitor metric for the given resource and returns a series set tagged by

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018
Author Member

finish this func comment

// Verify prefix is a defined resource and fetch the collection of clients
cc, clientFound := e.Backends.AzureMonitor[prefix]
if !clientFound {
return r, fmt.Errorf("azure client with name %v not defined", prefix)

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018
Author Member

... "%v" ...

st := e.now.Add(time.Duration(-sd)).Format(azTimeFmt)
en := e.now.Add(time.Duration(-ed)).Format(azTimeFmt)

// Set Dimensions (tag) keys for metrics that support them by building a filter

This comment has been minimized.

tg = azureIntervalToTimegrain(interval)
}

// Set azure aggregation method

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018
Author Member

Find various "azure" comments and replace with "Azure" (casing)

if err != nil {
slog.Errorf("failure to parse remaning reads from azure response")
} else {
collect.Sample("azure.remaining_reads", opentsdb.TagSet{"prefix": prefix}, float64(readsRemaining))

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018
Author Member

add comment explaining why this is sampled

// collectCache is a helper function for collecting metrics on
// the expression cache
func collectCacheHit(cacheName, qType string, hit bool) {
tags := opentsdb.TagSet{"query_type": qType, "name": cacheName}

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018
Author Member

add metadata for these metrics

}
series := make(Series)
tags := make(opentsdb.TagSet)
tags["rsg"] = rsg

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018
Author Member

ref constant here

}
}
for _, mValue := range *dataContainer.Data {
exValue := azureExtractMetricValue(&mValue, aggLong)

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018
Author Member

add comment

for res := range resCh {
queryResults = append(queryResults, res)
}
// Merge the query results into a single seriesSet

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018
Author Member

skip merge if length of set is not gt than 1

// or tags associated with that resource
func AzureFilterResources(e *State, T miniprofiler.Timer, resources AzureResources, filter string) (r *Results, err error) {
r = new(Results)
bqf, err := boolq.Parse(filter)

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018
Author Member

add comment

if len(sp) != 2 {
return false, fmt.Errorf("bad filter, filter must be in k:v format, got %v", filter)
}
key := strings.ToLower(sp[0]) // Make key case insensitive

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018
Author Member

note case insensitivity in expression docs

key := strings.ToLower(sp[0]) // Make key case insensitive
value := sp[1]
switch key {
case "name":

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018
Author Member

consts again for name, rsg

if re.MatchString(ar.Name) {
return true, nil
}
case "rsg", "resourcegroup":

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018
Author Member

remove "resourcegroup" alias, no real point, update expr docs to reflect

case string(insights.Maximum):
v = mv.Maximum
case string(insights.Total):
v = mv.Total

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018
Author Member

add missing None aggregation

This comment has been minimized.

@kylebrandt

kylebrandt Aug 1, 2018
Author Member

nm, None isn't a field in the sdk

return string(insights.Maximum), nil
case "total":
return string(insights.Total), nil
case "count":

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018
Author Member

add missing none aggregation here as well

Copy link
Contributor

@captncraig captncraig left a comment

Seems pretty good to me. Just finish out todos and comments.

@@ -84,6 +94,7 @@ func (sc *SystemConf) EnabledBackends() EnabledBackends {
b.Influx = sc.InfluxConf.URL != ""
b.Elastic = len(sc.ElasticConf["default"].Hosts) != 0
b.Annotate = len(sc.AnnotateConf.Hosts) != 0
b.AzureMonitor = sc.AzureMonitorConf["default"].ClientId != ""

This comment has been minimized.

@captncraig

captncraig Jul 31, 2018
Contributor

This feels like a weak thing to check. Are they required to have a "default"? Is ClientID required? (assume thats a credential of some kind?)

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018
Author Member

Agreed will make something better, this is left from my first pass

clients.MetricsClient = insights.NewMetricsClient(conf.SubscriptionId)
clients.MetricDefinitionsClient = insights.NewMetricDefinitionsClient(conf.SubscriptionId)
clients.ResourcesClient = resources.NewClient(conf.SubscriptionId)
if conf.DebugRequest {

This comment has been minimized.

@captncraig

captncraig Jul 31, 2018
Contributor

Do we really need these debug things long term? I'd probably prefer not having a million different config options if we can avoid it.

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018
Author Member

Just two options. Could help with people debugging issues, in particular since I saw what I think is a bug in the SDK about error being missed.

if err != nil {
// Should not hit this since we check for authorizer errors in Validation
// This is checked before because this method is not called until the an expression is called
slog.Fatal("Azure conf: ", err)

This comment has been minimized.

@captncraig

captncraig Jul 31, 2018
Contributor

Fatal feels wrong here. Does this method get invoked every time an expression runs? If azure is down intermittently, could this crash bosun?

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018
Author Member

I don't think this actually make any API calls and is just a validity thing. So it should be caught by Valid() method, but added this here in case I'm wrong.

return allClients
}

func azureLogRequest() autorest.PrepareDecorator {

This comment has been minimized.

@captncraig

captncraig Jul 31, 2018
Contributor

Are these even required long term?

F: AzureMultiQuery,
PrefixEnabled: true,
},
"azmd": { // TODO Finish and document this func

This comment has been minimized.

@captncraig

captncraig Jul 31, 2018
Contributor

todo

F: AzureResourcesByType,
PrefixEnabled: true,
},
"azrf": {

This comment has been minimized.

@captncraig

captncraig Jul 31, 2018
Contributor

azrf or azfr? Inner function is "FilterResources", so maybe azfr matches better? I kinda hate short names.

This comment has been minimized.

@kylebrandt

kylebrandt Aug 1, 2018
Author Member

I think like azrf because it shares the azr prefix with azrt

}

// AzureMetricDefinitions fetches metric information for a specific resource and metric tuple
// TODO make this return and not fmt.Printf

This comment has been minimized.

@captncraig

captncraig Jul 31, 2018
Contributor

todo

return
}
}
st := e.now.Add(time.Duration(-sd)).Format(azTimeFmt)

This comment has been minimized.

@captncraig

captncraig Jul 31, 2018
Contributor

"startTime" and "endTime" would not be unreasonable names.

This comment has been minimized.

@kylebrandt

kylebrandt Jul 31, 2018
Author Member

When this is done I want to go back and make a method on the State that does since we use it all the time.

@kylebrandt kylebrandt force-pushed the ads branch 2 times, most recently from 7919b3d to 4dce241 Aug 1, 2018
kylebrandt added 2 commits Aug 1, 2018
this includes an incomplete azure metadata function with TODOs to be
completed in a future commit.
@kylebrandt kylebrandt merged commit bcd2335 into master Aug 1, 2018
3 checks passed
3 checks passed
bosun All checks Passed!
continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details
@kylebrandt kylebrandt deleted the ads branch Sep 27, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

2 participants
You can’t perform that action at this time.