Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add allocation binpacking duration metric #687

Merged
merged 2 commits into from
Sep 20, 2021

Conversation

cjerad
Copy link
Contributor

@cjerad cjerad commented Sep 17, 2021

1. Issue, if available:
#612 "Emit Metrics for Karpenter"

This PR does not fully resolve the issue. More changes will be needed.

2. Description of changes:
Add a histogram metric to record the duration of binpacking operations.

3. Does this change impact docs?

  • Yes, PR includes docs updates
  • Yes, issue opened: link to issue
  • No

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@netlify
Copy link

netlify bot commented Sep 17, 2021

✔️ Deploy Preview for karpenter-docs-prod canceled.

🔨 Explore the source changes: e7213e5

🔍 Inspect the deploy log: https://app.netlify.com/sites/karpenter-docs-prod/deploys/6144fff406064000089faa32

return packings
}

func (p *packer) pack(ctx context.Context, schedule *scheduling.Schedule, instances []cloudprovider.InstanceType) []*Packing {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To remove the need for a private function that just calls the public, you could place this at the top of original and still make minimal code additions to the core packing logic.

Pack:

startTime := time.Now() 
defer packTimeHistogram.Observe(time.Since(startTime).Seconds())

The defer is advantageous on the Observe call anyways since you'll get observed durations even if a panic or some runtime errors wreak havoc.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To remove the need for a private function that just calls the public [...]

It's the other way around: the public method calls the private method. The wrapper allows separating the metric logic from the binpacking logic. Is there an advantage to breaking the separation of concerns?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, sorry mistyped: Public calling Private.

Since the metric logic is tiny (two lines), seems a bit heavy to do the function proxying. The metric logic can be placed at the top with the defer to keep it together to separate it a bit for the reader. The reason to keep the private without metrics is if we have a use-case (maybe in testing) to not emit metrics and use just the pack func. But that doesn't seem like a great thing to do anyways. I would guess it would be configurable elsewhere to make the Observe func a noop, if we don't want metrics.

@@ -63,6 +81,11 @@ type Packing struct {
// It follows the First Fit Decreasing bin packing technique, reference-
// https://en.wikipedia.org/wiki/Bin_packing_problem#First_Fit_Decreasing_(FFD)
func (p *packer) Pack(ctx context.Context, schedule *scheduling.Schedule, instances []cloudprovider.InstanceType) []*Packing {
startTime := time.Now()
defer func() {
packTimeHistogram.Observe(time.Since(startTime).Seconds())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool pattern!

@@ -63,6 +81,11 @@ type Packing struct {
// It follows the First Fit Decreasing bin packing technique, reference-
// https://en.wikipedia.org/wiki/Bin_packing_problem#First_Fit_Decreasing_(FFD)
func (p *packer) Pack(ctx context.Context, schedule *scheduling.Schedule, instances []cloudprovider.InstanceType) []*Packing {
startTime := time.Now()
defer func() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there a reason to wrap this in a func rather than just doing:

startTime := time.Now()
defer packTimeHistogram.Observe(time.Since(startTime).Seconds())

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to Defer, Panic, and Recover from the Go blog:

A deferred function’s arguments are evaluated when the defer statement is evaluated.

So the proposed code will not record the elapsed time because only the .Observe() call is defered; time.Since(startTime).Seconds() will be evaluated immediately.

@cjerad cjerad merged commit 4e9fb54 into aws:main Sep 20, 2021
@cjerad cjerad deleted the allocation-pack-time-metric branch September 20, 2021 13:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants