Skip to content
This repository has been archived by the owner on Oct 17, 2018. It is now read-only.

Generic code template for consolidated element logic #95

Merged
merged 4 commits into from
Mar 29, 2018

Conversation

xichen2020
Copy link
Contributor

cc @cw9 @jeromefroe

This PR factors out common code for aggregation elems for different aggregation elements and uses genny to generate type-specific implementations for reduced code duplication and maintenance overhead.

This PR is against dev branch.

require.Equal(t, testCounter.CounterVal, e.values[0].counter.Sum())
require.Equal(t, int64(1), e.values[0].counter.Count())
require.Equal(t, int64(0), e.values[0].counter.SumSq())
require.Equal(t, testCounter.CounterVal, e.values[0].aggregation.Sum())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should make all the Sum() Count() ... functions private in those aggregation types since only ValueOf() is being used by element now? And these test here should not verify the results of functions like Sum() or Count() but should verify the result of ValueOf()?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure can do in the next PR, would like to only include the code-gen related changes in this PR

) {
if useDefaultAggregation {
e.quantiles = aggTypesOpts.TimerQuantiles()
e.isQuantilesPooled = false
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can this check be replaced with just e.quantilesPool = nil?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm not sure I understand, you need to check isUseDefaultAggregation to determine how to set e.quantilesPool

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

think you can just do e.quantilesPool = nil here since you are resetting with the aggTypesOpts

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you suggesting removing e.isQuantilesPooled?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm I think it's less readable that way to save one boolean var.

	if useDefaultAggregation {
		e.quantiles = aggTypesOpts.TimerQuantiles()
		e.quantilesPool = nil
		return
	}

	var (
		quantilesPool     = aggTypesOpts.QuantilesPool()
		isQuantilesPooled bool
	)
	e.quantiles, isQuantilesPooled = aggTypes.PooledQuantiles(quantilesPool)
	if isQuantilesPooled {
		e.quantilesPool = quantilesPool
	} else {
		e.quantilesPool = nil
	}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated btw

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry, did not realize that you need the extra logic to check on the isQuantilesPooled, I'm ok either way in that case

// GenericElem is an element storing time-bucketed aggregations.
type GenericElem struct {
elemBase
genericElemBase
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

both elemBase and genericElemBase sound very generic and they sounded similar, maybe rename genericElemBase to genericTypedElemBase?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm I initially called it metricTypeElemBase but landed on just genericElemBase. I can rename to typeSpecificElemBase sure.

// NB(cw) Use default suffix slice for faster look up.
suffixes := e.DefaultAggregationTypeStrings(e.aggTypesOpts)
aggTypes := e.DefaultAggregationTypes(e.aggTypesOpts)
agg.Lock()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did we have this lock before? why do we need it now?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah it was there before (in Consume in elem.go). This is simply moving it into the processValues method.

return
}

agg.Lock()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's the thinking about merging this with the default case?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure can do in the next PR, would like to only include the code-gen related changes in this PR

Copy link
Contributor

@cw9 cw9 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@xichen2020 xichen2020 merged commit f9eb2df into dev Mar 29, 2018
@xichen2020 xichen2020 deleted the xichen-consolidate-elem-codegen branch March 29, 2018 13:11
xichen2020 added a commit that referenced this pull request Jun 8, 2018
* Ingest untimed metrics with pipeline metadata (#92)

* Generic code template for consolidated element logic (#95)

* Aggregation elements flush or forward metrics based on pipeline metadata (#96)

* Better test coverage for new aggregator logic (#97)

* Fix build

* Update travis
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants