-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Refactor EventTimer + Changes to commercial metrics #889
Conversation
169818c
to
035738b
Compare
035738b
to
0bd596f
Compare
0bd596f
to
a856b12
Compare
🦋 Changeset detectedLatest commit: 2c28c61 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
65265a2
to
3d3adc5
Compare
3d3adc5
to
e3f5d91
Compare
🚀 0.0.0-beta-20230706105953 published to npm as a beta release |
🚀 0.0.0-beta-20230706154936 published to npm as a beta release |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the very helpful diagram!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice one!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice work and thanks for the thorough explanation! Just some comments on naming conventions that we can discuss.
type TrackedSlot = (typeof trackedSlots)[number]; | ||
|
||
// marks that we want to save as commercial metrics | ||
const slotMarks = ['slotReady', 'adRenderStart', 'adOnPage'] as const; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can talk about this but I was hoping to align the naming convention for performance marks across commercial, webex and cmp to make data analysis easier.
I was thinking of a prefix e.g.:
gu.commercial
or gu.dcr
or gu.cmp
And then adopting a naming pattern with a consistent separator that goes generic -> detailed from left to right e.g.
gu.commercial.boot.start
gu.commercial.boot.end
gu.commercial.boot.duration
gu.commercial.slot.define.top-above-nav.start
gu.commercial.slot.define.top-above-nav.end
gu.commercial.slot.define.top-above-nav.duration
or
gu.dcr.island.hydration.mostviewed.start
gu.dcr.island.hydration.mostviewed.end
gu.dcr.island.hydration.mostviewed.duration
These are just ideas but 'duration' aligns with the measures we are computing here.
If we can agree on a naming pattern that works across the teams it would be ideal if we can implement it in this PR for commercial (or another before releasing).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Have raised this as a separate cross-team RFC so this is not a blocker to this PR.
We can update the naming as a follow up PR.
const slotMeasures = [ | ||
'adRender', | ||
'defineSlot', | ||
'prepareSlot', |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Very minor but I think in the PR diagrams loadSlot
and prepareSlot
were used interchangeably?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I changed my mind about the name at some point, will update the diagram
Could you update the handbook when this is live? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great work! 🎉
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking forward to seeing these timers roll out!
What does this change?
Refactor event-timer.ts with the following changes
Why?
First timer removal
Using the
first
for each event has led to some inconsistencies in the data, for roughly 21% of data points the timers are recorded in the wrong order, we believe could be caused by the first event of each type coming from different slots in some circumstances.In most cases the first slot is simply top-above-nav anyway, and using pageviews where that isn't the case could be adding some undesirable noise/variables to the data.
We also have very little insight into what goes on on ad slots further down the page, we we will add reporting for inline1 (often a video ad) and inline2 (floated right on desktop).
Deltas & new measurements
Most of the timers we don't actually even use for analysis, only
adOnPage
generally. Changing most of them to deltas will give us more useful information more easily about specific pieces of code that we're interested in e.g. how long does prebid, defineSlot (which incorporates IAS), loadSlot (fetching the ad from GAM) take.Hopefully deltas will be superior timestamps as they won't be subjected to variance from before the timer we're interested in has even started.
What we're doing in diagram form, we are keep commercialStart and adOnPage (green) as timestamps, throwing away the rest (red) and adding delta's (in blue)
Summary of deltas
Summary of timers