Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More key feature instrumentation #2811

Merged
merged 5 commits into from
Jan 14, 2021
Merged

More key feature instrumentation #2811

merged 5 commits into from
Jan 14, 2021

Conversation

paolodamico
Copy link
Contributor

@paolodamico paolodamico commented Dec 18, 2020

Changes

Introduces more custom front-end events as planned on #2809.

  • "person viewed" when the user views the person detail page is opened.
  • "annotation viewed" when the user views an annotation pop-up.

In addition:

  • It introduces an initial typing for "Annotation" (in the same spirit as Insights data instrumentation #2787) and fixes
  • Creates an eventUsageLogic to centralize custom front-end events reporting.

Checklist

  • All querysets/queries filter by Organization, by Team, and by User
  • Django backend tests
  • Jest frontend tests
  • Cypress end-to-end tests

@timgl timgl temporarily deployed to posthog-instrumentation-6d3mug December 18, 2020 00:19 Inactive
@timgl timgl temporarily deployed to posthog-instrumentation-6d3mug December 18, 2020 03:21 Inactive
@timgl timgl temporarily deployed to posthog-instrumentation-6d3mug December 18, 2020 03:28 Inactive
@paolodamico paolodamico marked this pull request as ready for review December 18, 2020 03:30
@macobo
Copy link
Contributor

macobo commented Dec 18, 2020

"person viewed" when the user views the person detail page for more than 5s.
"annotation viewed" when the user views an annotation pop-up for more than 1.5s

These definitions seem complex and and thus hard to keep stable over time. Having the definition fluctuate will in turn lead to more complexity in the long-term.

To dig a bit:
What are the timeouts trying to prevent?
Will the results be different enough from the same definition without the timelimit?
Are other teams using analytics implementing these KPIs the same way?

@paolodamico
Copy link
Contributor Author

Thank you so much for taking the time to think deeply about this and the push back. There's more context on #2809, however the goal of these more complex metrics is to have a metric that represents a strong signal that the user is actually using the feature in a way that generates value. For example, merely opening the /insights page is not really valuable. Value is driven when the user actually uses the graph to get some insights. The proxy I'm proposing for this example is for instance, after a user has moved the insights filters and stayed in the page for more than 5s. However I agree with you, we might be going down a rabbit hole that does not really change signaling too much; perhaps the best approach is to test this first.

Regardless, just documenting the above for posterity, as we've agreed to on Slack, we'll continue this conversation in Jan.

@paolodamico
Copy link
Contributor Author

Ready for another look @macobo, as discussed, it doesn't make sense right now to over-complicate things with an arbitrary timeout period. I'd appreciate if you can help me test the E2E behavior.

Copy link
Contributor

@macobo macobo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added some possible issues, LGTM otherwise (feel free to merge once resolved)

},
listeners: {
reportAnnotationViewed: async ({ annotations }: { annotations: AnnotationType[] | null }, breakpoint) => {
if (!annotations) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did not test this, but I'd assume breakpoint(500) would need to be first right - otherwise we'd still report it as viewed.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Q: Is it worth having code for the cancel case at all - KISS.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't think the breakpoint should go first, because the annotations won't change while the breakpoint is happening. If the variable is null, why wait, it will never lead to something getting reported anyways.

I clarified the comment a bit. Basically if the var is not set, we can't report usage.

frontend/src/lib/utils/eventUsageLogic.ts Outdated Show resolved Hide resolved
@paolodamico paolodamico merged commit d91cc21 into master Jan 14, 2021
@paolodamico paolodamico deleted the instrumentation-d17 branch January 14, 2021 13:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants