Sprint - June 10 to June 21, 2024

[marandaneto]

Feature request

Global Sprint Planning

3 things that might take us down

Team sprint planning

For your team sprint planning copy this template into a comment below for each team.

# Team ___

**Support hero:** ___

## Retro

<!-- Grab the high and low priority items from last time and add whether that item was completed or not -->

- 

## Hang over items from previous sprint

<!-- For each item, decide to re-prioritise (and add below) or deprioritise -->

- Item 1. prioritised/deprioritise

## OKR

1. OKR, status (red/yellow/green) and action points if yellow/red


### High priority

-

### Low priority / side quests

-

[marandaneto]

Team Error Tracking

Support hero: @pauldambra

• manoel (10th) and paul out a day

items from previous sprint

High priority

• hiring (@pauldambra) 🟡
• hogql filters performance (@pauldambra ) - (check with @mariusandra - most likely 💚)
  • we have marius change and will roll it out
  • i think we need to go further but we have marius change as a template
• users like the screenshot option for android - so add it to iOS (@marandaneto) 🟡
  • should have a draft PR either today or tomorrow for review
• continuing CSS issues as most important customer problem (@daibhin) 💚
  • CSS fix but there are some issues capturing mutations

OKR

1. OKR, status (red/yellow/green) and action points if yellow/red

• 🟡 quality and consolidation
  • mobile replay open beta
    • out free and being promoted in onboarding
  • replay captures every site
    • David working with rrweb to improve CSS parsing, already a major reduction in issues as a result
  • people can find the valuable recordings
    • queued behind the others
• 🟡 Error tracking MVP
  • need to decide if we start
• 🟡 Hire at least 2 amazing colleagues

High priority

• hiring Product Eng. (@pauldambra) and Mobile Eng. (@marandaneto)
• masking text in the screenshot images on Android and iOS @marandaneto
• Error tracking (@pauldambra and @daibhin)
  • we already gather $exception events
  • let's add a UI to explore them
  • and test generating embeddings for them as they're ingested
  • this would give us grouping
  • and let's us start testing this internally without any extra ingestion load
  • see https://github.com/PostHog/product-internal/blob/2651956b8abd06f3db368e7f681c375653193db4/requests-for-comments/2024-05-01-error-tracking-mvp.md for context

Low priority / side quests

• mobile usage report stuff @pauldambra
• persistent replay queue in posthog-js @pauldambra
• icons left/right for android wireframes @daibhin
• we are recruiting beta testers for Android and iOS - (we should have screenshots now) @marandaneto @annikaschmid

[benjackwhite]

Team CDP HogHooks

(Now @mariusandra and @benjackwhite ) - new team, using hog to power cdp

Retro

• Focus issues at first, got resolved
• Ben and Marius figured out how to split work effectively (Ben plumbing, Marius compiling)
• Got a MVP up in a few days which felt great

Hang over items from previous sprint

N/A

OKR

1. Get the new CDP in front of real users

Sprint plan

• Have the new "CDP" service in production, flagged to Team 2 @benjackwhite
• Get all of our existing Zapier actions (e.g. slack notifications) running as hog functions (dog-fooding) @benjackwhite
• Missing hog features to support the above (related issue) @mariusandra

[neilkakkar]

Team Feature Success

Support hero: @jurajmajerik then @Phanatic
Days off:
Juraj: 0 days
Phani: 5 days
Dylan: 3 days
Neil: 1 days

Retro

• Finish up repeated surveys - @Phanatic ✅
• next start on Event based triggers for surveys - https://github.com/PostHog/product-internal/pull/556 - @Phanatic 🟡
• Delay Survey popup #18102 - @dmarticus - 🟡
• html rendering bug, and NPS survey result fix (ex: this survey ) - @dmarticus - ✅
• Surveys popup refactor - @jurajmajerik - ✅
• Feature Flags Rewrite: List of things to get done #22131 continue on rewrite - @neilkakkar - property matching works now!

Hang over items from previous sprint

---

OKRs

1. Make sure feature flags can handle 10x current scale
2. Polish new experiments UI & collect feedback
3. Add most requested surveys functionality

High priority

• Extend event based triggers - @Phanatic
• Surveys branching logic - @jurajmajerik
• Survey popup UI polish (add all issues here) #20851 - @dmarticus
• Feature request: Survey revision history #22516 - @dmarticus
• Make a monorepo (RFC: Bringing back the monorepo for rust services and main app meta#207 ) - @neilkakkar
• Feature Flags Rewrite: List of things to get done #22131 continue on rewrite - @neilkakkar

Low priority / side quests / maybe Neil will get to this next year

• Temporal queues for feature success - @neilkakkar
• Setup instrumentation for flip-flopping problem of experiment significance - @neilkakkar

[EDsCODE]

Team Data Shack

OKR Q2 2024

Objective

Release data warehouse to everyone

• Key Results:
  • Integration first experience
    • schemas are reliable
    • modeling of each integration is clear
    • Good automatic roll up views and joins
    • Wizard to onboard people
  • Establish a solid pattern to build integrations
  • Complete data warehouse experience in the rest of the app (insights, feature flags, experiments)

Retro

Product

• complete Data Warehouse Public Beta Checklist #22001 (decide on pricing, UI changes, instrumentation) @EDsCODE
• Warehouse props in breakdown filters @Gilbert09
• warehouse props in feature flags and experiments

Infra

• move to team clickhouse!

High Priority

• incremental syncs @Gilbert09
• templated joins @EDsCODE
• warehouse props in feature flags and experiments
• person model batch exports @tomasfarias

[frankh]

Team Infra

Hang over items from previous sprint

1. 🔴 Investigate canary deploy rollback improvements @frankh
   • reworking how deploys are done in general to simplify rolling back before working on canary specific rollbacks/locks
2. 🟢 metabase pro set up
   • do we actually need pro? We can set up google sign on with free edition @benjackwhite

OKR

1. 💪 Deploy with confidence 🟢

• Finalize our Canary Deploy process 🟡

5. 🚨 Improved alerting and monitoring 🟢

• SIEM work has made this a priority

6. 🔒 Deeper Security 🟡
7. 💰 Continued cost control 🟢

High priority

• SEIM:
  • access logs to django @benjackwhite (fix axes?)
  • cloudtrail dashboard @danielxnj
  • eks audit dashboard @danielxnj
  • documentation/policy for sharing with auditors/clients (in cloud-infra?) @danielxnj
  • metabase logs into loki
• secrets
  • vault monitoring+backups @danielxnj
  • CDP integration @ZeleniJure
• placeholder cost optimisation tasks @frankh (to be decided in cost optimisation meeting)
• roll out new codified deployment strategy @frankh
• internal RBAC review (tbc)

For planning session

• Kafka S3 backed needs testing so we can move over to it
• Need a plan for RDS in terms of how Persons will move forward

[fuziontech]

Team Click Haus

OKR Q2 2024

Objective

James as a Service -> Clickhouse as a Service

• Key Results:
  • Better Visibility
    • Regularly testing backups
    • Monitoring/alerting
    • Mutations
    • Moves
  • Management
    • Managing/killing mutations
  • Self Serve
    • Schema design feedback (James non blocking
    • Schema management
  • Automation
    • Replace/Upgrade replicas
    • Upgrading to 24.04
    • Disk configs

Retro

Infra

• 🗑️ Fix delete code path
• ⬆️ Upgrade EU CH version Must complete
• 🏗️ Rebuild very broken EU coordinator
• 🗒️ Develop long-term plan for CH strategy
• 🔥 Fire-fighting replication issues in EU w/ sharded_sessions
• ⏩ Move parts around so last 3 months of data are on NVME on US

High priority

Infra

• ✍️ Write up the outcome from CH strategy planning
• 🏃 Onboarding plan for Dani
• 📟 Monitoring and Alerting on EU Coordinator
• 🧪 Test incremental backup restores
• ♻️ Replace offline nodes w/ nvme nodes
• 🗑️ Delete persons on teams that are still ingesting data (for personless events)
• 👯 Debug duplicate events in sharded_sessions on EU (bad replication)

[raquelmsmith]

Team Growth

Retro

Retro items

High priority

• @raquelmsmith
  • UI for personless events
    • App billing page
    • App pricing tables
    • Onboarding instructions
    • posthog.com pricing tables
  • Launch personless events / project management
  • Referrals: add/edit referrer and redeemer
• @xrdt
  • Deprecate available_features
  • Support multiple org owners
    • Can only invite at your perm level or lower
    • Can set membership level when inviting
  • Billing admin improvements - make plans_map a dropdown of selects
  • Doc for how to use multiple stripe customers to a single django admin customer
    • Not standalone doc
  • Maintenance
    • Track celery queue depth for billing workers
    • Check whether billing limits are being applied correctly - see https://posthog.slack.com/archives/C043VJ93L3B/p1715964424235799
    • Check whether mrr is being forecasted correctly for upcoming invoices
      • Add a field to the DB model for the current accumulated total so we can more easily debug future issues
• @zlwaterfield
  • Finish teams addon changes
    • remove old teams plan
    • make sure enterprise customers are on proper plans
    • Add includes team plan to enterprise plan in default config
  • Finish startup plan analysis
    • how many credits are being used each month
    • fix the starting/ending balance bug
  • Stripe audit stuff
    • mismatch subscriptions script
    • emailing users for failed payments
  • Start subscribing to all products
  • Side quests:
    • Teams annual plan

Q2 Goals

✅=finished 🟡=in progress 🔴=won't finish

1. ⚪ Create a flow in product analytics onboarding to fill out a dashboard template using actions (Raquel)
2. 🟡 Simplify our subscription flows (Zach, supported by Raquel)
3. 🟡 Launch pricing changes (Bianca, Raquel)
   • 🟡 Personless events - will help us reach more customers at an affordable price
   • Data warehouse - it's becoming pretty useful, we should charge for it
   • Session replay - we can reduce costs to improve retention and reach more people

This sprint

• @raquelmsmith
  • Finish roll-out of personless events
    • Figure out why conversion form anon -> identified isn't working for us
    • Update migrate script to also report usage at time of migration
    • Roll out to 5-10k customers at a time
    • Fix changelog link
    • Fix sentry errors
  • Start working on toolbar dashboard template thing
  • Prune usage report queries
  • Low priority
    • Referrals: add/edit referrer and redeemer
• @zlwaterfield
  • clean up outstanding feature flags (reverse-proxy-onboarding and email-verification-ticket-submission)
  • ship outstanding PRs (Teams addon - remove modal, Projects pay gate on free should be to paid not teams)
  • subscribe to all products
    • main API changes (basic serializer set up, products=all query, default paid/free plan map, functions for upgrading/downgrading
    • feature flag in UI on billing page with main button alterations (no comparison or other UI changes)
    • Make sure billing limit field is always visible
  • startup plan metadata RFC
  • emails new Teams addon customers
  • re-run the plans map and compare with the new auto-cancel functionality

• @xrdt
  • Billing api performance improvements
  • Data warehouse pricing?
  • Making sync invoices reactive
  • Billing limits in onboarding
  • Billing breakdown improvements
    • Show autocapture events
    • Show per-project usage breakdowns

[tiina303]

Team Pipeline

Off: Brett 4 days
Support: Brett

Retro

High priority

• Kafka migration (Brett)
• Capture-rs successfully processing all /batch requests on EU (Xavier)
• Changing what data pipeline add-on covers (Tiina)
  • not done, tbd what the new CDP team wants to do
• Fast configuration option to send events to overflow from capture for replay events (Tomas)
• Fixing partition stats (Ted)

OKR

✅=finished 🟢=on track to finish this quarter 🟡=might not finish 🔴=won't finish
✔️=progressed last sprint ; ➡️=planned work for this sprint

🟢✔️ Deprecate posthog-events by moving to capture-rs fully (21659)
🟢✔️ Visibility into what's in the ingestion queues and past performance (20985)
🟢✔️ Fast configuration options to speed up incident recovery, e.g. by token send to overflow or drop (21662)
🟢➡️ Iterate on person processing to make it faster and cheaper (21048)
🟢✔️ Batch exports UX improvements, e.g. error notifications and UI rewamp (21139)
_ Person data batch export (20339)
-> moving to Data Warehouse team
🟢➡️ Support adding new products (21665)
🟢✔️ Deprecate scheduler & jobs deployments, runEveryX plugins and kafkajs consumers (21656)

High priority

• Improve ingestion efficiency by moving the last Kafka waits at the end of the batch (Brett)
• New customers write to person only from $set and $identify, $create_alias, $merge_dangerously and new products also only read persons (Tiina)

Low priority / side quests

• [ ]

[robbie-c]

Team Bounce Rate

Support hero: @robbie-c

Retro

Had 5 days off this sprint, support was quite busy. Have passed on some issues to relevant teams (sorry product analytics)

• 🟢 test left join changes on persons table (sort of, we could test this more, haven't included this in this sprint but open to discussing ofc)
• 🟢 run personless upgrading test
• 🟡 write the SQL for the new sessions table that uses it

Torn between 2 things a bit:

• there's a pretty bad clickhousey bug with the sessions table on EU, somehow during ingestion events are double count (not in events table, only in sessions table which uses a clickhouse materialized view). This means that all bounce rate calculations are 0%. We already wanted to replace this and I've been working on a new design, so could prioritise this
• there's also still a lot left to do on personless events

I've prioritised the personless events due to it being less total work, more time sensitive, has more dependencies on it

OKR

1. Make querying fast enough for large customers
2. Do personless events work where necessary
3. Iterate on customer feedback
4. Product management work

High priority

• Start emailing customers for research
• Posthog-js: Rewrite posthog-js person props backfilling to include initial current url
• Posthog-js: handle setPersonPropertiesForFlags properly when personless
• Posthog-js: handle manually calling $set properly when personless
• Write RFC for client changes for personless events based on posthog-js, to be approved by mobile sdk teams

Low priority

• Sessions table: Profile different versions of the sessions table, this time with SAMPLE-ing, see Sessions table, profiling different ORDER BY and SAMPLE BY

[Twixes]

Team Caching

Support hero: @webjunkie

Time off: Thomas for 2 days

Retro

High priority work

• 🟢 Fix the most painful bugs reported by users: (we're on the way down with the pile of open tickets, currently at 90) @all-of-product-analytics A lot of improvements to loading data, and more random fixes
  • Related HogQL-based insights (actor modals, exports, caching reliability)
  • Related to complexity in dashboardLogic
• 🔴 Clean up the legacy ways of tracking performance We didn't get to it

Low priority work / side quests

• 🟢 Loading Bar™ @aspicer It should be good to go in terms of going live
• 🟠 Ensure live events in the Activity view are shipped Just needs some infra-level touch on auth in prod and docker-compose
• Project environments on hold Have been on hold

Team

• In terms of reviews/context, when someone switches teams (e.g. Marius), it would be great to have a clear transition period
• It's tough to meet the support SLAs - this needs more thinking on our part

OKR

• HogQL-based querying

  • Convert the remaining legacy queries to HogQL and release to public (Thomas, Julian, Marius)
    • 🟢 Insights – rolled out, although fixes needed to be rock-solid
    • ⚪ Cohorts
  • Remove legacy querying backend (Thomas, Julian)
    • 🟢 Clean up or rewrite dashboardLogic
    • 🟠 Convert filters to query (insights, notebooks, activity log, experiments) 👈 Thomas is on it
  • Missing Product Analytics features (Thomas, Julian)
    • 🟠 Breakdowns (multiple) in literally everything 👈 Georgii hopefully will be on it
    • 🟠 Make a list based on GitHub issues from customer requests…
    • ⚪ Fix those issues
  • Missing HogQL features (Tom, Marius)
    • 🟠 Type system, JSON 👈 Tom is on this, being in Data Warehouse
    • ⚪ Missing things when building funnels

• Querying and processing performance (Thomas, Julian)

  • Global performance overview dashboards 👈 Sandy is on it
    • 🟠 Insights
    • ⚪ Exports
    • ⚪ Cohort recalculations
  • Query request tracing
    • ⚪ Possibly query runner Python optimizations
    • ⚪ Exports improvements
  • ⚪ Identify top 5 query optimizations in terms of impact

• Artificial Hog / Post Intelligence (Michael)

  • ⚪ Ask a question to get a magical insight (aware of your taxonomy)
  • ⚪ Figure out infra for upgrading queries and models
  • ⚪ Product-wide framework for opting into sharing with OpenAI

• Activation (side quest: Michael)

  • ⚪ Michael to work with Growth to identify optimizations to getting started with Product Analytics 👈 pending the product manager hire a bit

High priority

• Getting rid of remaining legacy filters use (owner: @thmsobrmlr)
• Query performance measurement (owner: @aspicer)
• Project Environments finally (owner: @Twixes)

Low priority / side quests

• Insight background reloads monitoring/cleanup (@webjunkie)
• Onboarding Georgii (owner: @Twixes)