Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Turbo mode #6632

Merged
merged 43 commits into from Oct 26, 2021
Merged

Turbo mode #6632

merged 43 commits into from Oct 26, 2021

Conversation

mariusandra
Copy link
Collaborator

@mariusandra mariusandra commented Oct 25, 2021

Changes

  • Turbo mode! Like in the good old 486DX days!

  • Much better and cleaner version of Load insight from dashboard #6614 and Go back without reloading the previous scene #6600

  • This PR just the logics of previously visited scenes running in the background, instead of rendering the entire scene into the DOM and hiding it with display:none like the previous PRs did.

  • Before:
    2021-10-25 14 11 32

  • Activating Turbo Mode!
    2021-10-25 14 13 57

  • After:
    2021-10-25 14 12 27

  • Flyby bugfixes:

    • Pressing "back" and changing from "cohorts/7" to "cohorts/" now closes the drawer
    • "Jest Tests" pycharm run configuration
    • Split the "sceneLogic" file into "sceneLogic", "scenes" (this one links to all imports now) and "sceneTypes".
    • Split the "events" scene into 4 separately mountable scenes (events, actions, event stats, event property stats). Caching their respective logics is now handled by sceneLogic, instead of internally
    • The /i/:insight_short_id urls did a push, but should have done a replace
    • Remove the age-old code where each Scene was passed user as a prop, instead of taking it from userLogic, from the scenes that still were using this prop
    • Fixed a bunch of sort mutation bugs. Should switch kea's values to ReadOnly<> probably soon!
    • Split up the kea test init code into a function that takes no logics and can be put inside beforeEach()

How did you test this code?

  • A lot of manual testing
  • Added tests to make sure loaded scenes stay in memory in sceneLogic
  • Added test to make sure when opening a dashboard item with the right url, we take the data from the dashboard logic directly

@timgl timgl temporarily deployed to posthog-pr-6632 October 25, 2021 12:16 Inactive
@mariusandra mariusandra temporarily deployed to posthog-pr-6632 October 25, 2021 12:45 Inactive
Comment on lines +64 to +65
} else if (!cohortId) {
actions.setOpenCohort(null)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Flyby fix to a bug, where if you click to open a cohort, and then click back, the URL will change, but the cohort drawer will remain open.

@mariusandra mariusandra temporarily deployed to posthog-pr-6632 October 25, 2021 13:21 Inactive
@mariusandra mariusandra temporarily deployed to posthog-pr-6632 October 26, 2021 10:49 Inactive
@mariusandra mariusandra marked this pull request as ready for review October 26, 2021 10:49
@mariusandra mariusandra temporarily deployed to posthog-pr-6632 October 26, 2021 10:51 Inactive
@mariusandra
Copy link
Collaborator Author

Ready for a look @Twixes , no jest/TS errors locally, hoping for the same in the pending CI run.

Copy link
Collaborator

@Twixes Twixes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is nice overall, but I'm not completely sure if the restructuring of the Events page is good.

  1. Switching to the main Events tab (this does not seem to apply to Actions and other tabs), including from the sidebar, now feels significantly more sluggish.
  2. The sidebar item only is highlighted when in the Events tab, despite other tabs (e.g. Actions) being on the same page from the user's perspective.
    Zrzut ekranu 2021-10-26 o 16 00 10

@@ -26,6 +26,12 @@ export const MOCK_ORGANIZATION_ID: OrganizationType['id'] = 'ABCD'

export const api = apiNoMock as any as APIMockReturnType

export const mockDefaultTeam = {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Micronit: SCREAMING_SNAKE_CASE would be a bit more fitting

@@ -77,6 +82,22 @@ export function App(): JSX.Element | null {
return showingDelayedSpinner ? <SceneLoading /> : null
}

function LoadedSceneLogic({ scene }: { scene: LoadedScene }): null {
useMountedLogic(scene.logic?.(scene.paramsToProps?.(scene.sceneParams)) || noopLogic)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we get rid of the noopLogic hack by not rendering this component for scenes with no logic?

@mariusandra
Copy link
Collaborator Author

Changes done. The events page was always really slow to me thanks to that huge table. Check out some other pages with huge tables (e.g. feature flags) to see the same sluggishness... :(

Copy link
Collaborator

@Twixes Twixes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I may still be wrong but I don't think it's only as bad as before. Looks like handling the click event on the Events tab button is a worse slowdown than before, and this seems to be the main contributor to slugishness – loading is not as noticeable once the base of the scene is loaded, but if everything tries to render at once ASAP it's worse. Maybe after the restructuring things are now loaded earlier, causing this effect in a more abrupt way.

master turbo-mode
before after

But this definitely does all boil down to the table component (the flame graph points to it too), so I guess we can live with it for a bit. Once the table is optimized, this should be gone.

@mariusandra mariusandra temporarily deployed to posthog-pr-6632 October 26, 2021 19:52 Inactive
@mariusandra
Copy link
Collaborator Author

I guess that's because the table is always empty for the non-turbo case, and renders fast... but now it can be a full 200 row table, causing a lot of sluggishness. I'll document this as a followup in #6513 that we'll need to resolve before we can remove the flag.

@mariusandra mariusandra enabled auto-merge (squash) October 26, 2021 20:01
@mariusandra mariusandra merged commit f76d0b6 into master Oct 26, 2021
@mariusandra mariusandra deleted the turbo-mode branch October 26, 2021 20:08
aviator-app bot pushed a commit to GalDayan/posthog that referenced this pull request Nov 9, 2021
* feat(correlation): add multiselect for property names (#6407)

* refactor(taxonomic-filter): add api mock functions

* chore: remove makeApi from storybook preview.js

* feat(correlation): add multiselect for property names

Previously we had a simple text input box that was splitting the input
to get a list of property names. This doesn't give the greatest user
experience.

Now we have a simple select box that includes:

 1. search filtering for options
 2. multiselecting options
 3. a select $all, although this is pretty poorly implemented at the
    moment, for instance you can select $all, at the same time as
    selecting a subset of properties.

* add a little commentary around PropertyNamesSelect

* add deps to useEffect

* ignore return type eslint issues

* just use console.log instead of actions

* remove unused import

* chore: add error condition

* chore(correlation): update styling of property select

This puts it more in line with
https://www.figma.com/file/gQBj9YnNgD8YW4nBwCVLZf/PostHog-App?node-id=4200%3A30715

* Add no search results message

* input focus border

* fix type

* use onBlur

* make checkbox checked colour change

* make query bold in no results message

* add clear

* Implement search highlighting

* fix typing

* add regex split comment

* if no property names selected, default to $all

* typo

* Add test for selection component

* Add test highlighting onChange lag and clicking outside

* click out side should depend on hide

* make sure onChange triggered on clear or select all

* Use app context in `organizationLogic` too (#6565)

* Use app context in `organizationLogic` too

* Add `organizationLogic.test.ts`

* Org usage report: query for organization owner or first user via OrganizationMembership (#6578)

* query via organizationmembership rather than team__in

* address feedback

* Fix local variable not defined in org_usage_report (#6582)

* Debug queries (#6577)

* Include the insight ID in the url for "saved funnels" (#6531)

* adds a search box to the toolbar featureflag list (#6527)

* adds a search box to the toolbar featureflag list

* test flakes because react is re-rendering between the get and the click. add an assertion to try and slow cypress down to avoid this

* move filtering toolbar feature flags to a Set and out of CSS

* support initKeaTestLogic() with no args

* fix query-selector-all-deep jest bug

* add simple test case for feature flags logic

* combine selectors

* with more understanding of Fuse

* add simple test for flag filtering

Co-authored-by: Marius Andra <marius.andra@gmail.com>

* ci(backend): run backend tests on master (#6579)

At the moment we're just running tests on pull requests. The immediate
reason I'm making this change is to ensure we are producing a code
coverage report for master, which allows us to produce a patch
difference of coverage.

Sidenote: I'm also hoping will also trigger annotations to start being
added to PRs highlighting untested code, which I thought should have
started with https://github.com/PostHog/posthog/pull/6082

* ci(ci-backend): checkout branch when deciding path-filtering (#6585)

Previously we were just running on PRs. In
https://github.com/PostHog/posthog/pull/6579 we also run on master. As a
result we need to checkout the repo as well.

* add properties to path cleanfilters (#6554)

* chore(eslint): explicitly disable exhaustive deps on historical code (#6576)

I haven't made any attept here to resolve anything here, just to
explicitly acknowledge that these are issues that eslint has highlighted
and we should ignore, for the sake of easily picking up new issues. If
these are truely problematic, then I am assuming they should have been
picked up as bugs with the system.

This doesn't stop someone from coming in and reviewing the code in this
pr retrospectively.

* Search toolbar actions (#6587)

* add search to toolbar actions display for #6202

* better markup and spacing

* removes a console.log that snuck in

* Backend for querying persons given an event (#6588)

* Backend for querying persons given an event

* address comments

* add test

* [plugin-server] Refactor Kafka person update message creation into a function (https://github.com/PostHog/plugin-server/pull/607)

* Build FOSS release to Docker Hub (#6584)

* Open modals for everything in correlation events (#6592)

* Open modals for everything in correlation events

* address comment

* Closes 6541 add step to empty funnel (#6556)

* Closes 6541 add step to empty funnel

* added comments back

* hide delete button when single step exists

* use values directly in first step

* Update frontend/src/scenes/insights/ActionFilter/ActionFilterRow/ActionFilterRow.tsx

Co-authored-by: Neil Kakkar <neilkakkar@gmail.com>

* 🤖: Add Jaspreet-singh-1032 as a contributor 🎉 (#6595)

* Sync 1.29.0 (#6593)

* Fix error in recordings table (#6598)

* fix

* Make person header seem more click-able

* clean up

* .github/workflows/docker-vuln-scan.yml run on main/master (#6568)

* Fix broken docs link for secret key (#6596)

* Move `dev.Dockerfile` from slim to alpine (#6524)

* Saved insights filter & url sync (#6388)

* saved insights url sync WIP

* sync filters with the url

* improve trend and date filters

* convert utils test to typescript

* create simple object shallow diff function

* only persist the difference from the default filters in the url

* add tests and simplify operations

* improve tests

* better load checking

* Fix 'Load more People' (#6609)

* Add 'libpq-dev' to production.Dockerfile (#6610)

* you always need DEBUG=1 when running tests (#6613)

* Enable persons modal for properties correlation (#6611)

* Enable persons modal for properties correlation

* adjust header

* fix typings

* Migrate action-related and some insight-related logics to be project-based (#6464)

* Instrument legacy endpoint debugging

* Add `currentTeamId` to `teamLogic`

* Add logics utils

* Update annotations endpoint in the frontend

* Limit legacy endpoint logging to DEBUG

* Fix `annotationsTableLogic` usage

* Update test_annotation.py

* Add `test_deleting_annotation_of_other_team_prevented`

* Comment out debug code

* Migrate actions-related code to project-based approach

* Fix `infiniteTestLogic`

* Rework approach

* Remove redundant lines

* Fix mixup

* Fix non-logged-in scenes

* Fix Python cast

* Align approach to `teamLogic`-based

* Fix logic tests

* Clean up code

* Fix stupid omission

* Restore `props` in `connect`s

* Fix capitalization

* Fix action creation

* Fix `ActionEdit` props type

* Fix duplicate imports

* Update infiniteListLogic.ts

* Update ActionsTable.tsx

* Fix link to new action in Toolbar

* Try localStorage+cookie as persistence type (#6540)

* Path cleaning integration (#6488)

* initial refactoring

* popup UI

* refactor path cleaning logic

* add nullable

* all ui working

* fix migration

* use regex replacement from team object

* add flag

* add switch

* fix type

* fix type

* UI update

* restore removed arg

* add local path cleaning filters to api

* add test for local path filters

* working new UI

* reduced repeated code

* fix numbering

* minor refactoring

* update copy

* add under advanced features

* address comments, minor cleanup

Co-authored-by: Neil Kakkar <neilkakkar@gmail.com>

* Migrate event-related logics to be project-based (#6566)

* Instrument legacy endpoint debugging

* Add `currentTeamId` to `teamLogic`

* Add logics utils

* Update annotations endpoint in the frontend

* Limit legacy endpoint logging to DEBUG

* Fix `annotationsTableLogic` usage

* Update test_annotation.py

* Add `test_deleting_annotation_of_other_team_prevented`

* Comment out debug code

* Migrate actions-related code to project-based approach

* Fix `infiniteTestLogic`

* Rework approach

* Remove redundant lines

* Fix mixup

* Fix non-logged-in scenes

* Fix Python cast

* Align approach to `teamLogic`-based

* Fix logic tests

* Clean up code

* Fix stupid omission

* Restore `props` in `connect`s

* Fix capitalization

* Fix action creation

* Fix `ActionEdit` props type

* Migrate events-related code to project-based approach

* Remove `MOCK_ORGANIZATION_ID`

* Update sessionsPlayLogic.test.ts

* Fix logic tests

* Fix duplicate imports

* Reduce duplication in URL test instrumentation

* Update API interception paths in tests

* Fix `Person.cy-spec.js`

* Add comments in `posthog/api/__init__.py`

* reduce test noise

* Update infiniteListLogic.ts

* Simplify `teamLogic` seeding

* Simplify `teamLogic` seeding better

* Fix `organizationLogic` tests

* Migrate feature flags-related logics to be project-based (#6603)

* Migrate feature flags-related logics to be project-based

* Add comment

* Migrate insight-related logics to be project-based (#6574)

* Migrate insight-related logics to be project-based

* Migrate over the rest and fix logic tests

* Update funnelLogic.ts

* Fix URL formatting in tests

* Update dashboardLogic.tsx

* Remove now redundant `initTeamLogic`

* Add tracking comments

Co-authored-by: Marius Andra <marius.andra@gmail.com>

* Add more info to org usage report schema (#6620)

* cleanup; change event name; add sleep

* add org created_at, user count

* Rename `DashboardItem` model to `Insight` and deprecate `DashboardItem{Serializer,ViewSet}` (#6567)

* Only run `test_migrations_are_null` on new migrations

* Rename `DashboardItem` to `SavedInsight`

* Add explanation to `test_migrations_are_null` script

* Rename `test_dashboard` to `test_saved_insight_model`

* Ditch `DashboardItem{ViewSet,Serializer}` altogether

* Fix refresh via `InsightSerializer`

* "SavedInsight" to just "Insight"

* Update file names

* Update insight.py

* Update dashboardLogic.test.ts

* Update migration

Co-authored-by: Paolo D'Amico <paolodamico@users.noreply.github.com>

* Minor fixes to `dev.Dockerfile` (#6615)

* Tweak lifecycle tooltip copy (#6404)

* Update InsightsNav.tsx

Tweaked lifecycle tooltip copy

* Update InsightsNav.tsx

Pushed last two words to new line to fix prettier issue.

* prettier format InsightsNav.tsx

Co-authored-by: Harry Waye <harry@microwayes.net>

* Group analytics: Initial schema (#6462)

* Add table for group_type_mapping

* Remove materialized columns from events table schema

These are not used and not needed w/ new mat columns work

* WIP: Migration to add group analytics columns

* Remove event table changes temporarily

* Backend support for autocapture elements chain in Correlations (#6627)

* Backend support for autocapture elements chain properties

* address some comments

* Fix `teamId` usage in `pluginsLogsLogic` (#6634)

* Migrate dashboard-related logics to be project-based (#6623)

* Migrate dashboard-related logics to be project-based

* Update test_personal_api_keys.py

* Update dashboard.py

* Update test_dashboard.py

* Make mypy happy

* Don't return/show forbidden projects at all (#6622)

* Don't return/show forbidden projects at all

* Fix missing `lookup_field`

* Remove papercups widget (#6415)

* Remove papercups widget

* remove unused import

* helper button in the nav bar

* post-merge fix

* extra fixes & improvements

Co-authored-by: Paolo D'Amico <paolodamico@users.noreply.github.com>

* load public jobs from file (#6637)

* Enable support for Autocapture events in correlations - UI (#6635)

* log errors from kafka producer (#6631)

* [plugin-server] Remove unused person creation path (https://github.com/PostHog/plugin-server/pull/608)

* Better seekbar for session recordings (#6529)

* rename sessionRecording to session-recording

* logic for new player

* move rrweb-player into repo

* remove awkward wrapper and fix time not updating bug

* chunk load meta timing

* fix duration setting bug

* update icons and styling for controller

* styling icons

* slider styling

* ugly slider

* remove consoles

* sonarqube

* typescript

* typescript

* implement working seekbar

* fix time mismatches

* remove hardcode

* Fix icon from merge

* move report usage back to sessionsplaylogic

* refactor to seekbar

* implement real time seek time

* play on paused fixed

* bug with play pause

* fix buffer play state bug

* add comments back

* remove consoles and revert limit

* remove empty file

* address feedback

* fix scrubbing issue and onclick lag

* remove console

Co-authored-by: Rick Marron <rcmarron@gmail.com>

* Fix locking migration (#6640)

* Paginate recording compression (#6563)

* paginate recording compression

* some tests

* more accurate duration calculation

* add tests and types

* tons of decompression fixes

* rename test file to avoid conflict

* move decompression to helper

* add test for helper

* type fix

* rename method

* simplify paginated decomression

* handle case where offset exceeds length

* clean up

* test fixes

* clean up on aisle 12

* Add surrounding object for metadata response

* Remove DO image generator (#6594)

* GH actions for FOSS (#6645)

* Release 1.29.1 (#6644)

* Add retries to kafka producer to mitigate event loss (#6638)

* Fix before url in events api (#6639)

* Fix before url in events api

* Fix clickhosue

* [plugin-server] Add simple ingestion support for groups (https://github.com/PostHog/plugin-server/pull/601)

* Add support for ingesting groups

* $group_set, re-add tests

* Shuffle properties around a bit

* Add a new group

* Test for race condition in group type ingestion

* Add additional await

* [plugin-server] Bump version to 1.10.3

* Update plugin server to 1.10.3 (#6652)

* `.github/workflows/docker-vuln-scan.yml`: don't push the image (#6625)

* Fix clicking in command palette (#6653)

* [plugin-server] silence sentry errors (https://github.com/PostHog/plugin-server/pull/609)

* [plugin-server] Bump version to 1.10.4

* Update plugin server to 1.10.4 (#6655)

* Fix 1.29.0 more info link (#6650)

* 🤖: Add inbreaks as a contributor 🎉 (#6656)

* Update paths dashboard items (#6459)

* coloring updates

* set a min width for paths canvas

* account for steps less than 3

* Enable exclusions on all kinds of correlations (#6654)

* Enable exclusions on all kinds of correlations

* rm action

* address comment

* [plugin-server] testing sentry releases (https://github.com/PostHog/plugin-server/pull/614)

* testing sentry releases

* format readme

* [plugin-server] Bump version to 1.10.5

* Update plugin server to 1.10.5 (#6664)

Co-authored-by: posthog-bot <posthog-bot@users.noreply.github.com>

* Instrumentation for feedback with emojis (#6484)

* WIP: Instrumentation for feedback

* update feedback form & cleanup

* use const

* Add new `api` composition approach (#6612)

* Instrument legacy endpoint debugging

* Add `currentTeamId` to `teamLogic`

* Add logics utils

* Update annotations endpoint in the frontend

* Limit legacy endpoint logging to DEBUG

* Fix `annotationsTableLogic` usage

* Update test_annotation.py

* Add `test_deleting_annotation_of_other_team_prevented`

* Comment out debug code

* Migrate actions-related code to project-based approach

* Fix `infiniteTestLogic`

* Rework approach

* Remove redundant lines

* Fix mixup

* Fix non-logged-in scenes

* Fix Python cast

* Align approach to `teamLogic`-based

* Fix logic tests

* Clean up code

* Fix stupid omission

* Restore `props` in `connect`s

* Fix capitalization

* Fix action creation

* Fix `ActionEdit` props type

* Fix duplicate imports

* Update infiniteListLogic.ts

* Update ActionsTable.tsx

* Fix link to new action in Toolbar

* Add new `api` composition approach

* Use feedback

* Add premade compositions

* Fix `createActionFromEvent` tests

* Fix code structure

* Fix typing

* Update api.ts

* Add event with properties benchmark (#6666)

* 6331 pie chart persons bug (#6642)

* backend fixes and test

* add breakdown value to pie chart

* adjust test

* fix faulty test

* fill param

* fix formula tests

* more date passing

* more cleanup

* all tests working

* make test data explicit and add better checks

* support both ee and postgres

* length checks

* Turbo mode (#6632)

* fix router redirect

* remove dependence on user var

* split scenes and sceneTypes out of sceneLogic

* rename LoadedScene type to SceneExport

* export SceneExport from most scenes

* use exported scene objects, warn if not exported

* fix type import bugs

* remove dashboard

* keep all loaded scene logics in memory

* fix sorting bugs

* support scenes with params, make it work with dashboards

* fetch result from dashboard if mounted

* fix mutations

* add lastTouch

* refactor scene parameters to include searchParams and hashParams

* add insights scene to scene router

* add insight router scene to scene router

* fix cohort back/forward bug

* this works

* bring back delayed loading of scenes

* set insight from dashboard also in afterMount

* split events, actions, event stats and properties stats into their own scenes

* refactor to options object for setInsight

* override filters

* clean filters for comparison

* fix cohort bug

* get a better feature flag

* make turbo mode faster by making non-turbo-mode slower

* less noise in failed tests

* fix tests

* flyby: add jest tests pycharm launcher

* clean up scenes

* add test for loading from dashboardLogic

* fix bug

* split test init code into two

* have the same data in the context and in the api

* add basic tests for sceneLogic

* run the latest and greatest

* fix menu highlight

* implement screaming snake

* only show scenes with logics

* Clarify unique users vs. total count (#6648)

* UX revamp for split/merge persons (#6544)

* Saved insight fixes 1 (#6661)

* remove "view" button on dashboard

* prevent insight card views from opening in a new window

* create a storybook for all icons

* add funnel icons and improve new insight menu

* remove debug

* add insights sessions icon

* link without a link is a button

* fix icons in select box

* Move return out of finally block to fix sonarcloud (#6671)

* Move return out of finally block to fix sonarcloud

* fix type checking skips

* Fix whitescreen on PersonsModal: add nullish coercion for empty string

* Fix missing space in funnel persons modal (#6682)

* Remove unused imports from tests (#6678)

* Fresh top bar base (#6657)

* Add base header with logo, search, account icon, and announcement

* Rearrange code, add logic, and add announcement X button

* Add help button to top bar and create a story

* Make the top bar logo a link to posthog.com

* Only display help button caret in new top bar

* Fix help button position a bit

* Fix `Cohort` component importing raw AntD CSS (#6685)

* dev.Dockerfile COPY fix (#6686)

* Allow wildcard app urls (#6660)

* Allow wildcard app urls

* typing

* Convert to logic and write tests (#6689)

* [plugin-server] add benchmark (https://github.com/PostHog/plugin-server/pull/613)

* UX improvements to insights (#6675)

* Delete app.json (#6617)

* update app.json

* delete app.json

* Revert "Update the docker-compose dev environment to reflect prod better (include pgbouncer) (#6558)" (#6692)

* feat: remove exclude properties select box from funnel correlation, add exclude property button (#6694)

* chore: fix typo in usePersonProperties

* refactor(property-select): introduce useSelectedPropertiesContext

This is largely just me copying from
https://react-hook-form.com/api/useformcontext but the intent is to
remove the requirement for introducing a Provider in simple cases.

I could I suppose use react-hook-form for some of this, but I'm using it
as a little bit of a learning exercise as well.

* feat(correlation): add exclude property button next to correlations

This is quick and dirty and still uses the include logic. I'll follow up
with something to make this into a specific exclude of properties, and
make it sticky.

* chore(correlation): switch to exclude for property correlation

* fix lint

* Fix serialization to localstorage

* once more

* refactor(correlation): update popup to use kea logic
This commit creates a new kea logic that is intended to be the logic for
the PropertyNamesSelect component. This commit however only implements
the popup aspect of the component as a logic, with the intention that
the search implementation and selection logic will be handled in a
follow-up commit.

Note that this implementation doesn't actually currently work and I'm
still debugging what is wrong.

* chore(correlation): remove manual typing from kea logic

* refactor(property-selection): use concise action definitions

This makes the logic more consistent with others in the codebase. As
mentioned in the docs:

https://keajs.org/docs/guide/concepts#actions

* refactor(property-select): move ref prop reference outside of logic

The logic shouldn't be referencing anything relevant to `React` (or
anything else specific to a particular view implementation)

* chore(property-select): fix typing issues

* chore(property-select): make reducers a little neater

* refactor(property-select): move logic to separate file

This is to highlight that we should not have any React dependencies in
the logic. We still encapsulate the React specifics as the `usePopup`
hook.

* chore(property-select): make listeners functional

* wip: move selection into property names select logic as well

* refactor(property-select): complete select logic migration

This gets the tests passing again with logics!

* refactor(property-select): remove usePersonProperties

* fix typing

* remove unused import

* refactor: remove custom hooks and context

* wip

* add excludeProperty action, isSelect value

* chore: get storybook funnel working again

* refactor: move property search into logic

* refactor: rename popup actions/values/selectors/listeners

* chore: fix typing

* refactor: use kea-localstorage to persist excluded property names

* rename isSelected funnel logic selector to isPropertyExcluded

* chore: simplify storybook for `PropertyNamesSelect`

* avoid specifying key on propertynamesselect

* ensure excludeProperty triggers refresh

* button with a B

* use values in loadpropertycorrelations

* remove localstorage, fix action call on refresh

* chore: rename PropertyNamesSelectLogic to propertyNamesSelectLogic

* test(correlations): add tests for property exclusions

* chore: fix load correlations on loadResultsSuccess

* add initial select logic tests

* rename logic filename to lower

* feat: remove exclude properties select box

I'm removing this as we have the "Exclude Property" button which should
give us what we want for now wrt excluding properties to reveal more
relevant correlations past the first few ones.

* restrict changes to just the Exclude property button

Co-authored-by: Neil Kakkar <neilkakkar@gmail.com>

* feat(correlation): ability to exclude property correlations for events (#6665)

* Move toast to bottom of screen (#6687)

* Duplicate filters on insights (#6677)

* Don't send org reports if settings.TEST or DEBUG (#6679)

* don't send org report if settings.TEST

* also don't send if DEBUG

* Take posthog-js from the props to the toolbar (#6699)

* take posthog-js from the props to the toolbar

* fix test

* fix typing

Co-authored-by: Rick Marron <rcmarron@gmail.com>

* quick fix (#6703)

* Fix `addFilter` edge case (#6702)

* Update posthog-js to 1.15.4 (#6700)

Co-authored-by: posthog-bot <posthog-bot@users.noreply.github.com>

* 🤖: Add xrendan as a contributor 🎉 (#6707)

* Next url dates broken on `api/events` endpoint + missing order_by on CH (#6681)

* fix next url

* remove unused import

* make faster test

* remove irrelevant file

* remove unnecessary order param

* chore(correlation): switch to exclude for property correlation (#6597)

* chore: fix typo in usePersonProperties

* refactor(property-select): introduce useSelectedPropertiesContext

This is largely just me copying from
https://react-hook-form.com/api/useformcontext but the intent is to
remove the requirement for introducing a Provider in simple cases.

I could I suppose use react-hook-form for some of this, but I'm using it
as a little bit of a learning exercise as well.

* feat(correlation): add exclude property button next to correlations

This is quick and dirty and still uses the include logic. I'll follow up
with something to make this into a specific exclude of properties, and
make it sticky.

* chore(correlation): switch to exclude for property correlation

* fix lint

* Fix serialization to localstorage

* once more

* refactor(correlation): update popup to use kea logic
This commit creates a new kea logic that is intended to be the logic for
the PropertyNamesSelect component. This commit however only implements
the popup aspect of the component as a logic, with the intention that
the search implementation and selection logic will be handled in a
follow-up commit.

Note that this implementation doesn't actually currently work and I'm
still debugging what is wrong.

* chore(correlation): remove manual typing from kea logic

* refactor(property-selection): use concise action definitions

This makes the logic more consistent with others in the codebase. As
mentioned in the docs:

https://keajs.org/docs/guide/concepts#actions

* refactor(property-select): move ref prop reference outside of logic

The logic shouldn't be referencing anything relevant to `React` (or
anything else specific to a particular view implementation)

* chore(property-select): fix typing issues

* chore(property-select): make reducers a little neater

* refactor(property-select): move logic to separate file

This is to highlight that we should not have any React dependencies in
the logic. We still encapsulate the React specifics as the `usePopup`
hook.

* chore(property-select): make listeners functional

* wip: move selection into property names select logic as well

* refactor(property-select): complete select logic migration

This gets the tests passing again with logics!

* refactor(property-select): remove usePersonProperties

* fix typing

* remove unused import

* refactor: remove custom hooks and context

* wip

* add excludeProperty action, isSelect value

* chore: get storybook funnel working again

* refactor: move property search into logic

* refactor: rename popup actions/values/selectors/listeners

* chore: fix typing

* refactor: use kea-localstorage to persist excluded property names

* rename isSelected funnel logic selector to isPropertyExcluded

* chore: simplify storybook for `PropertyNamesSelect`

* avoid specifying key on propertynamesselect

* ensure excludeProperty triggers refresh

* button with a B

* use values in loadpropertycorrelations

* remove localstorage, fix action call on refresh

* chore: rename PropertyNamesSelectLogic to propertyNamesSelectLogic

* test(correlations): add tests for property exclusions

* chore: fix load correlations on loadResultsSuccess

* add initial select logic tests

* rename logic filename to lower

* feat: remove exclude properties select box

I'm removing this as we have the "Exclude Property" button which should
give us what we want for now wrt excluding properties to reveal more
relevant correlations past the first few ones.

* restrict changes to just the Exclude property button

* Revert "restrict changes to just the Exclude property button"

This reverts commit cd356744896b3c935a59a2719f4a395757770b47.

* Revert "feat: remove exclude properties select box"

This reverts commit 5b2e6202fb17a02bfcd8cdfe8b0d09f7e68e155f.

Co-authored-by: Neil Kakkar <neilkakkar@gmail.com>

* Refactor taxonomy selector/property naming (#6709)

* Rename groups => taxonomicGroups

Avoids confusion with group analytics down the line

* Rename groupTypes => taxonomicGroupTypes

This avoids confusion with groups

* Persist view/edit for insight in URL + refactor insightLogic tests (#6708)

* refactor tests

* persist edit=true in edit mode

* Improve correlation analysis feedback (#6705)

* improve the UI of the box

* fix tests

Co-authored-by: Neil Kakkar <neilkakkar@gmail.com>

* Update posthog-js to 1.16.0 (#6713)

* Migrate cohort-related logics to be project-based (#6683)

* Migrate cohort-related logics to be project-based

* Fix `cohortsDetail`

* Remove `teamId` param from `determineEndpoint` methods

* Modify query to not use properties for autocapture (#6711)

* modify query to not use properties for autocapture

* address comments

* add benchmark test to check if worth materialising

* Logging: change default formatting (#6697)

* [plugin-server] Event to have original passed set and set-once properties (https://github.com/PostHog/plugin-server/pull/615)

* add version to person (#6628)

* add version to person

* add migration

* set all initial version values to 0

* remove defaults

* fix dep

* Make person update function migrations reversible (#6629)

* make person update func migrations reversible

* Update posthog/migrations/0176_update_person_props_function.py

Co-authored-by: Tiina Turban <tiina303@gmail.com>

* Update posthog/migrations/0176_update_person_props_function.py

Co-authored-by: Tiina Turban <tiina303@gmail.com>

Co-authored-by: Tiina Turban <tiina303@gmail.com>

* Saved insight fixes 2 (#6674)

* insights buttons padding right fix

* add icon column

* fix all types

* fix select icons

* improve display and empty states

* searching for a number gives a direct result

* more debouncing time to throttle typing in the searchfield

* 2 columns > 960px, 3 columns > 1580px

* more height for card view

* split options column

* add person icon

* added profile picture

* fix column sort

* add min-width to funnels to make them readable on tiny dashboards

* add "trends" as default type

* stop screaming

* fix api call

* Polish Correlation Analysis UI - Part 1 (#6716)

* things finally look amahzing

* update tests

* [plugin-server] update Readme to reflect archive and monorepo location

* Use separated session recording endpoints (#6641)

* remove heigh check (#6718)

* make sure edge filters are used (#6723)

* Revert "Monorepo with updated history"

* Remove deprecation warning

* 🤖: Add Nishant-Sagar as a contributor 🎉 (#6736)

* Fix running `yarn` in plugin-server/ directory (#6738)

This broke with the following error:

```
ESLint: 7.28.0

ESLint couldn't determine the plugin "@typescript-eslint" uniquely.

- /home/macobo/backups/2/projects/posthog/plugin-server/node_modules/@typescript-eslint/eslint-plugin/dist/index.js (loaded in "../../../.eslintrc.js")
- /home/macobo/backups/2/projects/posthog/node_modules/@typescript-eslint/eslint-plugin/dist/index.js (loaded in "../../../../.eslintrc.js")

Please remove the "plugins" setting from either config or remove either plugin installation.

If you still can't figure out the problem, please stop by https://eslint.org/chat/help to chat with the team.
```

Relate github issue: https://github.com/eslint/eslint/issues/13385

* update pycharm plugin server run configurations (#6742)

* Remove `.github/workflows/docker-vuln-scan.yml` (#6747)

* Add site popover on profile picture to fresh top bar (#6701)

* Keep top bar always visible

* Add `LemonPopover`

* Add empty account control sections

* Rename `AccountControl` to `SitePopover`

* Add basic Sign out button

* Add `LemonButton` and use it for sign out

* Scale down popover scaling

* Remove `LemonPopover` in favor of `Popup`

* Improve icons story

* Polish account info

* Add organizations to site popover

* Add `CreateOrganizationModal` to  `TopBar`

* Polish things

* Add PostHog status items

* Update FunnelTab.tsx

* Rename `InitialBlob` to `Lettermark`

* Update utils.test.ts

* Address feedback

* Handle long emails

* Update Cypress test for sentence case title

* Fix "Manage account" positioning (#6750)

* Instance preferences - disable feature upselling stub (#6706)

* fix session recording permission bug (#6751)

* Don't rely on cache to return result (#6752)

* Split out v1 and v2 of sessionsPlayLogic to fix bugs in prod (#6727)

Co-authored-by: Rick Marron <rcmarron@gmail.com>

* 🤖: Add romj as a contributor 🎉 (#6760)

* Proposal: Use unique topic names for kafka in test (#6746)

Without this, to run plugin-server tests you need to reset all
containers every time since otherwise both test- and non-test clickhouse
would attempt to read from the same topic.

* Some more polishing & restore property selector semantics (#6748)

* restore property selector semantics

* fix tests

* add another test

* Only load correlations for Funnels (#6765)

* Only load correlations for Funnels

* CI: Run EE tests in 5 minutes (#6759)

* Unify SAML and non-saml tests workflows

* Include matrix name in cache key

* Prettify yaml

* Measure test duration time

* Install pytest-split

* Try out parallelism

* Update name

* Fixup yaml

* Upload artifacts

* Run FOSS tests separately

* Run mark.ee tests only once

* Set up needed files properly

* Run backend tests only on one python version

* Run only cloud tests for cloud repo

* Remove test_durations file for posthog/

* Skip setting up ch on foss tests

* Fixup cloud tests

* Cache .test_durations file

* Fix ternary expression

* Only upload artefacts if needed

* .test_durations change

* Default to 3.8.5 in tests

* Add a note

* make migration 0174 reversible (#6724)

* make 0174 reverisble

* remove line altogether

* Remove unused import in test_preflight.py (#6647)

Co-authored-by: Paolo D'Amico <paolodamico@users.noreply.github.com>

* Disable Storybook previews on forks (#6777)

* Fixes for Correlation UI: pagination & persons modal (#6767)

* Fixes for Correlation  UI: pagination & persons modal

* Unify new correlation icons (#6773)

* feat(correlation): load exclude properties from team settings (#6715)

* feat(correlation): add team wide person property exclusion list

This change just adds the exclusion list to the `Team` model and checks
that it can be updated. Separately we can:

 1. add interface for the Project page to display
 2. pull and update this list from the funnels correlation page

NOTE: We don't perform any validation on the structure, :fingerscrossed:
this won't be an issue. Alternative would be to use ArrayField, but use
of JSONField is consistent with other fields.

* test(correlation): add test checking exclude properties pulled from team

* feat(correlation): load exclude properties from team settings

Previously we were loading exclude properties from local storage, so we
would not be sharing the exclusion list between users. This change
simply plugs the `excludePropertyNames` into the `teamLogic` for the
purpose of persisting and loading the values.

* use [teamLogic.actionTypes.loadCurrentTeamSuccess]

* get tests passing

* switch frontend to using `correlation_config`

* refactor: use Team.correlation_config for persistence

* add project settings for correlation

* fix lint

* ensure excluded properties are saved to project config

* Add default excluded properties

* format

* make propertyCorrelations not null

* ensure excluded property names config is unique

* rename excludeProperty to excludePropertyFromProject

* update var names

* change to targetProperties

* remove null special casing

* update to filter client side on exclude from project clicked

* update test name to reflect new functionality

* fix tests

* Add storybook support for updating correlation config (#6785)

* 🤖: Add pixlwave as a contributor 🎉 (#6786)

* perf improvement on person query in recordings (#6754)

* Update plugin server deps (Dependabot alerts) (#6774)

* change url access to https (#6779)

* Grammar nit: write out number, hyphenate for consistency (#6418)

* Bump browserslist from 4.16.0 to 4.17.5 in /plugin-server (#6790)

Bumps [browserslist](https://github.com/browserslist/browserslist) from 4.16.0 to 4.17.5.
- [Release notes](https://github.com/browserslist/browserslist/releases)
- [Changelog](https://github.com/browserslist/browserslist/blob/main/CHANGELOG.md)
- [Commits](https://github.com/browserslist/browserslist/compare/4.16.0...4.17.5)

---
updated-dependencies:
- dependency-name: browserslist
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump hosted-git-info from 2.8.8 to 2.8.9 in /plugin-server (#6791)

Bumps [hosted-git-info](https://github.com/npm/hosted-git-info) from 2.8.8 to 2.8.9.
- [Release notes](https://github.com/npm/hosted-git-info/releases)
- [Changelog](https://github.com/npm/hosted-git-info/blob/v2.8.9/CHANGELOG.md)
- [Commits](https://github.com/npm/hosted-git-info/compare/v2.8.8...v2.8.9)

---
updated-dependencies:
- dependency-name: hosted-git-info
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Paolo D'Amico <paolodamico@users.noreply.github.com>

* Show org access level in site popover (#6763)

* Show org access level in site popover

* Simplify access level indicator code and shorten "administrator"

* Update posthog-js to 1.16.1 (#6787)

Co-authored-by: posthog-bot <posthog-bot@users.noreply.github.com>

* Add top level danger `TopBar` indicator for potential system issues (#6789)

* Add top level danger `TopBar` indicator for potential system issues

* Update frontend/src/layout/lemonade/TopBar/SitePopover.tsx

Co-authored-by: Paolo D'Amico <paolodamico@users.noreply.github.com>

* Run prettier

Co-authored-by: Paolo D'Amico <paolodamico@users.noreply.github.com>

* Fix test depending on `OrganizationBasicSerializer` (#6795)

* Update test_user.py

* Account for "administrator" being shortened to "admin"

* Describe `AnalyticsDestroyModelMixin` (#6803)

* Bundle our frontend via esbuild (#6758)

* esbuild package

* almost get esbuild working

* fix react-virtualized imports

* add splitting

* fix funny import reorder bug

* fix squeakAudio referring to itself

* write index.html file

* fix some bad imports

* update antd paths

* remove raw-loader usage, it didn't work anyway

* refactor and copy public

* build app and toolbar

* get toolbar working, but without styles

* make toolbar and its styles work

* shared dashboards

* clean frontend build before rebuilding

* add watch mode

* reorder tasks

* revert js url

* incremental builds of app with debounced chokidar watching

* common build/watch script

* improve logs

* watch during firrst build

* fix toolbar url

* fix wrongly exported scene

* create sceneProxyLogic to untangle sceneLogic from all bundles

* disconnect sceneLogic and refactor setPageTitle

* live reloading server

* rename utils file

* only wait for /static

* fix encoding

* simplify

* add missing dayjs plugins

* fix pathless logics

* simplify options

* add jsx for webapck

* slight delay to catch changes

* a type is a type

* fix build

* esbuild in start

* funnelLogic path

* include all files with a "." (so .mjs, etc) in /frontend/ to docker

* rename to "utils.mjs", make "build.mjs" executable

* improve erroring

* revert some needless changes

* more reverts

* change some scripts

* remove setuff

* clarify function

* make "--host 0.0.0.0" work

* fix import order issue in webpack

* remove webpack css inlining for toolbar to simplify config

* make toolbar with external styles work in storybook

* move live server injection into django

* fix undefined bug

* simplify setup to work with injection directly in http://localhost:8000 (no proxying needed on :8234)

* add comments

* Fix `fse` usage

I was getting this otherwise:

$ node frontend/build.mjs
file:///Users/twixes/Developer/posthog/frontend/utils.mjs:46
    fse.copySync(srcDir, destDir, { overwrite: true }, function (err) {
        ^

TypeError: fse.copySync is not a function
    at copyPublicFolder (file:///Users/twixes/Developer/posthog/frontend/utils.mjs:46:9)
    at file:///Users/twixes/Developer/posthog/frontend/build.mjs:5:1
    at ModuleJob.run (internal/modules/esm/module_job.js:146:23)
    at async Loader.import (internal/modules/esm/loader.js:165:24)
    at async Object.loadESM (internal/process/esm_loader.js:68:5)

* Mock `process` for VFile used by ReactMarkdown

I was getting this otherwise:

core.js:55 Uncaught ReferenceError: process is not defined
    at new VFile (core.js:55)
    at VFile (core.js:49)
    at Function.parse (index.js:273)
    at ReactMarkdown2 (react-markdown.js:42)
    at renderWithHooks (react-dom.development.js:14803)
    at mountIndeterminateComponent (react-dom.development.js:17482)
    at beginWork (react-dom.development.js:18596)
    at HTMLUnknownElement.callCallback2 (react-dom.development.js:188)
    at Object.invokeGuardedCallbackDev (react-dom.development.js:237)
    at invokeGuardedCallback (react-dom.development.js:292)

* Mock `process.env` for VFile used by ReactMarkdown

I was getting this otherwise:

platform.ts:73 Uncaught TypeError: Cannot read properties of undefined (reading 'ENABLE_VSCODE_BROWSER_CODE_LOADING')
    at platform.ts:73
    at platform.ts:79
    at Function.r._invokeFactory (loader.js:1118)
    at r.complete (loader.js:1128)
    at r._onModuleComplete (loader.js:1754)
    at r._resolve (loader.js:1714)
    at r.defineModule (loader.js:1357)
    at _ (loader.js:1804)
    at numbers.ts:10
    at fake:1

* pass the heavy appScenes to sceneLogic through props via App.tsx

* remove sceneProxyLogic

* remove exported variables

* fix sceneLogic test

Co-authored-by: Michael Matloka <dev@twixes.com>

* Use group analytics in some event capture (#6745)

* Update posthog-analytics to 1.4.3

* Integrate group analytics into the frontend

* Use groups for instance status reports

* Remove domain_whitelist

* Solve weird CI cache issue

* Make UTM tag property descriptions more universal (#6805)

* Update package.json (#6741)

I've been running a custom version of package.json since forever.

Here's my suggestion for cleaning it up

* Fresh layout sidebar (#6768)

* Add new sidebar base with project indicator

* Point TopBar logo to app default path

* Fix sidebar sizing

* Make sidebar toggleable

* Constrain Popup max dimensions for mobile

* Fix `.main-app-content` width

* Put icons story at top level of Storybook

* Add pages to SideBar

* Make LemonRow transitions smoother

* Help popup refresh (#6769)

* Add sections

* Add `SideBar` to Storybook

* Rename files for clarity

* Use sidebar spacers instead of titled sections

* Use `<strong>` rather than `<b>`

* Less cleaning for the build (#6812)

* less cleaning for the build

* push layout.html

* copy the right file, fix build script

* add build timestamp (#6813)

* Disable Cypress parallelization on forks (#6792)

* Saved insight fixes 3 (#6719)

* remove unused action

* remove "add to dashboard" for now until we have support for multiple

* add saved filters to insight logic

* show discard button when something to discard

* show better datetime in table

* simpler "save to dashboard" box

* editable textarea

* enable date picker on view mode

* save tags and remove insight metadata components

* remove duplicate name

* fetch insights from saved insights logic as well

* fix props

* close modal after insight updates, show loading until then

* don't override loaded results if anyway fetching them

* show "discard" on all pages

* revert regression

* fix layout issues

* max-width for edit mode

* improve button

* fix api call

* close new tag input on blur

* remove "Created by" on "your" tab

* remove useless tooltip

* button on dashboard

* add "... changes" to "Discard..."

Co-authored-by: Michael Matloka <dev@twixes.com>

* Fix new chunk loading error with esbuild, add reload guard (#6810)

* fix new chunk loading error

* show error if reloading a second time in 20 seconds

* Make correlation config select styling like other configs (#6794)

* Make correlation config select styling like other configs

This just adds the ability to select person properties to be excluded,
as before but this time with the blue pill styling as with the others on
the Project settings page.

* chore: move select component to components dir

* chore: add storybook for PersonPropertySelect

* Show latest version if update available in `SitePopover` (#6814)

* Fetch session recording events (#6601)

* rename sessionRecording to session-recording

* logic for new player

* move rrweb-player into repo

* remove awkward wrapper and fix time not updating bug

* chunk load meta timing

* fix duration setting bug

* update icons and styling for controller

* styling icons

* slider styling

* ugly slider

* remove consoles

* sonarqube

* typescript

* typescript

* implement working seekbar

* fix time mismatches

* remove hardcode

* Fix icon from merge

* move report usage back to sessionsplaylogic

* refactor to seekbar

* implement real time seek time

* play on paused fixed

* paginate recording compression

* some tests

* bug with play pause

* more accurate duration calculation

* implement event fetching from frontend

* add tests and types

* fix buffer play state bug

* add comments back

* remove consoles and revert limit

* remove empty file

* tons of decompression fixes

* rename test file to avoid conflict

* move decompression to helper

* add test for helper

* type fix

* rename method

* simplify paginated decomression

* address feedback

* handle case where offset exceeds length

* clean up

* test fixes

* fix scrubbing issue and onclick lag

* remove console

* clean up on aisle 12

* Add surrounding object for metadata response

* split out endpoint consumption

* fix cypress tests

* use kea-window-values plugin and remove isTouchDevice check

* remove todo

* report usage refactor

* set buffering on last time catch up

* clamp buffer ms

* add logic tests

* fix eventusage

* finish up tests

* remove console

* update duration on first pain

* split out v1 and v2 sessionsPlayLogic and cleanup logic

* revert ts

* remove defaults from tests

* Move mocks into session-recording folder

* fix test

* rename to sessionRecordingLogic

* kea

* rename to sessionRecordingLogic

* cherry pick rename

* update yarn.lock

* prettier

* revert lockfiles

* order events ascending

* fix time buffer

Co-authored-by: Rick Marron <rcmarron@gmail.com>

* Add mobile mode to the new sidebar (#6819)

* Use span for person property select in project settings page (#6816)

* Use span for person property select in project settings page

Previously they were antd Rows, but this means it's a really tall list,
especially with the defaults.

* add a list of default selection, so can test wrapping behaviour

* fix styling

* remove unsed import

* Remove unused import

* Fix storybook title

* update styles in line with figma

From https://www.figma.com/file/gQBj9YnNgD8YW4nBwCVLZf/PostHog-App?node-id=4325%3A30315

* further figma updates

* remove unused import

* Track all actions happening with correlations (#6788)

Co-authored-by: Paolo D'Amico <paolodamico@users.noreply.github.com>

* Wire up `ToolbarModal` in `SideBar` (#6827)

* Wire up `ToolbarModal` in `SideBar`

* Clean styling up a bit

* Introduce pay gate for correlation analysis (#6720)

Co-authored-by: Marcus Hyett (PostHog) <85295485+marcushyett-ph@users.noreply.github.com>
Co-authored-by: Paolo D'Amico <paolodamico@users.noreply.github.com>

* Update dev start commands for ESBuild (#6829)

* Add "New insight" button to new sidebar (#6826)

* Add "New insight" button to new sidebar

* Highlight sidebar item parent if side action active

* Simplify `LemonButton` link handling

* BE: Allow filtering by group properties in trends (#6761)

* Add group type, group_type_index

* Raise an error when handling unsupported properties in CH

* Improve repr

* Fix is_superset function

This was previously broken - sorting and zipping doesn't really work for
this intent.

* Add group_type_index to analysis results

* Add `group_types_to_query`

* Minor typing fixes

* Create groups tables in tests

* Simple first filter by groups query

* isort

* Use snapshot testing in event_query tests, add test for groups

* `LemonRow` polishes

* Revert "dev(react): add react hook eslint rules, as warnings" (#6831)

* Revert "dev(react): add react hook eslint rules, as warnings (#6571)"

This reverts commit f06e0b4c51f3dbb5a300fe294afba72542cda12f.

* revert all eslint comments

* Un-authorise the toolbar when we can't load feature flags (#6832)

* use toolbarFetch

* clear toolbar session if feature flags are unauthenticated

* Remove "posthog-toolbar-feature-flags" flag (#6833)

* remove "posthog-toolbar-feature-flags" flag

* add instrumentation

* fix plugin-server db test (#6781)

* fetchPerson to be able to use in a transaction (#6780)

* fetchPerson to be able to use in a transaction

* remove fetch overloads

* Scroll to top when clicking links (#6822)

* scroll to top when clicking links

* add comment

* also scroll to top if not loaded before

* give 50ms to the logic to load before showing the scene

* fix bug

* handle dates better in plugin server (#6749)

* handle dates better in plugin server

* move to utils

* Delete reload.txt

* Fix cohort creation from trends (#6636)

* Fix cohort creation from trends

* REVERT debug issue

* fix nit

* Revert "REVERT debug issue"

This reverts commit ea50db28cecf1af89d951a713fe1f5f9a7a83443.

* REVERT try debugging issue

* REVERT Try final

* Don't insert into clickhouse twice

* remove unused var

* debug

* fix and revert

* test mixin

* fix black

* fix

* Update ee/tasks/test/test_calculate_cohort.py

Co-authored-by: Michael Matloka <dev@twixes.com>

Co-authored-by: Michael Matloka <dev@twixes.com>

* Make the API safe to introduce multi-property breakdown (#6757)

* adds a test to prove that without any changes the API receives a string and returns one see #938

* put a guard in place so that funnel query can be shifted to arrays without affecting existing clients

* make checking for single property breakdowns safer

* sort imports

* sort imports

* reformat files

* Revert "reformat files"

This reverts commit 58530f134f191d53b927abe6fb6f5d2d384cb255.

* expand short variable names

* corrects a typo

* it wasn't a typo it was a logic error

* Adds basic functionality to filter plugin logs by log type (#6778)

* adds basic functionality to filter plugin logs by log type

* don't use default value with checkbox

* addresses review comments (or tries to)

* more space around checkboxes in the pluginlogs screen

* use the toparams from utils to encode the pluginlogs search parameters

* more clarity in query param parsing

* Show spinner if the scene's JS takes more than a second to import (#6898)

* show spinner if the scene takes more than a second to import

* down to 500

* fix funnelLogic key (#6900)

* [funnel ui] make sure interval state updates (#6717)

* make sure state changes

* change reducer to selector

* use existing debounce dependency to debounce the local state change of conversion window

Co-authored-by: Paul D'Ambra <paul.dambra@gmail.com>

* fix: Aggregate viz all time (#6824)

* populate 'all' when there's no dates

* populate 'all' when there's no dates

* [proposal] suggestion for reducing runtimes (#6725)

* suggestion for reducing runtimes

* refactor tests ingestion

* django tests

* fix types

* add comment

* Cleanup unused python  imports (#6695)

* Cleanup unused imports

* run black

Co-authored-by: Michael Matloka <dev@twixes.com>

* Fix pre-commit hook (#6902)

* pre commit hook

* final config options

* try new approach

* update prettierignore

* [trends] don't allow persons on formula (#6830)

* don't allow persons on formula

* format

* Plugin server tests CI (#6897)

* remove workflows from plugin server subdir

* wip plugin server ci tests

* prettier

* add ignore paths

* update paths

* Add event exclusion correlation config to project page (#6817)

* feat: add project level event exclusion settings controls

This doesn't add the actual exclusions to the correlations, which will
be done in a separate PR.

It also doesn't try to unify the `PersonPropertySelect` and
`EventSelect` components, although this seems likely a good idea,
although possibly there will be some specifics that come up in review
that highlight differences required between them. If not then worth a
followup PR to merge.

* Fix typing of `EventDefinition.description`

This previously was required string, but it seems that the backend does
not return anything for this field in some (all?) cases

* update styling in line with person props

* Update correlation config

* use excluded names in funnel correlation

* make description non-optional again

It causes type error in other places by doing this, to I've just
modified the api response.

* Make tests work with paywall

* factor out config update logic

* chore: clean up EventSelect to not be explicitly about exclusions

* fix storybook typing

* Fix signup css (#6905)

* fix cypress funnel tests, volkswagen style (#6906)

* Adds popup styling so that the filter list popup has a background (#6916)

* adds popup styling so that the filter list popup has a background and drop shadow

* add break word overflow wrap to pop-ups

* Bump color-string from 1.5.4 to 1.6.0 (#6908)

Bumps [color-string](https://github.com/Qix-/color-string) from 1.5.4 to 1.6.0.
- [Release notes](https://github.com/Qix-/color-string/releases)
- [Changelog](https://github.com/Qix-/color-string/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Qix-/color-string/compare/1.5.4...1.6.0)

---
updated-dependencies:
- dependency-name: color-string
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Cleanup unused python imports (#6911)

* Don't GET licenses endpoint every time `SitePopover` is opened (#6913)

* Fix undefined error in extractObjectDiffKeys (#6807)

* fix undefined error

* remove duplicate check

* Show session events in seekbar (#6756)

* rename sessionRecording to session-recording

* logic for new player

* move rrweb-player into repo

* remove awkward wrapper and fix time not updating bug

* chunk load meta timing

* fix duration setting bug

* update icons and styling for controller

* styling icons

* slider styling

* ugly slider

* remove consoles

* sonarqube

* typescript

* typescript

* implement working seekbar

* fix time mismatches

* remove hardcode

* Fix icon from merge

* move report usage back to sessionsplaylogic

* refactor to seekbar

* implement real time seek time

* play on paused fixed

* paginate recording compression

* some tests

* bug with play pause

* more accurate duration calculation

* implement event fetching from frontend

* add tests and types

* fix buffer play state bug

* add comments back

* remove consoles and revert limit

* remove empty file

* tons of decompression fixes

* rename test file to avoid conflict

* move decompression to helper

* add test for helper

* type fix

* rename method

* simplify paginated decomression

* address feedback

* handle case where offset exceeds length

* clean up

* test fixes

* fix scrubbing issue and onclick lag

* remove console

* clean up on aisle 12

* Add surrounding object for metadata response

* split out endpoint consumption

* fix cypress tests

* use kea-window-values plugin and remove isTouchDevice check

* remove todo

* report usage refactor

* set buffering on last time catch up

* clamp buffer ms

* add logic tests

* fix eventusage

* finish up tests

* remove console

* update duration on first pain

* split out v1 and v2 sessionsPlayLogic and cleanup logic

* revert ts

* remove defaults from tests

* Move mocks into session-recording folder

* fix test

* rename to sessionRecordingLogic

* kea

* rename to sessionRecordingLogic

* cherry pick rename

* update yarn.lock

* prettier

* revert lockfiles

* create seekbar markers selector

* order events ascending

* add tick styling

* fix time buffer

* add events to seekbar

* prettier broke sass

* comment out testing

* fix event click when paused

* fix events next_url

* add info bubble to ticks

* fix existing logic tests

* fix tests

* remove commented out line

Co-authored-by: Rick Marron <rcmarron@gmail.com>

* Instrument saved insights (#6901)

* instrument saved insight filters

* instrument saved insight filters even more

* mount the logic

* new insight opens URLs

* instrument "new insight" button

* fix capitalization

* fix feature flag sync script with local deleted flags (#6919)

* Fresh pinned dashboards (#6912)

* Add `LemonButtonWithPopup`

* Add `sameWidth` option to `Popup`

* Add New project button

* Hide project switcher on click inside

* Unify `LemonButton` and `LemonButtonWithPopup` for extensibility

* Refactor for future pinned dashboards

* Refactor for future pinned dashboards

* Add pinned dashboards to the sidebar

* Fix minor issues

* BE (Groups/Trends): Allow aggregating by groups (#6894)

* Type math in Entity

* Allow passing group_type_index from FE to BE

* Get a initial query running

* Add group value filter if aggregating by groups

* Add snapshot testing for trends queries

* isort

* Update tests

* Add test for column_optimizer

* Update ee/clickhouse/queries/trends/util.py

Co-authored-by: Neil Kakkar <neilkakkar@gmail.com>

Co-authored-by: Neil Kakkar <neilkakkar@gmail.com>

* Fix Popup styling (#6920)

* BE (Groups/Trends): Make breakdowns work with groups (#6899)

* Extract GroupsJoinQuery

* Add test for breakdown filtering

* Unify breakdown mixins

* Allow passing breakdown_type == 'group' with breakdown_group_type_index

* Allow breakdown by group props in trends

* Add tests for trends breakdown_props function on group breakdowns

* Solve common issues

* Output snapshot diff into console

* Clean up materialized columns after tests

* Add zero protection

* Solve test failure

* Enable excluding event properties (#6820)

* Enable excluding event properties

* some cleanup

* clean things up

* add tests

* further simplification

* rename exclude event

* address comments

* Support filtering by group properties in retention (#6904)

* Feature flags turbo mode (#6895)

* rename "experimentation" to "feature flags"

* remove BackTo from flags

* make feature flags turbo-mode compatible

* fix feature flags loading

* fix bug if object exists and returns None

* update url when saving

* update index when saving

* take data from featureFlagsLogic if mounted

* sync values with form both ways

* Add feature flag-based lemonade opt-in (#6914)

* Add feature flag-based lemonade opt-in

* Add a `posthog.capture`

* Only show `RedesignOptIn` on cloud or in dev

* Polish `RedesignOptIn`

* Add `LemonSwitch`

* Add `LemonSwitch` loading state

* WIP: Groups FE filtering (#6764)

* Add /api/projects/@current/groups/property_definitions

To load property definitions for groups

* Add routes for fetching group types

* FE: Group analytics feature flag

* FE: Group analytics base types for properties

* FE: Initial implementation of fetching logics

* add group tabs to filter

* make sure group properties populate and refactor things

* prettier

* disable unused vars

* allow indexing objects by number

* mount group properties logic

Co-authored-by: Li Yi Yu <li@posthog.com>

* Lemonade polish (#6925)

* Add Billing button

* Instrument `data-attr`s analogous to old UI

* Hide sidebar in fullscreen

* Fix issues mentioned in standup

* Remove unused imports

* Update posthog-js to 1.16.2 (#6933)

* BE (Groups/Retention): Support aggregating by groups (#6922)

* Add groups stuff

* Rename column from person_id to `target` in retention queries

No behavioral change, preparing for groups work :)

* Remove dead if statement

* WIP: Retention aggregation by groups

* Handle aggregation by groups in retention

Also handles the case where not every event has a property defined

* Test groups validation mixin

* Reformat

* Improve test for aggregation in retention

* Style links in `Announcement` based on `:visited` (#6815)

* BE (Groups/Trends): Testing follow-up PR (#6923)

* Improve process_math

* Add test for overlapping group keys

* Improve event query tests

* Add test for filtering by person properties together with groups

* Avoid flaky tests due to cohort_id changing

* Update queries and snapshots

* New UI shows status as danger  (#6927)

* lemonade shows status as danger because status check assumes only people with access to status metric will see it

* reverse boolean in name to reflect actual value

* Fix issues with correlation instrumentation (#6921)

* Fix issues with instrumentation

* rm extras

* Backend: Groups Aggregation on Funnels (#6937)

* smallest change to make aggregation work
* use team in filter

* Funnel Groups: Rename person_id to aggregation_target (#6939)

* smallest change to make aggregation work

* address comments

* add snapshot

* move function to groups model

* update funnel snapshot

* rename person_id to aggregation_target

* update snapshots as well

* dont support persons query mods for now

* update snapshot

* make array orders deterministic

* New insight through insightRouter (#6924)

* route new insight through insight router

* remove comments

* Groups: Use materialized columns for groups (#6938)

* Migration to use materialized columns for groups

Workaround for https://github.com/PostHog/posthog/issues/6422

* Use groups materialized columns in queries

* Update mat column creation tests

* Simplify aggregation_target_field

* Fix migration

* Update snapshots

* New billing pages UI (#6753)

* [local testing] Track executed migrations when locally testing (#6915)

* experiment with saving a migrations table

* change flow

* change names

* missing var

* Unify Insight page background/shadow/border style with rest of app (#6943)

* Unify Insight page background/shadow/border style with rest of app

* Reduce main area padding to 32px

* Fix help button alignment (#6956)

* Fix help button alignment

* remove unused button block

* Player tweaks and fixes (#6928)

* fix player close button position

* reduce player padding

* allow recording on seek bar

* remove skip inactivity toggle

* style fix

* update rrweb

* add svg for remvoed content

* add ts ignore to legacy files

* Fix member join email template (#6958)

Co-authored-by: Sam Winslow <sammywinslow@gmail.com>

* Ingest window_id for recordings (#6806)

* add window-id to ingestion

* move window_id into snapshot data

* Add tests

* add window_id to snapshot_data only if included in the event

* add window_id to session_recording_events table

* revert ph-js downgrade

* fix data generator

* remove partition change

* update window_id defaults for tests

* missed one window_id default

* rename migration for conflict

* Fix seekbar dragging outside of click handler boundaries (#6962)

* fix buffer bug (#6969)

* Add path…
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants