Add new person matieralized #1944
Conversation
| ) | ||
| ARRAY JOIN array_property_keys, array_property_values | ||
| ) ep | ||
| WHERE {filters} |
| ) | ||
|
|
||
| MAT_PERSONS_WITH_PROPS_TABLE_SQL = """ | ||
| CREATE MATERIALIZED VIEW persons_with_array_props_mv |
There was a problem hiding this comment.
We'll need to manually push the persons table into this materialized view when we deploy on prod because this is a TO table so it doesn't have auto-populating capabilities with the POPULATE
EDsCODE
requested review from
fuziontech and
timgl
and removed request for
fuziontech
…/PostHog/posthog into ch-person-materialized-aggregate
| def _destroy_person_tables(self): | ||
| sync_execute(DROP_PERSON_VIEW_SQL) | ||
| sync_execute(DROP_PERSON_MATERIALIZED_SQL) | ||
| sync_execute(DROP_PERSON_TABLE_SQL) | ||
| sync_execute(DROP_PERSON_DISTINCT_ID_TABLE_SQL) | ||
|
|
||
| sync_execute(DROP_PERSONS_PROP_UP_TO_DATE_VIEW_SQL) | ||
| sync_execute(DROP_MAT_PERSONS_PROP_TABLE_SQL) | ||
| sync_execute(DROP_MAT_PERSONS_WITH_ARRAY_PROPS_TABLE_SQL) | ||
| sync_execute(DROP_PERSONS_WITH_ARRAY_PROPS_TABLE_SQL) | ||
|
|
||
| def _create_person_tables(self): | ||
| sync_execute(PERSONS_TABLE_SQL) | ||
| sync_execute(PERSONS_DISTINCT_ID_TABLE_SQL) | ||
| sync_execute(PERSONS_UP_TO_DATE_MATERIALIZED_VIEW) | ||
| sync_execute(PERSONS_UP_TO_DATE_VIEW) | ||
|
|
||
| sync_execute(PERSONS_WITH_PROPS_TABLE_SQL) | ||
| sync_execute(MAT_PERSONS_WITH_PROPS_TABLE_SQL) | ||
| sync_execute(MAT_PERSONS_PROP_TABLE_SQL) | ||
| sync_execute(PERSONS_PROP_UP_TO_DATE_VIEW) |
ee/clickhouse/views/person.py
Outdated
| # queryset = queryset.filter( | ||
| # Filter(data={"properties": json.loads(request.GET["properties"])}).properties_to_Q(team_id=team.pk) | ||
| # ) | ||
| print(PEOPLE_BY_TEAM_SQL.format(filters=all_filters)) |
ee/clickhouse/sql/person.py
Outdated
| ENGINE = AggregatingMergeTree() ORDER BY ( | ||
| id, | ||
| team_id, | ||
| updated_at | ||
| ) |
There was a problem hiding this comment.
Ah just realized as I was migrating this:
You want the sort order to be setup so that you can find groups easily, usually with broad to fine granularity.
For instance on this id, team_id, updated_at everything will be first sorted by id which will make the other order keys pretty much useless since id's will be unique.
The best order really depends on how you are going to query the table. In this case I would think that maybe team_id, id might be the best ordering if we are going to be picking persons out of the db by team and id. If we are looking to count the number of persons by period then we will want to do team_id, updated_at, id so that you can say for team x I want to look at people between a and b times.
There was a problem hiding this comment.
since this is aggregatingMT we have to have all non agg fields in the sort so the last one is prob best.
Changes
Please describe.
If this affects the front-end, include screenshots.
Checklist