Skip to content

Commit

Permalink
Deprecation of postgres notice (#2465)
Browse files Browse the repository at this point in the history
Co-authored-by: Paolo D'Amico <paolodamico@users.noreply.github.com>
  • Loading branch information
timgl and paolodamico committed Nov 17, 2021
1 parent b3c4894 commit 1c9922e
Show file tree
Hide file tree
Showing 2 changed files with 38 additions and 28 deletions.
28 changes: 18 additions & 10 deletions contents/blog/the-posthog-array-1-30-0.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,15 +13,27 @@ excerpt: Brand new and faster user interface, automatic conversion signal detect

PostHog 1.30.0 is a milestone release! We've introduced a brand new, faster user interface, automatic conversion signal detection with correlation analysis, the ability to save insights for future use and a fully revamped recordings playback experience. And that's just for starters!

<blockquote class='warning-note'>
<b>Postgres-based deployments are now deprecated</b> in favor of ClickHouse-backed installations. It's important to migrate your installation to keep getting the latest updates and features. <a href="#deprecation--removal-notices">Read more</a> about this below.
</blockquote>

## PostHog 1.30.0 release notes

> Don't see the new features on your self-hosted deployment? Remember to [update your PostHog instance](/docs/self-host/configure/upgrading-posthog).
**Release highlights:**
- [Fresh new look-and-feel](#fresh-new-look-and-feel)
- [Correlation analysis](#correlation-analysis)
- [Saved insights](#saved-insights)
- [Fully revamped recordings](#fully-revamped-recordings)
- [PostHog 1.30.0 release notes](#posthog-1300-release-notes)
- [Fresh new look-and-feel](#fresh-new-look-and-feel)
- [Correlation analysis](#correlation-analysis)
- [Saved insights](#saved-insights)
- [Fully revamped recordings](#fully-revamped-recordings)
- [Other improvements & fixes](#other-improvements--fixes)
- [Deprecation & removal notices](#deprecation--removal-notices)
- [Help us improve PostHog](#help-us-improve-posthog)
- [Community](#community)
- [Community MVP 🏆](#community-mvp-)
- [Community shoutouts](#community-shoutouts)
- [Open roles](#open-roles)

### Fresh new look-and-feel

Expand Down Expand Up @@ -80,19 +92,15 @@ The recordings experience just got a lot better. We added a new recordings tab t

### Deprecation & removal notices

1. We're now fully removing the legacy Sessions list page. Read more about it, [in this blog post](/blog/sessions-removal).

1. This version (`1.30.0`) will be the last version where we support a Postgres-only deployment of PostHog. See [our migration guide](/docs/self-host/migrate-from-postgres-to-clickhouse) for instructions on moving over to a ClickHouse version. ClickHouse provides faster queries and is optimized for very large volumes of data, and you will also get a new lot of features.
2. We're now fully removing the legacy Sessions list page. Read more about it, [in this blog post](/blog/sessions-removal).

### Help us improve PostHog

We’re working hard to improve the PostHog experience and would love to talk to you! Please join one of our Product, Engineering, or Marketing team members on a quick 30-min call to help us understand how to improve. Schedule directly [on Calendly](https://calendly.com/posthog-feedback).

As a small thank you for your time, we're giving away awesome [PostHog merch](https://merch.posthog.com)!

## PostHog News

No new joiners this month, but stay tuned for new people joining real soon!

## Community
### Community MVP 🏆

Expand Down
38 changes: 20 additions & 18 deletions contents/docs/self-host/migrate-from-postgres-to-clickhouse.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -4,20 +4,22 @@ sidebar: Docs
showTitle: true
---

> ⚠️ This doc is still in **Alpha**. If you're attempting this migration, feel free to ask questions and provide feedback via the [PostHog Communty Slack workspace](/slack) or [a GitHub issue](https://github.com/PostHog/posthog.com/issues). You should also be aware that **some of the steps on this document are potentially destructive**! Proceed with caution.
> ⚠️ This doc is still in **Alpha**. If you're attempting this migration, feel free to ask questions and provide feedback via the [PostHog Communty Slack workspace](/slack) or [a GitHub issue](https://github.com/PostHog/posthog.com/issues). You should also be aware that **some of the steps on this document are potentially destructive**! Proceed with caution.
**PostHog backed by PostgreSQL is now deprecated**. We will continue to provide support to Postgres backed installs but we strongly encourage you to migrate to PostHog backed by ClickHouse for [vastly superior performance](docs/self-host/postgres-vs-clickhouse), as well as to receive new features and continued support in the future.

## Requirements

- You should have a **clean** [ClickHouse-backed PostHog instance up and running](docs/self-host). Your new PostHog instance should have **no ingested events**. We recommend using a fresh and unused installation.
- Your old and new instances should both be running **the exact same version of PostHog**, with a minimum version of `1.29.0` (note: if you do not follow release tags and instead update PostHog by pulling from `master`, you should make sure both versions are on the **same commit**.)
- You should have a **clean** [ClickHouse-backed PostHog instance up and running](docs/self-host). Your new PostHog instance should have **no ingested events**. We recommend using a fresh and unused installation.
- Your old and new instances should both be running **the exact same version of PostHog**, with a minimum version of `1.29.0` (note: if you do not follow release tags and instead update PostHog by pulling from `master`, you should make sure both versions are on the **same commit**.)

> **Note:** PostHog users on version 1.29.0 will be able to migrate events to a new instance, but autocaptured events will not be migrated correctly. This bug will be fixed in version 1.30.0. More information on the [Exporting events](#exporting-events) section.
> **Note:** PostHog users on version 1.29.0 will be able to migrate events to a new instance, but autocaptured events will not be migrated correctly. This bug will be fixed in version 1.30.0. More information on the [Exporting events](#exporting-events) section.
<details>

<summary><b>Having difficulties upgrading your Postgres-backed instance due to long-running migrations?</b></summary>
<summary>
<b>Having difficulties upgrading your Postgres-backed instance due to long-running migrations?</b>
</summary>

<br />

Expand All @@ -29,7 +31,7 @@ Check out [this guide](/docs/self-host/postgres-upgrade-migrations).

ClickHouse-backed PostHog instances still use a Postgres database to store data that is not used in analytical queries, such as user information, feature flags, dashboard configurations, etc.

However, tables that contain data used to run analytical queries such as events, persons, and person distinct IDs is now stored in ClickHouse.
However, tables that contain data used to run analytical queries such as events, persons, and person distinct IDs is now stored in ClickHouse.

With this migration, we will copy over the Postgres data that is still stored in Postgres to the new instance using tools from the Postgres ecosystem, and then migrate over your events using a PostHog plugin. The events migration will in turn create the necessary person, person distinct ID, and related records in ClickHouse.

Expand All @@ -42,7 +44,7 @@ With this migration, we will copy over the Postgres data that is still stored in
Access your old Postgres-backed PostHog instance and run the following command:

```shell
# tip: use the flag -U to specify a username if necessary
# tip: use the flag -U to specify a username if necessary
pg_dump -d posthog -f export.sql --no-owner --data-only \
-t posthog_action \
-t posthog_actionstep \
Expand Down Expand Up @@ -72,7 +74,7 @@ pg_dump -d posthog -f export.sql --no-owner --data-only \
### 2. Accessing Postgres on your new instance

To access Postgres on your new PostHog instance (Kubernetes cluster), you should do the following:

> **Tip:** Find out your pod names with `kubectl get pods -n posthog`
1. Find out your Postgres password from the web pod:
Expand All @@ -97,7 +99,6 @@ To access Postgres on your new PostHog instance (Kubernetes cluster), you should

Postgres will ask you for the password. Use the value you found from step 1.


### 3. Deleting data in the new instance

In order to correctly migrate your data over, we need to make sure the tables we're importing into are completely empty. This will ensure that the foreign key mappings are set up correctly.
Expand All @@ -117,9 +118,9 @@ posthog_annotation,
posthog_cohort,
posthog_dashboard,
posthog_dashboarditem,
posthog_featureflag,
posthog_featureflagoverride,
posthog_messagingrecord,
posthog_featureflag,
posthog_featureflagoverride,
posthog_messagingrecord,
posthog_organization,
posthog_organizationinvite,
posthog_organizationmembership,
Expand Down Expand Up @@ -148,7 +149,7 @@ To do so, run the following in the directory of your export (data dump) file:
kubectl cp export.sql posthog-posthog-postgresql-0:/tmp/export.sql -n posthog
```

This will copy the export file to a `tmp/` directory in the Postgres pod.
This will copy the export file to a `tmp/` directory in the Postgres pod.

### 5. Loading the data into the new instance

Expand All @@ -169,9 +170,11 @@ psql -d posthog -U postgres < tmp/export.sql

<details>

<summary><b>I'm seeing some errors, what should I do?</b></summary>
<summary>
<b>I'm seeing some errors, what should I do?</b>
</summary>

<br/>
<br />

If you see some errors on this final step, you can try to use `pg_restore` instead. You should follow all steps from the beginning, with two modifications:

Expand Down Expand Up @@ -231,7 +234,6 @@ posthog-web-78dns2f5d7-6zdkc 1/1 Running 0
posthog-worker-7857nd8268-j8c4f 1/1 Running 0 11d
```


## Exporting events

Once you have **completed the steps above**, you can now move on to migrating your events.
Expand All @@ -242,7 +244,7 @@ For this, we will use the [PostHog Migrator 3000 plugin](https://github.com/Post

This plugin works on PostHog instances with version 1.29.0 or above.

Version 1.30.0 (coming soon) fixes a bug with exporting autocaptured events, so, unless you do not use autocapture at all, we recommend upgrading to 1.30.0 on both instances before performing the events migration.
Version 1.30.0 fixes a bug with exporting autocaptured events, so, unless you do not use autocapture at all, we recommend upgrading to 1.30.0 on both instances before performing the events migration.

### Instructions

Expand All @@ -254,6 +256,6 @@ Version 1.30.0 (coming soon) fixes a bug with exporting autocaptured events, so,

4. A plugin configuration drawer should open. Here, add the hostname for your new instance, as well as the project API key from step 1. Also, make sure the toggle at the top is 'Enabled'.

5. Specify a date to start exporting events in the format `YYYY-MM-DD` (e.g. 2021-10-26) and click 'Save'.
5. Specify a date to start exporting events in the format `YYYY-MM-DD` (e.g. 2021-10-26) and click 'Save'.

6. That's it! The plugin will now start exporting your events to the new instance. To keep track of its progress, click on the 'Logs' icon for the plugin. Ignore the progress bar, focusing instead on the timestamps in the logs. Once the plugin is done exporting all of your historical events, it will then continue to export the events that are coming in live, but with a delay. Our recommendation is to change the event sources to point to the new instance once you notice all historical events have been exported.

0 comments on commit 1c9922e

Please sign in to comment.