Using projects as environments: syncing actions/insights/dashboards/playlists etc.

[lharries]

Is your feature request related to a problem?

Many users use projects as environments (production, staging, local dev).

However, if you create a dashboard/actions/insight in staging you need to re-create that manually within PostHog.

This came up with an onboarding user interview where we made an action on the staging env and then he needed to re-create it again on the production project

Describe the solution you'd like

• Be able to copy over or sync the actions, insights, dashboards (everything?) between two projects

There's already work by Team Experimentation on enabling this for Feature Flags. I think it's a sensible pattern to introduce across all the products.

This could be particularly unlocked by the PostHog 3000 concept of treating them all as files, hence allowing more easy duplication.

Describe alternatives you've considered

Have a separate concept of environments within a project. However, this is likely substantially more work including changing our assumptions built into queries.

Additional context

Came from a discussion with @Twixes

Another way we could make PostHog projects feel better suited to act as environments would be by enabling customers to explicitly label the projects as either Production/Staging/Test etc.

Thank you for your feature request – we love each and every one!

[lharries]

Further evidence for replicating dashboards across envs: https://posthog.slack.com/archives/C011L071P8U/p1671640760520999

[tmchow]

Other analytics systems handle environments better than Posthog. Commonplace is for you to build out reports/dashboards in a pre-production environment ahead of a feature launching. When it launches, then all your reporting is set up. Doing this in staging, for example, first, allows you to use it against actual live data which is espeically important if there are new custom events being tracked not yet in production.

Ideally, in Posthog, environments are a part of a project so you can flip between environments within a project. This allows you to do what I stated.

As is, mixing projects and environments is what is limiting today.

[millansingh]

One solution (though this may be a little hacky) would be to allow users to link multiple projects in a group which then syncs all insights, dashboards, etc. across all those projects. But either way, having multiple linked environments that share the same core config seems like a table-stakes feature tbh and I'm kinda shocked it's not implemented already.

[SaadBazaz]

This is super needed. Recreating feature flags can be prone to human error, and can be devastating and time-consuming for teams.

I like how Mixpanel enables you to "mix" multiple projects into one dashboard, and syncs them. Growthbook also has an excellent feature flag setup; one feature flag has all environments within it ("staging", "dev", "production").

We can go with either one of the two, or have "shared environments" i.e. a separate entity called environment (staging, dev, production) where we can set these as variables, or use these shared environments in the feature flags themselves.

[lksrpp]

I agree with the other posts: when setting up posthog with multiple environments for the first time now, I was quite surprised that it's impossible to re-use information from one project (staging) to the other (production). This leads to a lot of additional work that would be unnecessary.

Most importantly, I'm missing insights, dashboards, and so on that I created on staging. And I would like to maintain event and property data, include descriptions and so on - but doing this multiple times for every instance to cover all environments is not feasible.

[shareefhadid]

This is awful for the Flutter package because you are setting up the API key in Info.plist and AndroidManifest.xml. To swap the API key based on the environment you have to write build scripts which modify those files.

I can't understand why you don't have an init method where you can pass in the API key like every other package. Having it in a dart file makes it much easier to manipulate.

I don't want my test data mixed with prod data. For this reason alone I am not going to use PostHog.

[Twixes]

To this end, we'll be implementing project environments. See the RFC.

[Twixes]

Reopening to have a single place to keep track of the implementation. Based on the scope of work defined in the RFC, here's the current status:

Core scope

1. Project model added.
2. Every Team has a matching Project and is assigned to it.
3. Filter by team_id=self.team_id instead of team__project_id=self.team_id in the relevant API querysets.
4. Move TeamViewSet from /api/projects/ to /api/environments/. Add /api/projects/:project_id/environments/.
5. Turn the project selector into the environment selector.
6. Moving group types from team_id over to project_id.
7. Update TeamManager and GroupTypeManager in ingestion with project_id.
8. Moving event and property definitions from team_id over to project_id.
9. Update PropertyDefinitionsManager in ingestion with project_id.
10. Turn "Project settings" into "Environment settings".
11. In the navbar, turn the organization name into just an icon.
12. RELEASE ENVIRONMENTS FEATURE FLAG TO THE PUBLIC.
13. Update billing.
14. Update docs. (https://posthog.com/docs/settings/organizations-and-projects, https://posthog.com/tutorials/multiple-environments) Rewrite docs on projects for the environments era posthog.com#10819 – merge once out of beta

Polishes – discovered in the preview phase

1. Calculate cohorts for each environment separately.
2. Update FeatureFlagMatcher in terms of GroupTypeMappingCache and cohorts.
3. Allow feature flag rollout groups to be either project-wide or environment-specific. (E.g. ZEN-24780)
4. Update UI of personal API key scopes.
5. Move activity logs to be nested under environments, due to CDP destinations being environment-level.
6. Clarify mechanics of SQL tables and views. (e.g. ZEN-25191)
7. Make it 100% clear in the UI what is project-level and what is environment-level. (e.g. ZEN-25134)
8. Make the environments selector simpler. (e.g. ZEN-25317, Slack)
9. Move onboarding state and home dashboard from Team to Project.
10. Move some remaining project settings from Team to Project.
11. Ensure TeamAccessControl is 100% environment-level, and not project-level (RBAC-related?)
12. Allow users to migrate from multiple projects to environments. (e.g. ZEN-24145, ZEN-24225, Slack)

Cleanup

1. Mark the team_id field deprecated on models already mover over to project_id.
2. Migrate app URLs from /project/ to /env/.
3. Zapier integration: Rename project selection to environment selection.
4. Denormalize models used in querysets where team__project_id is used, so that those models have project_id – this unlocks straightforward deletion of environments.

[marandaneto]

This is awful for the Flutter package because you are setting up the API key in Info.plist and AndroidManifest.xml. To swap the API key based on the environment you have to write build scripts which modify those files.

I can't understand why you don't have an init method where you can pass in the API key like every other package. Having it in a dart file makes it much easier to manipulate.

I don't want my test data mixed with prod data. For this reason alone I am not going to use PostHog.

This is the issue to track the setting config on the Dart side.
There's a workaround but I agree, it'd be a much better UX setting everything on the Dart side, we just have not prioritized that yet but the issue is tagged with help-wanted as well, happy to guide any contribution coming from the community.