Skip to content
This repository has been archived by the owner on Nov 4, 2021. It is now read-only.

Set precedence to DATABASE_URL over POSTHOG_DB_* fields #441

Merged
merged 10 commits into from
Jun 4, 2021
Merged

Conversation

neilkakkar
Copy link
Contributor

Changes

This reconciles precedence order to be the same as on PostHog/posthog. This is important, because otherwise cases are possible where the Django app connects to postgres without issues, but the plugin server doesn't: so events are shown on the UI, but aren't ingested properly.

Specially, have seen this happen with SSL enabled DB urls, like in this case: https://posthogusers.slack.com/archives/C01GLBKHKQT/p1621416865003200

Checklist

  • Updated Settings section in README.md, if settings are affected
  • Jest tests

@neilkakkar neilkakkar requested a review from Twixes May 27, 2021 15:23
Copy link
Collaborator

@Twixes Twixes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should not be needed for any setup though 🤔 POSTHOG_DB_NAME – AFAIK – has to be explicitly set by the user, in which case we can assume it's more deliberate than DATABASE_URL – which practically always has a value, because it has a default.

@@ -135,7 +135,8 @@ export async function createHub(
timeStr ? DateTime.fromSQL(timeStr, { zone: 'utc' }).toISO() : null
)

const postgres = createPostgresPool(serverConfig)
const configOrDatabaseUrl = serverConfig.DATABASE_URL ? serverConfig.DATABASE_URL : serverConfig
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will always result in DATABASE_URL being used, because it has a non-empty default value (while POSTHOG_DB_NAME is null by default).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, right, this is annoying :$ - looks like it's not required on main PostHog: https://github.com/PostHog/posthog/blob/master/posthog/settings.py#L369-L376 - and hence the precedence order there.

If we use POSTHOG_DB_NAME, while PostHog/posthog uses DATABASE_URL, that's.. not good.

I wonder if we can get around this by removing the default value for DATABASE_URL ? Does that break things / workflows elsewhere? (I'm guessing probably yes)

.. or maybe it's a documentation thing? We explicitly mention to use one or the other? Although, I'd prefer to fix this ourselves so things are consistent.

Copy link
Collaborator

@Twixes Twixes May 27, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, def not good to have a discrepancy. I suppose the fix would then be removing the default value of DATABASE_URL here in the non-test non-dev case (should be OK), and adjusting createPostgresPool for same control flow as in the main repo.

@neilkakkar neilkakkar self-assigned this May 28, 2021
@neilkakkar neilkakkar added this to In Progress in Extensibility May 28, 2021
@neilkakkar neilkakkar requested a review from Twixes June 2, 2021 10:37
connectionString: configOrDatabaseUrl.DATABASE_URL,
}
: {
database: configOrDatabaseUrl.POSTHOG_DB_NAME || undefined,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a better way to convert null to undefined for types?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can also use ?? which is the nullish coalescing operator – where || returns right side when left side is falsy, ?? returns right side only when left side is null or undefined. So depends on whether we want the plugin server to treat '' as a valid (if empty) string value (→ ??), or as a lack of value i.e. undefined (→ ||).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do what PostHog main does: null is not ok, falsy is ok.

Copy link
Collaborator

@Twixes Twixes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, discrepancy is no more. A few extra edge cases

README.md Outdated Show resolved Hide resolved
tests/helpers/sql.ts Outdated Show resolved Hide resolved
tests/helpers/sql.ts Outdated Show resolved Hide resolved
tests/helpers/graphile.ts Outdated Show resolved Hide resolved
@Twixes Twixes moved this from In Progress to In Review/Pending Deployment in Extensibility Jun 3, 2021
@neilkakkar neilkakkar requested a review from Twixes June 4, 2021 09:50
.github/workflows/ci.yml Outdated Show resolved Hide resolved
@Twixes Twixes merged commit 14038ae into master Jun 4, 2021
@Twixes Twixes deleted the pgprec branch June 4, 2021 20:15
Extensibility automation moved this from In Review/Pending Deployment to Done This Sprint Jun 4, 2021
@Twixes Twixes moved this from Done This Sprint to Done Historically in Extensibility Jun 8, 2021
fuziontech pushed a commit to PostHog/posthog that referenced this pull request Oct 12, 2021
…ds (PostHog/plugin-server#441)

* set precedence to DATABASE_URL

* make DATABASE_URL non mandatory, check env var configuration

* fix tests

* address comments

* check what went wrong

* Remove extra cat

😾

Co-authored-by: Michael Matloka <dev@twixes.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
Extensibility
Done Historically
Development

Successfully merging this pull request may close these issues.

None yet

2 participants