Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Install psycopg2 based on platform #60

Merged
merged 11 commits into from
Apr 12, 2024
Merged

Install psycopg2 based on platform #60

merged 11 commits into from
Apr 12, 2024

Conversation

mikealfare
Copy link
Contributor

@mikealfare mikealfare commented Apr 12, 2024

resolves #50

Problem

psycopg2 takes a long time to build, and needs additional os dependencies, so it's not great for development. psycopg2-binary solves this problem. However, psycopg2-binary has some performance issues due to being build platform-agnostic, so it's not great for production.

Solution

  • make the assumption that most production workloads are hosted on linux and most development workloads are hosted on macos/windows
  • identify desired version of psycopg2 based on host platform

Side effect:

  • remove partial parsing tests; they started failing and they don't use any dbt-postgres or dbt-adapters functionality

Benefits

  • remove hatch build hooks and associated complexities
  • gain standard automation tools, e.g. dependabot alerts
  • remove need to rely on an environment variable at runtime

Costs

  • there will be development users on linux and production workloads on windows (likely Windows Server)
  • document workaround for edge cases on how to get the other

Checklist

  • I have read the contributing guide and understand what's expected of me
  • I have run this code in development and it appears to resolve the stated issue
  • This PR includes tests, or tests are not required/relevant for this PR
  • This PR has no interface changes (e.g. macros, cli, logs, json artifacts, config files, adapter interface, etc) or this PR has already received feedback and approval from Product or DX

@mikealfare mikealfare self-assigned this Apr 12, 2024
@@ -0,0 +1,6 @@
kind: Fixes
body: Determine `psycopg2` based on `platform_system` (Linux or other)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
body: Determine `psycopg2` based on `platform_system` (Linux or other)
body: Determine `psycopg2` based on `platform_system` (Linux or other), remove usage of `DBT_PSYCOPG2_NAME` environment variable

@mikealfare mikealfare merged commit 32c8547 into main Apr 12, 2024
17 checks passed
@mikealfare mikealfare deleted the dependencies/psycopg2 branch April 12, 2024 23:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Regression] Installing psycopg2-binary instead of psycopg2
2 participants