Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add docs for using psycopg2 with dbt-postgres #5707

Merged
merged 11 commits into from
Jul 25, 2024
Merged

Conversation

mikealfare
Copy link
Contributor

What are you changing in this pull request and why?

We changed how dbt-postgres is installed with regards to psycopg2 versus psycopg2-binary.
In prior versions you could configure it with an environment variable. In 1.8, that's not an option because we changed our build tooling to a more modern stack. We initially tried an OS-driven approach, but that had issues mainly due to CI workflows that normally run on linux images. So in 1.8.2 we reverted back to installing psycopg2-binary by default and prescribed the manual workaround of uninstalling psycopg2-binary and reinstalling psycopg2.

Checklist

@mikealfare mikealfare requested a review from a team as a code owner June 27, 2024 14:58
Copy link

vercel bot commented Jun 27, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
docs-getdbt-com ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jul 25, 2024 7:52am

@github-actions github-actions bot added content Improvements or additions to content size: medium This change will take up to a week to address labels Jun 27, 2024
@mikealfare mikealfare added the external-adapters-team auto labeled issues from the adapter team label Jun 27, 2024
@mikealfare mikealfare requested review from jtcohen6 and removed request for a team June 27, 2024 14:59
Copy link
Collaborator

@jtcohen6 jtcohen6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mikealfare Thank you for this - looks great!


In versions 1.8.0 and 1.8.1, `psycopg2-binary` is installed on MacOS and Windows operating systems and `psycopg2` is installed on Linux operating systems.
This has the side effect of requiring the OS dependencies identified above to install `dbt-postgres` on Linux.
Users will either need to update their workflows to install these dependencies, or upgrade to 1.8.2.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Given that we recommend users always install the latest (patch) version, no one should be installing v1.8.0 or v1.8.1 (instead of v1.8.2+) — but given you've already written it, I'm happy to have it in here :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed on the recommendation. I'm trying to be cognizant of the fact that dbt-postgres tends to be used more as a base adapter than other adapters (e.g. what we used to do with dbt-redshift). I thought it might be more likely in those scenarios that folks are hard pinning the version.

@mikealfare mikealfare assigned matthewshaver and unassigned nghi-ly Jul 9, 2024
Copy link
Contributor

@VersusFacit VersusFacit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Little nits and thoughts for final revision.

Co-authored-by: Mila Page <67295367+VersusFacit@users.noreply.github.com>
mikealfare and others added 4 commits July 12, 2024 12:02
Co-authored-by: Mila Page <67295367+VersusFacit@users.noreply.github.com>
Co-authored-by: Mila Page <67295367+VersusFacit@users.noreply.github.com>
Copy link
Contributor Author

@mikealfare mikealfare left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@VersusFacit I incorporated most of your feedback. Let me know what you think of the updated version.


In versions 1.8.0 and 1.8.1, `psycopg2-binary` is installed on MacOS and Windows operating systems and `psycopg2` is installed on Linux operating systems.
This has the side effect of requiring the OS dependencies identified above to install `dbt-postgres` on Linux.
Users will either need to update their workflows to install these dependencies, or upgrade to 1.8.2.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed on the recommendation. I'm trying to be cognizant of the fact that dbt-postgres tends to be used more as a base adapter than other adapters (e.g. what we used to do with dbt-redshift). I thought it might be more likely in those scenarios that folks are hard pinning the version.

@colin-rogers-dbt
Copy link
Contributor

@matthewshaver is there a blocker to merging this?

@mirnawong1
Copy link
Contributor

mirnawong1 commented Jul 24, 2024

Happy to merge this for you tomorrow @mikealfare if there are no issues. This isn't tied to v1.9 right?

@mikealfare
Copy link
Contributor Author

Happy to merge this for you tomorrow @mikealfare if there are no issues. This isn't tied to v1.9 right?

I'm not sure what is meant by "tied to v1.9". This takes effect in 1.8.2, and then would continue into 1.9.0 and beyond.

@mirnawong1
Copy link
Contributor

great thanks @mikealfare, was just making sure this wasn't an upcoming 1.9 feature but you're right - it's very clear this is live already. ! will merge this pr for yoU!

@mirnawong1 mirnawong1 enabled auto-merge (squash) July 25, 2024 07:46
@mirnawong1 mirnawong1 merged commit 512994e into current Jul 25, 2024
9 of 10 checks passed
@mirnawong1 mirnawong1 deleted the psycopg2-installation branch July 25, 2024 07:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
content Improvements or additions to content external-adapters-team auto labeled issues from the adapter team size: medium This change will take up to a week to address
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Regression] Forced psycopg2 dependency on Linux causes sudden build failures
7 participants