Tp2000 1273 Draft - Do Not Merge - Upgrade Psycopg from v2 to v3 #1218
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
TP2000-1273 Upgrade Psycopg - Draft PR DNM
Why
As part of the prep work for migrating from Govuk Paas to DBT Paas, we upgraded Postgres from v12 to v16 in Docker. It seemed fitting that we upgrade Psycopg as well seeing as they work together. We currently are on V2 known as Psycopg2, and want to upgrade to V3 known as just Psycopg. This ticket has been parked for the moment as upgrading has been tricker and more time consuming than we had hoped.
In the major upgrade, with little to no documentation, psycopg seems to move some modules around, and also remove the DateTimeRange class that we currently extend from for our TaricDateTimeRange. In efforts to sort of patch things back together, we're getting some strange behaviour where the end dates only of our ranges are having a day added on to them.
For example - If you go to the UI to make a quota, and give a validity date of 01/01/2024 to 01/01/2025, it will save in the DB as 2024-01-01 to 2025-01-02. This date range change shows in both the UI, shell plus, and in psql so is being saved to the DB incorrectly rather than a template mishap or something.
After doing some investigating, we've boiled it down to either an issue brought in by the range class I try to use to replace the deprecated DateTimeRange we use in our TaricDateTimeRange class.. or a more discreet issue that has appeared in the still supported DateRange class that's moved from the extras module to the types.ranges module (as they're the only things I change.. it's a bit baffling)
We parked this ticket as it's considered a 'nice to have' in comparison to the other tickets required for migrating onto the new DBT PaaS platform.. but when anyone picks this up, I hope this rundown helps!
Checklist