Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(storage): move from lib/pq to pgx #2996

Merged
merged 4 commits into from
Apr 23, 2024
Merged

refactor(storage): move from lib/pq to pgx #2996

merged 4 commits into from
Apr 23, 2024

Conversation

erka
Copy link
Collaborator

@erka erka commented Apr 17, 2024

closes #2982

Copy link

codecov bot commented Apr 17, 2024

Codecov Report

Attention: Patch coverage is 97.05882% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 72.57%. Comparing base (f997fb9) to head (a9a3a03).
Report is 251 commits behind head on main.

❗ Current head a9a3a03 differs from pull request most recent head ebb37d6. Consider uploading reports for the commit ebb37d6 to get more accurate results

Files Patch % Lines
internal/storage/sql/migrator.go 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2996      +/-   ##
==========================================
+ Coverage   70.78%   72.57%   +1.78%     
==========================================
  Files          91       95       +4     
  Lines        8729     7270    -1459     
==========================================
- Hits         6179     5276     -903     
+ Misses       2165     1604     -561     
- Partials      385      390       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@erka
Copy link
Collaborator Author

erka commented Apr 19, 2024

@GeorgeMac I am stuck a bit with this and need some help. pgx has special ConnectError which contains host and credentials to database. The thing is that the error message goes to the client without any preprocessing.

I see that Flipt has driver.AdaptError func but it isn't used for all sql storage. Any reason for that? Any advice how to prevent credentials leakage to the client?

@GeorgeMac
Copy link
Contributor

Sorry for the slow reply @erka !

I just did a little walkthrough as it has been a while since I looked at all this.

AdaptError came about when we added the authn and oplock packages.
Previously, with core storage/sql package an implementation of the storage.Store was created per driver, to support error mapping. This was getting a little verbose and repetetive, so I didn't want to perpetuate that same pattern into these new storage layers. Instead, I created AdaptError to be dropped in to the authn package where needed, so we could have a single sql storage implementation in these places.

Obviously, now we have two patterns, which is a shame:

I think the quick win for your PR would be to add support for this ConnectError to the storage/sql/postgres package. Perhaps a longer term fix would be to lean into AdaptError everywhere more. What do you think?

Sadly it is all a bit fiddly to implement. One thing I considered, but would be quite a subtle and fiddly thing to implement, would be to create an implementation of driver.Driver and intercept these errors there. That would at-least stop the error mapping from having to creep throughout the entire codebase, but still would probably have some subtle and weird edge-cases.

@erka erka force-pushed the pgx branch 4 times, most recently from e51d2e5 to c58f9b1 Compare April 22, 2024 21:31
@erka erka marked this pull request as ready for review April 22, 2024 21:47
@erka erka requested a review from a team as a code owner April 22, 2024 21:47
@erka
Copy link
Collaborator Author

erka commented Apr 22, 2024

@GeorgeMac driver.Driver is a great idea. It fixes the problem with minimal impact on codebase and simplifies maintenance in future. Thank you!

Signed-off-by: Roman Dmytrenko <rdmytrenko@gmail.com>
Signed-off-by: Roman Dmytrenko <rdmytrenko@gmail.com>
Copy link
Contributor

@GeorgeMac GeorgeMac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change looks great. Glad the driver.Driver approach worked out!
I think the change looks good. I put one take it or leave it on there.

Signed-off-by: Roman Dmytrenko <rdmytrenko@gmail.com>
Copy link
Collaborator

@markphelps markphelps left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks great to me! thank you @erka !!

@markphelps markphelps added the automerge Used by Kodiak bot to automerge PRs label Apr 23, 2024
@kodiakhq kodiakhq bot merged commit a2d18fc into flipt-io:main Apr 23, 2024
26 checks passed
@erka erka deleted the pgx branch April 23, 2024 13:53
markphelps pushed a commit that referenced this pull request Apr 24, 2024
* refactor(storage): move from lib/pq to pgx

Signed-off-by: Roman Dmytrenko <rdmytrenko@gmail.com>

* ignore golang mock files by codecov

Signed-off-by: Roman Dmytrenko <rdmytrenko@gmail.com>

* address PR feedback

Signed-off-by: Roman Dmytrenko <rdmytrenko@gmail.com>

---------

Signed-off-by: Roman Dmytrenko <rdmytrenko@gmail.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
Signed-off-by: Mark Phelps <209477+markphelps@users.noreply.github.com>
markphelps added a commit that referenced this pull request Apr 24, 2024
* 'main' of https://github.com/flipt-io/flipt:
  chore(deps): bump golang.org/x/net from 0.22.0 to 0.23.0 in /sdk/go (#3006)
  chore(deps): bump golang.org/x/net from 0.22.0 to 0.23.0 in /core (#3005)
  chore: update changelog for v1.40.2 (#3020)
  refactor(storage): move from lib/pq to pgx (#2996)
  fix(cmd/grpc): skip eval data service when evaluation API auth excluded (#3018)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
automerge Used by Kodiak bot to automerge PRs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Connect to multiple PostgreSQL hosts
3 participants