Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(core): add service scoped database clients #647

Merged
merged 24 commits into from
Apr 29, 2024

Conversation

jrschumacher
Copy link
Member

@jrschumacher jrschumacher commented Apr 23, 2024

This PR introduces a significant change to how we perceive database clients within the platform. The original implementation has a concept of a single database client which services could use. As work was being executed, it became clear that the policy service was one (if not only) service which required a database client.

Recently, we learned that downstream projects that wrap OpenTDF required access to a database client, but this posed a couple concerns:

  1. Downstream clients need the ability to add migrations and have those bound to the migration CLI tool as well as the auto-migration feature
  2. We need to ensure a bad actor cannot poison a PEP and thus have easy access to mutate the policy outside an audited and authenticated flow
  3. We need to ensure PEPs are interacting with services via contracts and not by direct database access, which could slow down development

Supported features

  • create a new database client for each service / PEP
    • this sets us up to support limiting scope in the future by giving theoretical access to managing multiple DB user identities per instance of the platform
  • scope the database client to a schema scoped to the service namespace
    • this will give us theoretical control over limiting the access of the database user
    • this might need to change to a database per service depending on the access controls needed
  • add support for service based migrations
    • having all services use the same migrations directory is brittle since clients are scoped to schemas we can support multiple migration directories
    • additionally, when deploying a policy service apart from another PEP we don't want to automatically migrate a schema that won't be used
  • migration commands now support service and/or --all
    • this enables developers to migrate up and down as they are exploring the process of supporting a new feature
    • this gives us the opportunity to enhance the tool to support migrations for services that are currently "enabled"

Unsupported features

  • robust support for service auto-migration based on server registration
  • rigorous performance testing
  • DX review

@jrschumacher jrschumacher requested review from a team as code owners April 23, 2024 17:10
@jrschumacher jrschumacher requested a review from a team as a code owner April 23, 2024 18:51
@jrschumacher jrschumacher requested a review from a team as a code owner April 24, 2024 00:14
@jrschumacher jrschumacher linked an issue Apr 24, 2024 that may be closed by this pull request
@jrschumacher jrschumacher added this pull request to the merge queue Apr 29, 2024
Merged via the queue into main with commit 019a3bf Apr 29, 2024
16 checks passed
@jrschumacher jrschumacher deleted the jrschumacher/issue559 branch April 29, 2024 17:15
github-merge-queue bot pushed a commit that referenced this pull request Apr 29, 2024
🤖 I have created a release *beep* *boop*
---


##
[0.3.0](service/v0.2.0...service/v0.3.0)
(2024-04-29)


### Features

* **core:** add service scoped database clients
([#647](#647))
([019a3bf](019a3bf))


### Bug Fixes

* **config:** update docs for enforce dpop config and clean up markdown
tables ([#697](#697))
([983ce71](983ce71))
* **policy:** normalize FQN lookup to lower case
([#668](#668))
([cd8a875](cd8a875)),
closes [#669](#669)

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: opentdf-automation[bot] <149537512+opentdf-automation[bot]@users.noreply.github.com>
tech-guru42 added a commit to tech-guru42/TDF that referenced this pull request Jun 3, 2024
🤖 I have created a release *beep* *boop*
---


##
[0.3.0](opentdf/platform@service/v0.2.0...service/v0.3.0)
(2024-04-29)


### Features

* **core:** add service scoped database clients
([#647](opentdf/platform#647))
([019a3bf](opentdf/platform@019a3bf))


### Bug Fixes

* **config:** update docs for enforce dpop config and clean up markdown
tables ([#697](opentdf/platform#697))
([983ce71](opentdf/platform@983ce71))
* **policy:** normalize FQN lookup to lower case
([#668](opentdf/platform#668))
([cd8a875](opentdf/platform@cd8a875)),
closes [#669](opentdf/platform#669)

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: opentdf-automation[bot] <149537512+opentdf-automation[bot]@users.noreply.github.com>
passion-127 added a commit to passion-127/TDF that referenced this pull request Jun 6, 2024
🤖 I have created a release *beep* *boop*
---


##
[0.3.0](opentdf/platform@service/v0.2.0...service/v0.3.0)
(2024-04-29)


### Features

* **core:** add service scoped database clients
([#647](opentdf/platform#647))
([019a3bf](opentdf/platform@019a3bf))


### Bug Fixes

* **config:** update docs for enforce dpop config and clean up markdown
tables ([#697](opentdf/platform#697))
([983ce71](opentdf/platform@983ce71))
* **policy:** normalize FQN lookup to lower case
([#668](opentdf/platform#668))
([cd8a875](opentdf/platform@cd8a875)),
closes [#669](opentdf/platform#669)

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: opentdf-automation[bot] <149537512+opentdf-automation[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ADR: downstream PEP database integration
3 participants