Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add structured schema policy config #51

Merged
merged 29 commits into from
Feb 5, 2024
Merged

Conversation

jrschumacher
Copy link
Member

No description provided.

strantalis and others added 5 commits January 18, 2024 16:39
…nships (#48)

* fix Timestamp proto type value in PolicyMetadata

* simplify attributes.proto according to ERD

* provide grpcurl examples for attributes and respond with created/updated values in those rpc flows
@jrschumacher jrschumacher marked this pull request as draft January 19, 2024 15:04
@jakedoublev
Copy link
Contributor

Addresses all of these issues:

#41
#39
#34
#33
#21

* additional tweaks to proto and client and update sdk

* feat: implement attributes

* Update protos

* Update attributes to work with new proto
@jrschumacher jrschumacher changed the title Policy config changes 🛑🛑🛑 DO NOT MERGE 🛑🛑🛑 Policy config changes Jan 19, 2024
* refactor

* Add migrate down command

* Fix sql command

* Use protojson.Unmarshal with json.RawMessage

* Add examples to proto

* Update attributes based on feedback

* Remove dead code

* Disable broken code (will undo as we work)
jrschumacher and others added 7 commits January 23, 2024 07:26
* feat(subject-mappings): refactor to meet db schema

* WIP

* Add subject mapping
* save kas

* key access server registry impl

* rename kasr dao filename

* add example request and response comments

---------

Co-authored-by: Ryan Schumacher <jschumacher@virtru.com>
…82)

* add helper for checking constraint violations

* improve error handling

* add tests

* update to define and test more types of postgres 'bad request' type errors

* clean up tests and add not found case to error handlers

* handle SELECT with no rows found error and dry up conflict error

* improve error wrapping by moving it down into the query and exec level of the db package

* avoid nil pointer dereference panic
…mespaces table (#54)

* Define protos for simple namespaces CRUD

* add generated namespaces sdk files

* add grpcurl namespaces examples

* feat: implement attributes

* add db layer for namespaces and add the serviceclient to the SDK after cherry-picking attributes DB work

* provide namespace back in response when creating/updating and add service layer code for namespaces

* make sure to register namespaces service on start

* namespaces cleanup

* namespaces test suite boilerplate

* move all row scanning to db layer

* service work for namespaces

* use tableField func in attributes

* use proper namespace table name

* require name and id, but id only once on update, and require only name on create

* ensure working crud of namespaces

* lint fix

* fix grpcurl update example

* add helper for checking constraint violations

* improve error handling

* consume error handling functions

* update to define and test more types of postgres 'bad request' type errors

* consume latest error helper updates

* validate working error handling with logs and messages in namespaces

* fix deletion

* improve error wrapping by moving it down into the query and exec level of the db package

* consume latest db error changes

* avoid nil pointer dereference

* declutter diff with varied lint settings

---------

Co-authored-by: Ryan Schumacher <jschumacher@virtru.com>
jakedoublev and others added 8 commits January 29, 2024 07:06
Closes #80

---------

Co-authored-by: Ryan Schumacher <jschumacher@virtru.com>
…namespaces table within attributes protos/schema (#99)

Closes: #75 
Now the following get/list requests for attributes look like this: 

```
Request: 
grpcurl -plaintext localhost:9000 attributes.AttributesService/ListAttributes

Response: 
{
  "attributes": [
    {
      "id": "ebec024c-9ec2-4465-9b5a-dc643abf07cb",
      "metadata": {
        "createdAt": "2024-01-26T21:45:15.978314Z",
        "updatedAt": "2024-01-26T21:45:15.978314Z"
      },
      "namespace": {
        "id": "348f3698-a0c9-4ed6-a90c-e589aaf59c57",
        "name": "example.com"
      },
      "name": "attribute_name",
      "rule": "ATTRIBUTE_RULE_TYPE_ENUM_ALL_OF"
    }
  ]
}
```
…ate/update, and consumes schema setup changes in namespaces that were introduced for integration testing (#100)

Closes #90 

A create request now looks like: 
```bash
grpcurl -plaintext -d '{"attribute_id":"ad244746-eff3-4719-ac0f-45bdc03ad59c", "value":{"value":"hello"}}' localhost:9000 attributes.AttributesService.CreateAttributeValue
```

An update request now looks like:
```bash
grpcurl -plaintext -d '{"attribute_id":"ad244746-eff3-4719-ac0f-45bdc03ad59c","id":"f6639485-9da8-41a0-acf7-fabfcfa1621d","value":{"value":"world"}}
' localhost:9000 attributes.AttributesService.UpdateAttributeValue
```

The aspect `value` is still required twice because the top-level `value`
points to the `ValueCreateUpdate` type `Message` which contains the
nested `value` string, optional `members`, and optional `metadata`.
@jrschumacher jrschumacher marked this pull request as ready for review February 2, 2024 23:48
@jrschumacher jrschumacher requested a review from a team as a code owner February 2, 2024 23:48
@jrschumacher jrschumacher changed the title 🛑🛑🛑 DO NOT MERGE 🛑🛑🛑 Policy config changes feat: add structured schema policy config Feb 2, 2024
@strantalis strantalis merged commit 8a6b876 into main Feb 5, 2024
5 of 6 checks passed
@strantalis strantalis deleted the policy-config-changes branch February 5, 2024 01:23
github-merge-queue bot pushed a commit that referenced this pull request Apr 22, 2024
🤖 I have created a release *beep* *boop*
---


##
[0.1.0](sdk-v0.1.0...sdk/v0.1.0)
(2024-04-22)


### Features

* add structured schema policy config
([#51](#51))
([8a6b876](8a6b876))
* **auth:** add authorization via casbin
([#417](#417))
([292f2bd](292f2bd))
* in-process service to service communication
([#311](#311))
([ec5eb76](ec5eb76))
* **kas:** support HSM and standard crypto
([#497](#497))
([f0cbe03](f0cbe03))
* key access server assignments
([#111](#111))
([a48d686](a48d686)),
closes [#117](#117)
* key access server registry impl
([#66](#66))
([cf6b3c6](cf6b3c6))
* **namespaces CRUD:** protos, generated SDK, db interactivity for
namespaces table ([#54](#54))
([b3f32b1](b3f32b1))
* **PLAT-3112:** Initial consumption of ec_key_pair functions by nanotdf
([#586](#586))
([5e2cba0](5e2cba0))
* **policy:** add FQN pivot table
([#208](#208))
([abb734c](abb734c))
* **policy:** add soft-delete/deactivation to namespaces, attribute
definitions, attribute values
[#96](#96)
[#108](#108)
([#191](#191))
([02e92a6](02e92a6))
* **resourcemapping:** resource mapping implementation
([#83](#83))
([c144db1](c144db1))
* **sdk:** BACK-1966 get auth wired up to SDK using `Options`
([#271](#271))
([f1bacab](f1bacab))
* **sdk:** BACK-1966 implement fetching a DPoP token
([#45](#45))
([dbd3cf9](dbd3cf9))
* **sdk:** BACK-1966 make the unwrapper retrieve public keys as well
([#260](#260))
([7d051a1](7d051a1))
* **sdk:** BACK-1966 pull rewrap into auth config
([#252](#252))
([84017aa](84017aa))
* **sdk:** Include auth token in grpc
([#367](#367))
([75cb5cd](75cb5cd))
* **sdk:** normalize token exchange
([#546](#546))
([9059dff](9059dff))
* **sdk:** Pass dpop key through to `rewrap`
([#435](#435))
([2d283de](2d283de))
* **sdk:** read `expires_in` from token response and use it to refresh
access tokens ([#445](#445))
([8ecbe79](8ecbe79))
* **sdk:** sdk stub
([#10](#10))
([8dfca6a](8dfca6a))
* **sdk:** take a function so that callers can use this the way that
they want ([#340](#340))
([72059cb](72059cb))
* **subject-mappings:** refactor to meet db schema
([#59](#59))
([59a073b](59a073b))
* **tdf:** implement tdf3 encrypt and decrypt
([#73](#73))
([9d0e0a0](9d0e0a0))
* **tdf:** sdk interface changes
([#123](#123))
([2aa2422](2aa2422))
* **tdf:** sdk interface cleanup
([#201](#201))
([6f7d815](6f7d815))
* **tdf:** TDFOption varargs interface
([#235](#235))
([b3fb720](b3fb720))


### Bug Fixes

* **archive:** remove 10gb zip file test
([#373](#373))
([6548f55](6548f55))
* attribute missing rpc method for listing attribute values
([#69](#69))
([1b3a831](1b3a831))
* **attribute value:** fixes attribute value crud
([#86](#86))
([568df9c](568df9c))
* **issue 90:** remove duplicate attribute_id from attribute value
create/update, and consumes schema setup changes in namespaces that were
introduced for integration testing
([#100](#100))
([e0f6d07](e0f6d07))
* **issue-124:** SDK kas registry import name mismatch
([#125](#125))
([112638b](112638b)),
closes [#124](#124)
* **proto/acre:** fix resource encoding service typo
([#30](#30))
([fe709d2](fe709d2))
* remove padding when b64 encoding
([#437](#437))
([d40e94a](d40e94a))
* SDK Quickstart
([#628](#628))
([f27ab98](f27ab98))
* **sdk:** change unwrapper creation
([#346](#346))
([9206435](9206435))
* **sdk:** double bearer token in auth config
([#350](#350))
([1bf4699](1bf4699))
* **sdk:** fixes Manifests JSONs with OIDC
([#140](#140))
([a4b6937](a4b6937))
* **sdk:** handle err
([#548](#548))
([ebabb6c](ebabb6c))
* **sdk:** make KasInfo fields public
([#320](#320))
([9a70498](9a70498))
* **sdk:** shutdown conn
([#352](#352))
([3def038](3def038))
* **sdk:** temporarily move unwrapper creation into options func.
([#309](#309))
([b34c2fe](b34c2fe))
* **sdk:** use the dialoptions even with no client credentials
([#400](#400))
([a7f1908](a7f1908))
* **security:** add a new encryption keypair different from dpop keypair
([#461](#461))
([7deb51e](7deb51e))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: opentdf-automation[bot] <149537512+opentdf-automation[bot]@users.noreply.github.com>
tech-guru42 added a commit to tech-guru42/TDF that referenced this pull request Jun 3, 2024
🤖 I have created a release *beep* *boop*
---


##
[0.1.0](opentdf/platform@sdk-v0.1.0...sdk/v0.1.0)
(2024-04-22)


### Features

* add structured schema policy config
([#51](opentdf/platform#51))
([8a6b876](opentdf/platform@8a6b876))
* **auth:** add authorization via casbin
([#417](opentdf/platform#417))
([292f2bd](opentdf/platform@292f2bd))
* in-process service to service communication
([#311](opentdf/platform#311))
([ec5eb76](opentdf/platform@ec5eb76))
* **kas:** support HSM and standard crypto
([#497](opentdf/platform#497))
([f0cbe03](opentdf/platform@f0cbe03))
* key access server assignments
([#111](opentdf/platform#111))
([a48d686](opentdf/platform@a48d686)),
closes [#117](opentdf/platform#117)
* key access server registry impl
([#66](opentdf/platform#66))
([cf6b3c6](opentdf/platform@cf6b3c6))
* **namespaces CRUD:** protos, generated SDK, db interactivity for
namespaces table ([#54](opentdf/platform#54))
([b3f32b1](opentdf/platform@b3f32b1))
* **PLAT-3112:** Initial consumption of ec_key_pair functions by nanotdf
([#586](opentdf/platform#586))
([5e2cba0](opentdf/platform@5e2cba0))
* **policy:** add FQN pivot table
([#208](opentdf/platform#208))
([abb734c](opentdf/platform@abb734c))
* **policy:** add soft-delete/deactivation to namespaces, attribute
definitions, attribute values
[#96](opentdf/platform#96)
[#108](opentdf/platform#108)
([#191](opentdf/platform#191))
([02e92a6](opentdf/platform@02e92a6))
* **resourcemapping:** resource mapping implementation
([#83](opentdf/platform#83))
([c144db1](opentdf/platform@c144db1))
* **sdk:** BACK-1966 get auth wired up to SDK using `Options`
([#271](opentdf/platform#271))
([f1bacab](opentdf/platform@f1bacab))
* **sdk:** BACK-1966 implement fetching a DPoP token
([#45](opentdf/platform#45))
([dbd3cf9](opentdf/platform@dbd3cf9))
* **sdk:** BACK-1966 make the unwrapper retrieve public keys as well
([#260](opentdf/platform#260))
([7d051a1](opentdf/platform@7d051a1))
* **sdk:** BACK-1966 pull rewrap into auth config
([#252](opentdf/platform#252))
([84017aa](opentdf/platform@84017aa))
* **sdk:** Include auth token in grpc
([#367](opentdf/platform#367))
([75cb5cd](opentdf/platform@75cb5cd))
* **sdk:** normalize token exchange
([#546](opentdf/platform#546))
([9059dff](opentdf/platform@9059dff))
* **sdk:** Pass dpop key through to `rewrap`
([#435](opentdf/platform#435))
([2d283de](opentdf/platform@2d283de))
* **sdk:** read `expires_in` from token response and use it to refresh
access tokens ([#445](opentdf/platform#445))
([8ecbe79](opentdf/platform@8ecbe79))
* **sdk:** sdk stub
([#10](opentdf/platform#10))
([8dfca6a](opentdf/platform@8dfca6a))
* **sdk:** take a function so that callers can use this the way that
they want ([#340](opentdf/platform#340))
([72059cb](opentdf/platform@72059cb))
* **subject-mappings:** refactor to meet db schema
([#59](opentdf/platform#59))
([59a073b](opentdf/platform@59a073b))
* **tdf:** implement tdf3 encrypt and decrypt
([#73](opentdf/platform#73))
([9d0e0a0](opentdf/platform@9d0e0a0))
* **tdf:** sdk interface changes
([#123](opentdf/platform#123))
([2aa2422](opentdf/platform@2aa2422))
* **tdf:** sdk interface cleanup
([#201](opentdf/platform#201))
([6f7d815](opentdf/platform@6f7d815))
* **tdf:** TDFOption varargs interface
([#235](opentdf/platform#235))
([b3fb720](opentdf/platform@b3fb720))


### Bug Fixes

* **archive:** remove 10gb zip file test
([#373](opentdf/platform#373))
([6548f55](opentdf/platform@6548f55))
* attribute missing rpc method for listing attribute values
([#69](opentdf/platform#69))
([1b3a831](opentdf/platform@1b3a831))
* **attribute value:** fixes attribute value crud
([#86](opentdf/platform#86))
([568df9c](opentdf/platform@568df9c))
* **issue 90:** remove duplicate attribute_id from attribute value
create/update, and consumes schema setup changes in namespaces that were
introduced for integration testing
([#100](opentdf/platform#100))
([e0f6d07](opentdf/platform@e0f6d07))
* **issue-124:** SDK kas registry import name mismatch
([#125](opentdf/platform#125))
([112638b](opentdf/platform@112638b)),
closes [#124](opentdf/platform#124)
* **proto/acre:** fix resource encoding service typo
([#30](opentdf/platform#30))
([fe709d2](opentdf/platform@fe709d2))
* remove padding when b64 encoding
([#437](opentdf/platform#437))
([d40e94a](opentdf/platform@d40e94a))
* SDK Quickstart
([#628](opentdf/platform#628))
([f27ab98](opentdf/platform@f27ab98))
* **sdk:** change unwrapper creation
([#346](opentdf/platform#346))
([9206435](opentdf/platform@9206435))
* **sdk:** double bearer token in auth config
([#350](opentdf/platform#350))
([1bf4699](opentdf/platform@1bf4699))
* **sdk:** fixes Manifests JSONs with OIDC
([#140](opentdf/platform#140))
([a4b6937](opentdf/platform@a4b6937))
* **sdk:** handle err
([#548](opentdf/platform#548))
([ebabb6c](opentdf/platform@ebabb6c))
* **sdk:** make KasInfo fields public
([#320](opentdf/platform#320))
([9a70498](opentdf/platform@9a70498))
* **sdk:** shutdown conn
([#352](opentdf/platform#352))
([3def038](opentdf/platform@3def038))
* **sdk:** temporarily move unwrapper creation into options func.
([#309](opentdf/platform#309))
([b34c2fe](opentdf/platform@b34c2fe))
* **sdk:** use the dialoptions even with no client credentials
([#400](opentdf/platform#400))
([a7f1908](opentdf/platform@a7f1908))
* **security:** add a new encryption keypair different from dpop keypair
([#461](opentdf/platform#461))
([7deb51e](opentdf/platform@7deb51e))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: opentdf-automation[bot] <149537512+opentdf-automation[bot]@users.noreply.github.com>
passion-127 added a commit to passion-127/TDF that referenced this pull request Jun 6, 2024
🤖 I have created a release *beep* *boop*
---


##
[0.1.0](opentdf/platform@sdk-v0.1.0...sdk/v0.1.0)
(2024-04-22)


### Features

* add structured schema policy config
([#51](opentdf/platform#51))
([8a6b876](opentdf/platform@8a6b876))
* **auth:** add authorization via casbin
([#417](opentdf/platform#417))
([292f2bd](opentdf/platform@292f2bd))
* in-process service to service communication
([#311](opentdf/platform#311))
([ec5eb76](opentdf/platform@ec5eb76))
* **kas:** support HSM and standard crypto
([#497](opentdf/platform#497))
([f0cbe03](opentdf/platform@f0cbe03))
* key access server assignments
([#111](opentdf/platform#111))
([a48d686](opentdf/platform@a48d686)),
closes [#117](opentdf/platform#117)
* key access server registry impl
([#66](opentdf/platform#66))
([cf6b3c6](opentdf/platform@cf6b3c6))
* **namespaces CRUD:** protos, generated SDK, db interactivity for
namespaces table ([#54](opentdf/platform#54))
([b3f32b1](opentdf/platform@b3f32b1))
* **PLAT-3112:** Initial consumption of ec_key_pair functions by nanotdf
([#586](opentdf/platform#586))
([5e2cba0](opentdf/platform@5e2cba0))
* **policy:** add FQN pivot table
([#208](opentdf/platform#208))
([abb734c](opentdf/platform@abb734c))
* **policy:** add soft-delete/deactivation to namespaces, attribute
definitions, attribute values
[#96](opentdf/platform#96)
[#108](opentdf/platform#108)
([#191](opentdf/platform#191))
([02e92a6](opentdf/platform@02e92a6))
* **resourcemapping:** resource mapping implementation
([#83](opentdf/platform#83))
([c144db1](opentdf/platform@c144db1))
* **sdk:** BACK-1966 get auth wired up to SDK using `Options`
([#271](opentdf/platform#271))
([f1bacab](opentdf/platform@f1bacab))
* **sdk:** BACK-1966 implement fetching a DPoP token
([#45](opentdf/platform#45))
([dbd3cf9](opentdf/platform@dbd3cf9))
* **sdk:** BACK-1966 make the unwrapper retrieve public keys as well
([#260](opentdf/platform#260))
([7d051a1](opentdf/platform@7d051a1))
* **sdk:** BACK-1966 pull rewrap into auth config
([#252](opentdf/platform#252))
([84017aa](opentdf/platform@84017aa))
* **sdk:** Include auth token in grpc
([#367](opentdf/platform#367))
([75cb5cd](opentdf/platform@75cb5cd))
* **sdk:** normalize token exchange
([#546](opentdf/platform#546))
([9059dff](opentdf/platform@9059dff))
* **sdk:** Pass dpop key through to `rewrap`
([#435](opentdf/platform#435))
([2d283de](opentdf/platform@2d283de))
* **sdk:** read `expires_in` from token response and use it to refresh
access tokens ([#445](opentdf/platform#445))
([8ecbe79](opentdf/platform@8ecbe79))
* **sdk:** sdk stub
([#10](opentdf/platform#10))
([8dfca6a](opentdf/platform@8dfca6a))
* **sdk:** take a function so that callers can use this the way that
they want ([#340](opentdf/platform#340))
([72059cb](opentdf/platform@72059cb))
* **subject-mappings:** refactor to meet db schema
([#59](opentdf/platform#59))
([59a073b](opentdf/platform@59a073b))
* **tdf:** implement tdf3 encrypt and decrypt
([#73](opentdf/platform#73))
([9d0e0a0](opentdf/platform@9d0e0a0))
* **tdf:** sdk interface changes
([#123](opentdf/platform#123))
([2aa2422](opentdf/platform@2aa2422))
* **tdf:** sdk interface cleanup
([#201](opentdf/platform#201))
([6f7d815](opentdf/platform@6f7d815))
* **tdf:** TDFOption varargs interface
([#235](opentdf/platform#235))
([b3fb720](opentdf/platform@b3fb720))


### Bug Fixes

* **archive:** remove 10gb zip file test
([#373](opentdf/platform#373))
([6548f55](opentdf/platform@6548f55))
* attribute missing rpc method for listing attribute values
([#69](opentdf/platform#69))
([1b3a831](opentdf/platform@1b3a831))
* **attribute value:** fixes attribute value crud
([#86](opentdf/platform#86))
([568df9c](opentdf/platform@568df9c))
* **issue 90:** remove duplicate attribute_id from attribute value
create/update, and consumes schema setup changes in namespaces that were
introduced for integration testing
([#100](opentdf/platform#100))
([e0f6d07](opentdf/platform@e0f6d07))
* **issue-124:** SDK kas registry import name mismatch
([#125](opentdf/platform#125))
([112638b](opentdf/platform@112638b)),
closes [#124](opentdf/platform#124)
* **proto/acre:** fix resource encoding service typo
([#30](opentdf/platform#30))
([fe709d2](opentdf/platform@fe709d2))
* remove padding when b64 encoding
([#437](opentdf/platform#437))
([d40e94a](opentdf/platform@d40e94a))
* SDK Quickstart
([#628](opentdf/platform#628))
([f27ab98](opentdf/platform@f27ab98))
* **sdk:** change unwrapper creation
([#346](opentdf/platform#346))
([9206435](opentdf/platform@9206435))
* **sdk:** double bearer token in auth config
([#350](opentdf/platform#350))
([1bf4699](opentdf/platform@1bf4699))
* **sdk:** fixes Manifests JSONs with OIDC
([#140](opentdf/platform#140))
([a4b6937](opentdf/platform@a4b6937))
* **sdk:** handle err
([#548](opentdf/platform#548))
([ebabb6c](opentdf/platform@ebabb6c))
* **sdk:** make KasInfo fields public
([#320](opentdf/platform#320))
([9a70498](opentdf/platform@9a70498))
* **sdk:** shutdown conn
([#352](opentdf/platform#352))
([3def038](opentdf/platform@3def038))
* **sdk:** temporarily move unwrapper creation into options func.
([#309](opentdf/platform#309))
([b34c2fe](opentdf/platform@b34c2fe))
* **sdk:** use the dialoptions even with no client credentials
([#400](opentdf/platform#400))
([a7f1908](opentdf/platform@a7f1908))
* **security:** add a new encryption keypair different from dpop keypair
([#461](opentdf/platform#461))
([7deb51e](opentdf/platform@7deb51e))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: opentdf-automation[bot] <149537512+opentdf-automation[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants