diff --git a/proto/LICENSE b/proto/LICENSE new file mode 100644 index 0000000..e454a52 --- /dev/null +++ b/proto/LICENSE @@ -0,0 +1,178 @@ + + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + diff --git a/proto/README.md b/proto/README.md new file mode 100644 index 0000000..b72dec8 --- /dev/null +++ b/proto/README.md @@ -0,0 +1,27 @@ +# Styra Load gRPC API + +This folder contains the definitions of [Protocol Buffers][protobuf] used by [Styra Load][gh-styra-load]. + +We use [Buf][buf] to manage and generate source code from the protocol buffer definitions. +The protobuf definitions here are a mirror of what we have pushed to the [`buf.build/styra/load`][buf-styra-load] repository in the Buf Registry. + + [protobuf]: https://developers.google.com/protocol-buffers/ + [buf]: https://github.com/bufbuild/buf + [gh-styra-load]: https://github.com/StyraInc/load + [buf-styra-load]: https://buf.build/styra/load + +## Build + +Running `buf generate` in this folder (or `./buf.gen.yaml` if you're on a Linux system) should create the necessary Golang files under a folder named `gen/`. + +For supporting other languages, you will need to modify the `buf.gen.yaml` file to add the appropriate generation arguments for your language of choice. + +## Linting + +To lint the protobuf files, try running `buf lint` in this folder. + +## License + +The contents of this folder (Buf files and protobuf definitions) are released under the Apache v2.0 license][apache-v2] license. + + [apache-v2]: https://www.apache.org/licenses/LICENSE-2.0.html diff --git a/proto/buf.gen.yaml b/proto/buf.gen.yaml new file mode 100755 index 0000000..25b1506 --- /dev/null +++ b/proto/buf.gen.yaml @@ -0,0 +1,15 @@ +#!/usr/bin/env -S buf generate --template +--- +version: v1 +managed: + enabled: true + go_package_prefix: + default: github.com/styrainc/load/proto/gen/go +plugins: + - plugin: go + out: gen/go + opt: paths=source_relative + - plugin: go-grpc + out: gen/go + opt: + - paths=source_relative diff --git a/proto/buf.yaml b/proto/buf.yaml new file mode 100644 index 0000000..4872812 --- /dev/null +++ b/proto/buf.yaml @@ -0,0 +1,8 @@ +version: v1 +name: buf.build/styra/load +breaking: + use: + - FILE +lint: + use: + - DEFAULT diff --git a/proto/load/bulk/v1/bulk.proto b/proto/load/bulk/v1/bulk.proto new file mode 100644 index 0000000..63cba2e --- /dev/null +++ b/proto/load/bulk/v1/bulk.proto @@ -0,0 +1,136 @@ +// Copyright 2023 Styra, Inc. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// https://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +syntax = "proto3"; +package load.bulk.v1; + +import "load/data/v1/data.proto"; +import "load/policy/v1/policy.proto"; +import "google/protobuf/any.proto"; + +// BulkRWRequest can contain any combination of read/write operations on +// policies and data. +message BulkRWRequest { + // WritePolicyRequest provides a union of possible Policy request types. + // This allows creating arbitrary sequences of policy store operations. + message WritePolicyRequest { + oneof req { + load.policy.v1.CreatePolicyRequest create = 1; + load.policy.v1.UpdatePolicyRequest update = 2; + load.policy.v1.DeletePolicyRequest delete = 3; + } + } + + // WriteDataRequest provides a union of possible Data request types. + // This allows creating arbitrary sequences of data store operations. + message WriteDataRequest { + oneof req { + load.data.v1.CreateDataRequest create = 1; + load.data.v1.UpdateDataRequest update = 2; + load.data.v1.DeleteDataRequest delete = 3; + } + } + + // ReadPolicyRequest is currently a simple wrapper over the GetPolicy + // request type. + message ReadPolicyRequest { + load.policy.v1.GetPolicyRequest req = 1; + } + + // ReadDataRequest is currently a simple wrapper over the GetData + // request type. + message ReadDataRequest { + load.data.v1.GetDataRequest req = 1; + } + + // writes_policy provides a sequence of WritePolicy operations to apply. + repeated WritePolicyRequest writes_policy = 2; + // writes_data provides a sequence of WriteData operations to apply. + repeated WriteDataRequest writes_data = 3; + + // reads_policy provides a sequence of ReadPolicy operations to apply. + repeated ReadPolicyRequest reads_policy = 4; + // reads_data provides a sequence of ReadData operations to apply. + repeated ReadDataRequest reads_data = 5; +} + +// BulkRWResponse contains lists of the appropriate response types for each +// operation in the BulkRWRequest. +message BulkRWResponse { + // WritePolicyResponse provides a union of possible response types, + // mirroring the union of possible request types. + message WritePolicyResponse { + oneof resp { + load.policy.v1.CreatePolicyResponse create = 1; + load.policy.v1.UpdatePolicyResponse update = 2; + load.policy.v1.DeletePolicyResponse delete = 3; + } + } + + // WriteDataResponse provides a union of possible response types, + // mirroring the union of possible request types. + message WriteDataResponse { + oneof resp { + load.data.v1.CreateDataResponse create = 1; + load.data.v1.UpdateDataResponse update = 2; + load.data.v1.DeleteDataResponse delete = 3; + } + } + + // ReadPolicyResponse provides fields for a response or list of errors. + // The two should be mutually exclusive. + message ReadPolicyResponse { + load.policy.v1.GetPolicyResponse resp = 1; + ErrorList errors = 2; + } + + // ReadDataResponse provides fields for a response or list of errors. The + // two should be mutually exclusive. + message ReadDataResponse { + load.data.v1.GetDataResponse resp = 1; + ErrorList errors = 2; + } + + // writes_policy provides a sequence of WritePolicy results. + repeated WritePolicyResponse writes_policy = 2; + // writes_data provides a sequence of WriteData results. + repeated WriteDataResponse writes_data = 3; + + // reads_policy provides a sequence of ReadPolicy results or errors. + repeated ReadPolicyResponse reads_policy = 4; + // reads_data provides a sequence of ReadData results or errors. + repeated ReadDataResponse reads_data = 5; +} + +// Context-dependent error messages. +message ErrorList { + // The errors in the list. + repeated google.protobuf.Any errors = 1; +} + +// BulkService is an API for specifying batches of read/write operations. +service BulkService { + // BulkRW specifies a fixed-structure, bulk read/write operation. + // + // WritePolicy and WriteData operations are executed sequentially, aborting + // the entire gRPC call if any operations fail. + // + // The ReadPolicy and ReadData operations are then executed in parallel, + // but will report errors inline in their responses, instead of aborting + // the entire gRPC call. + // + // Warning: The same performance hazards described for the Policy API + // apply for PolicyWrite operations here as well. + rpc BulkRW(BulkRWRequest) returns (BulkRWResponse); +} diff --git a/proto/load/data/v1/data.proto b/proto/load/data/v1/data.proto new file mode 100644 index 0000000..9d01a07 --- /dev/null +++ b/proto/load/data/v1/data.proto @@ -0,0 +1,132 @@ +// Copyright 2023 Styra, Inc. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// https://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +syntax = "proto3"; +package load.data.v1; + +import "google/protobuf/struct.proto"; + +// DataDocument is a thin wrapper type around a JSON value located at a +// particular path in the virtual document store. +message DataDocument { + string path = 1; + google.protobuf.Value document = 2; // json value, like {"foo":true} or [1, 2, 3] +} + +// InputDocument is a thin wrapper type around OPA inputs, which are always +// JSON object types. +message InputDocument { + google.protobuf.Struct document = 1; // json object, like {"foo":true} +} + +// PatchOp allows specifying a subset of JSON-Patch operations, namely +// "add", "remove", and "replace". +enum PatchOp { + PATCH_OP_UNSPECIFIED = 0; + PATCH_OP_ADD = 1; + PATCH_OP_REMOVE = 2; + PATCH_OP_REPLACE = 3; +} + +// CreateDataRequest provides a document and the path to insert it at. +message CreateDataRequest { + DataDocument data = 1; +} + +// CreateDataResponse is an empty confirmation message type. +message CreateDataResponse { +} + +// GetDataRequest queries the virtual document store at the specified path. +// This path can target a rule, or any value under the `data` document. +// +// Input to a query can also be provided, as a key-value mapping that will +// appear under the `input` document at runtime. For example, the following +// input mapping: +// ```json +// { +// "a": 2, +// "b": "example", +// "c": true +// } +// ```` +// would map to `input.a`, `input.b`, and `input.c`, where `input.a` has +// the value `2`, and so forth in a Rego policy. +message GetDataRequest { + string path = 1; + InputDocument input = 2; +} + +// GetDataResponse is the query result returned from a GetData operation. +message GetDataResponse { + DataDocument result = 1; +} + +// UpdateDataRequest provides an optional document to patch in, and the +// patch operation specifying whether it's a create/update/delete operation +// to be performed. +message UpdateDataRequest { + DataDocument data = 1; // JSON value to be patched in, like {"foo":true} + PatchOp op = 2; // If unspecified, defaults to "overwrite/replace". +} + +// UpdateDataResponse is an empty confirmation message type. +message UpdateDataResponse { +} + +// DeleteDataRequest provides the path of a document to delete from the +// document store. +message DeleteDataRequest { + string path = 1; +} + +// DeleteDataResponse is an empty confirmation message type. +message DeleteDataResponse { +} + +service DataService { + // CreateData looks up the document by path, and inserts a new JSON value + // at the end of the path. + // + // This is equivalent in functionality to OPA's + // [Data REST API Create/Overwrite method](https://www.openpolicyagent.org/docs/latest/rest-api/#create-or-overwrite-a-document). + rpc GetData(GetDataRequest) returns (GetDataResponse); + + // GetData looks up the document by path. This can be either a plain JSON + // value (a "Base" document in OPA parlance), or a rule head (a + // "Virtual"/computed document). + // + // This is equivalent in functionality to OPA's + // [Data REST API Get with Input method](https://www.openpolicyagent.org/docs/latest/rest-api/#get-a-document-with-input). + // + // Note that the input field should not be wrapped with + // `{ "input": }`, you can simply put the JSON serialized `` + // in the input field directly. + rpc CreateData(CreateDataRequest) returns (CreateDataResponse); + + // UpdateData looks up the document by path, and then attempts to perform + // one of three patching operations at that location: add, remove, or + // replace. + // + // This is roughly equivalent in functionality to OPA's + // [Data REST API Patch method](https://www.openpolicyagent.org/docs/latest/rest-api/#patch-a-document) + rpc UpdateData(UpdateDataRequest) returns (UpdateDataResponse); + + // DeleteData looks up the document by path, and then attempts to remove + // it from the store. + // + // This is equivalent in functionality to OPA's + // [Data REST API Delete method](https://www.openpolicyagent.org/docs/latest/rest-api/#delete-a-document). + rpc DeleteData(DeleteDataRequest) returns (DeleteDataResponse); +} diff --git a/proto/load/policy/v1/policy.proto b/proto/load/policy/v1/policy.proto new file mode 100644 index 0000000..687459a --- /dev/null +++ b/proto/load/policy/v1/policy.proto @@ -0,0 +1,127 @@ +// Copyright 2023 Styra, Inc. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// https://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +syntax = "proto3"; +package load.policy.v1; + +// Policy is a thin wrapper type around a Rego policy. +message Policy { + string path = 1; + string text = 2; +} + +// ListPoliciesRequest is used to start a ListPolicy operation. +message ListPoliciesRequest {} + +// ListPoliciesResponse contains the list of all policies enumerated by a +// ListPolicy operation. +message ListPoliciesResponse { + repeated Policy results = 1; +} + +// CreatePolicyRequest provides the policy and path to insert it at for a +// CreatePolicy operation. +message CreatePolicyRequest { + Policy policy = 1; // Rego module. +} + +// CreatePolicyResponse is an empty confirmation message type. +message CreatePolicyResponse {} + +// GetPolicyRequest requires the path of the policy to fetch. +message GetPolicyRequest { + string path = 1; +} + +// GetPolicyResponse contains the retrieved Rego module from a GetPolicy +// operation. +message GetPolicyResponse { + Policy result = 2; // Rego module. +} + +// UpdatePolicyRequest provides the policy and path to update for an +// UpdatePolicy operation. +message UpdatePolicyRequest { + Policy policy = 2; // Rego module. +} + +// UpdatePolicyResponse is an empty confirmation message type. +message UpdatePolicyResponse {} + +// DeletePolicyRequest specifies which path to delete a policy from. +message DeletePolicyRequest { + string path = 1; +} + +// DeletePolicyResponse is an empty confirmation message type. +message DeletePolicyResponse {} + +service PolicyService { + // ListPolicies returns the set of stored policies in the policy store. + // + // This is equivalent in functionality to OPA's + // [Policy REST API List method](https://www.openpolicyagent.org/docs/latest/rest-api/#list-policies). + // + // Warning: This request will enumerate *all* policies stored by the Load + // instance. This can have substantial overheads if the policies are large + // in size. + rpc ListPolicies(ListPoliciesRequest) returns (ListPoliciesResponse); + + // GetPolicy fetches a policy module's code from the policy store. + // + // This is roughly equivalent in functionality to OPA's + // [Policy REST API Get method](https://www.openpolicyagent.org/docs/latest/rest-api/#get-a-policy). + // + // Note: Only the raw policy text is returned in this version of the API. + rpc GetPolicy(GetPolicyRequest) returns (GetPolicyResponse); + + // CreatePolicy inserts a new policy module into the policy store. + // + // This is equivalent in functionality to OPA's + // [Policy REST API Create/Update method](https://www.openpolicyagent.org/docs/latest/rest-api/#create-or-update-a-policy). + // + // Warning: Inserting a new policy will trigger a full recompilation + // of *all* policies in the store, in order to ensure the new policy does + // not break existing policies. This is a known performance hazard for the + // OPA REST API, and remains a hazard for gRPC as well. Keeping the unique + // number of policies down, or using Bundles are the recommended + // workarounds for most OPA users. + rpc CreatePolicy(CreatePolicyRequest) returns (CreatePolicyResponse); + + // UpdatePolicy updates a policy module in the policy store. + // + // This is equivalent in functionality to OPA's + // [Policy REST API Create/Update method](https://www.openpolicyagent.org/docs/latest/rest-api/#create-or-update-a-policy). + // + // Warning: Modifying an existing policy will trigger a full recompilation + // of *all* policies in the store, in order to ensure the updated policy does + // not break existing policies. This is a known performance hazard for the + // OPA REST API, and remains a hazard for gRPC as well. Keeping the unique + // number of policies down, or using Bundles are the recommended + // workarounds for most OPA users. + rpc UpdatePolicy(UpdatePolicyRequest) returns (UpdatePolicyResponse); + + // DeletePolicy removes a policy module from the policy store. + // + // This is equivalent in functionality to OPA's + // [Policy REST API Delete method](https://www.openpolicyagent.org/docs/latest/rest-api/#delete-a-policy). + // + // Warning: Removing a policy will trigger a full recompilation of *all* + // policies in the store, in order to ensure that removing the policy + // module does not break existing policies. This is a known performance + // hazard for the OPA REST API, and remains a hazard for gRPC as well. + // Keeping the unique number of policies down, or using Bundles are the + // recommended workarounds for most OPA users. + rpc DeletePolicy(DeletePolicyRequest) returns (DeletePolicyResponse); +}