Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

server: Deprecate the watch and partial query parameters #2614

Merged
merged 2 commits into from
Aug 10, 2020
Merged

server: Deprecate the watch and partial query parameters #2614

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
2a1fe50
server: Deprecate the watch and partial query parameters
tsandall Aug 7, 2020
c080905
Merge branch 'master' into deprecate-watch-and-partial-params
tsandall Aug 10, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,12 @@ $ opa build ./policies/example.rego --capabilities ./capabilities/v0.22.0.json
$ opa build ./policies/example.rego --capabilities ./capabilities/v0.21.1.json
```

### Deprecations

* The `watch` query parameter on the Data API has been deprecated. The query watch feature was unused and the lack of incremental evaluation would have introduced scalability issues for users. The feature will be removed in a future release.

* The `partial` query parameter on the Data API has been deprecated. Note, this only applies to the `partial` query parameter that the Data API supports, not Partial Evaluation itself. The `partial` parameter allowed users to lazily trigger Partial Evaluation (for optimization purposes) during a policy query. While this is useful for kicking the tires in a development environment, putting optimization into the policy query path is not recommended. If users want to kick the tires with Partial Evaluation, we recommend running the `opa build` command.

## 0.22.0

### Bundle Signing
Expand Down
264 changes: 0 additions & 264 deletions docs/content/rest-api.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -730,7 +730,6 @@ The path separator is used to access values inside object and array documents. I
- **explain** - Return query explanation in addition to result. Values: **full**.
- **metrics** - Return query performance metrics in addition to result. See [Performance Metrics](#performance-metrics) for more detail.
- **instrument** - Instrument query evaluation and return a superset of performance metrics in addition to result. See [Performance Metrics](#performance-metrics) for more detail.
- **watch** - Set a watch on the data reference if the parameter is present. See [Watches](#watches) for more detail.

#### Status Codes

Expand Down Expand Up @@ -801,221 +800,6 @@ Content-Type: application/json
}
```

#### Example Watch Request

If we make the following GET request:

```http
GET /v1/data/servers?watch&pretty=true HTTP/1.1
```

Followed by these PATCH requests:

```http
PATCH /v1/data/servers HTTP/1.1
Content-Type: application/json-patch+json
```

```json
[
{"op": "add",
"path": "-",
"value": {
"id": "s5",
"name": "job",
"protocols": ["amqp"],
"ports": ["p3"]
}}
]
```

```http
PATCH /v1/data/servers HTTP/1.1
Content-Type: application/json-patch+json
```

```json
[
{
"op": "remove",
"path": "1"
}
]
```

#### Example Watch Response

The response below represents the response _after_ the chunked encoding has been processed.
It is not complete, as further changes to `/data/servers` would cause more notifications to be streamed.

```http
HTTP/1.1 200 OK
Content-Type: application/json
Transfer-Encoding: chunked
```

```json
{
"result": [
{
"id": "s1",
"name": "app",
"ports": [
"p1",
"p2",
"p3"
],
"protocols": [
"https",
"ssh"
]
},
{
"id": "s2",
"name": "db",
"ports": [
"p3"
],
"protocols": [
"mysql"
]
},
{
"id": "s3",
"name": "cache",
"ports": [
"p3"
],
"protocols": [
"memcache",
"http"
]
},
{
"id": "s4",
"name": "dev",
"ports": [
"p1",
"p2"
],
"protocols": [
"http"
]
}
]
}

{
"result": [
{
"id": "s1",
"name": "app",
"ports": [
"p1",
"p2",
"p3"
],
"protocols": [
"https",
"ssh"
]
},
{
"id": "s2",
"name": "db",
"ports": [
"p3"
],
"protocols": [
"mysql"
]
},
{
"id": "s3",
"name": "cache",
"ports": [
"p3"
],
"protocols": [
"memcache",
"http"
]
},
{
"id": "s4",
"name": "dev",
"ports": [
"p1",
"p2"
],
"protocols": [
"http"
]
},
{
"id": "s5",
"name": "job",
"ports": [
"p3"
],
"protocols": [
"amqp"
]
}
]
}

{
"result": [
{
"id": "s1",
"name": "app",
"ports": [
"p1",
"p2",
"p3"
],
"protocols": [
"https",
"ssh"
]
},
{
"id": "s3",
"name": "cache",
"ports": [
"p3"
],
"protocols": [
"memcache",
"http"
]
},
{
"id": "s4",
"name": "dev",
"ports": [
"p1",
"p2"
],
"protocols": [
"http"
]
},
{
"id": "s5",
"name": "job",
"ports": [
"p3"
],
"protocols": [
"amqp"
]
}
]
}
```

### Get a Document (with Input)

```
Expand All @@ -1041,13 +825,11 @@ The request body contains an object that specifies a value for [The input Docume

#### Query Parameters

- **partial** - Use the partial evaluation (optimization) when evaluating the query.
- **pretty** - If parameter is `true`, response will formatted for humans.
- **provenance** - If parameter is `true`, response will include build/version info in addition to the result. See [Provenance](#provenance) for more detail.
- **explain** - Return query explanation in addition to result. Values: **full**.
- **metrics** - Return query performance metrics in addition to result. See [Performance Metrics](#performance-metrics) for more detail.
- **instrument** - Instrument query evaluation and return a superset of performance metrics in addition to result. See [Performance Metrics](#performance-metrics) for more detail.
- **watch** - Set a watch on the data reference if the parameter is present. See [Watches](#watches) for more detail.

#### Status Codes

Expand Down Expand Up @@ -1426,13 +1208,11 @@ GET /v1/query
- **pretty** - If parameter is `true`, response will formatted for humans.
- **explain** - Return query explanation in addition to result. Values: **full**.
- **metrics** - Return query performance metrics in addition to result. See [Performance Metrics](#performance-metrics) for more detail.
- **watch** - Set a watch on the query if the parameter is present. See [Watches](#watches) for more detail.

#### Status Codes

- **200** - no error
- **400** - bad request
- **404** - watch ID not found
- **500** - server error
- **501** - streaming not implemented

Expand Down Expand Up @@ -1994,50 +1774,6 @@ OPA currently supports the following query provenance information:
the `revision` field which is the _revision_ string included in a .manifest file (if present)
within a bundle

## Watches

OPA can set watches on queries and report whenever the result of evaluating the query has changed. When a watch is set on a query, the requesting connection will be maintained as the query results are streamed back in HTTP Chunked Encoding format. A notification reflecting a certain change to the query results will be delivered _at most once_. That is, if a watch is set on `data.x`, and then multiple writes are made to `data.x`, say `1`, `2` and `3`, only the notification reflecting `data.x=3` is always seen eventually (assuming the watch is not ended, there are no connection problems, etc). The notifications reflecting `data.x=1` and `data.x=2` _might_ be seen. However, the notifications sent are guaranteed to be in order (`data.x=2` will always come after `data.x=1`, if it comes).

The notification stream will not be delimited by any value; the client is expected to be able to parse JSON values from the stream one by one, recognizing where one ends and the next begins.

The notification stream is a stream of JSON objects, each of which has the following structure:
```
{
"result": <result>,
"error": <error>,
"metrics": <metrics>,
"explanation": <explanation>,
}
```

The `error` field is optional; it is omitted when no errors occur. If it is present, it is an [Error](#errors) describing any errors encountered while evaluating the query the watch was set on. If the policies on the server are changed such that the query no longer compiles, the contents of the error's `message` field will start with the text "watch invalidated:" and will be followed with the reason for invalidation. The watch will be ended by the server and the stream closed.

The `metrics` field represents [Performance Metrics](#performance-metrics) for the evaluation of the query. It will only be present if metrics were requested in the API call that started the watch.

The `explanation` field represents an [Explanation](#explanations) of how the query answer was found. It will only be present if explanations were requested in the API call that started the watch.

If there are no errors, the `result` field will be a JSON array of the results of evaluating the query. The format of a result is:
```
{
"expressions": [
{
"value": true,
"text": "a = data.x",
"location":{"row":1,"col":1}
},
...
],
"bindings": {...}
}
```

The `expressions` field is an array of the results of evaluating each of the expressions in the query. `value` is the expression's value, `text` is the actual expression, and `location` describes the location of the expression within the query. The values above are examples.

The `bindings` field is a JSON object mapping `string`s to JSON values that describe the variable bindings resulting from evaluating the query.

If the watch was set on a data reference instead of a query, the `result` field will simply be the value of the document requested, instead of an array of values.


## Health API

The `/health` API endpoint executes a simple built-in policy query to verify
Expand Down
10 changes: 10 additions & 0 deletions runtime/logging.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,16 @@ func (h *LoggingHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
}
}

params := r.URL.Query()

if _, ok := params["watch"]; ok {
logrus.Warn("Deprecated 'watch' parameter specified in request. See https://github.com/open-policy-agent/opa/releases/tag/v0.23.0 for details.")
}

if _, ok := params["partial"]; ok {
logrus.Warn("Deprecated 'partial' parameter specified in request. See https://github.com/open-policy-agent/opa/releases/tag/v0.23.0 for details.")
}

h.inner.ServeHTTP(recorder, r)

dt := time.Since(t0)
Expand Down
3 changes: 2 additions & 1 deletion server/types/types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -413,7 +413,7 @@ const (

// ParamPartialV1 defines the name of the HTTP URL parameter that indicates
// the client wants the partial evaluation optimization to be used during
// query evaluation.
// query evaluation. This parameter is DEPRECATED.
ParamPartialV1 = "partial"

// ParamProvenanceV1 defines the name of the HTTP URL parameter that indicates
Expand All @@ -422,6 +422,7 @@ const (

// ParamWatchV1 defines the name of the HTTP URL parameter that indicates
// the client wants to set a watch on the current query or data reference.
// This parameter is DEPRECATED.
ParamWatchV1 = "watch"

// ParamBundleActivationV1 defines the name of the HTTP URL parameter that
Expand Down
8 changes: 0 additions & 8 deletions watch/doc.go
View file
Open in desktop

This file was deleted.