Skip to content

Commit

Permalink
Reworked logging in general and error handling for jobs. #214, #100
Browse files Browse the repository at this point in the history
  • Loading branch information
@m-mohr
m-mohr committed Oct 31, 2019
1 parent 7ec1550 commit 514a259
Show file tree
Hide file tree
Showing 3 changed files with 175 additions and 61 deletions.
4 changes: 4 additions & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- `GET /collections` and `GET /collections/{collectionId}`:
- New field `deprecated` can be used to indicate outdated collections. Links with relation type `successor-version` can point to a newer version. [#226]( https://github.com/Open-EO/openeo-api/issues/226)
- Added a Data Cube Dimension of type `bands` to the `cube:dimensions` property. [#208](https://github.com/Open-EO/openeo-api/issues/208)
- `POST /result`: May add a link to a log file in the header. [#214](https://github.com/Open-EO/openeo-api/issues/214)
- `GET /jobs/{job_id}/logs` and `GET /services/{service_id}/logs`: Endpoints that publish logging information. [#214](https://github.com/Open-EO/openeo-api/issues/214)

### Changed

Expand All @@ -38,9 +40,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Removed

- `GET /job/{job_id}`: Removed property `error`. Request information from `GET /job/{job_id}/logs` instead.
- `GET /job/{job_id}/results`:
- Metalink XML encoding has been removed. [#205](https://github.com/Open-EO/openeo-api/issues/205)
- `Expires` header has been removed, use `expires` property in the response body instead.
- The response body of status code 424 has been extended.
- `GET /credentials/basic` doesn't return a `user_id`. Instead request it from `GET /me`.
- `GET /collections/{collectionId}`: Removed optional STAC extensions from the API specification. Inform yourself about useful [STAC extensions](https://github.com/radiantearth/stac-spec/tree/master/extensions#list-of-content-extensions) instead. [#176](https://github.com/Open-EO/openeo-api/issues/176)

Expand Down
230 changes: 170 additions & 60 deletions openapi.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ info:
## HTTP / REST
This uses [HTTP REST](https://en.wikipedia.org/wiki/Representational_state_transfer) [Level 2](https://martinfowler.com/articles/richardsonMaturityModel.html#level2) for communication between client and back-end server.
This uses [HTTP REST](https://en.wikipedia.org/wiki/Representational_state_transfer) [Level 2](https://martinfowler.com/articles/richardsonMaturityModel.html#level2) for communication between client and back-end server.
Public APIs MUST be available via HTTPS only.
Expand Down Expand Up @@ -2587,6 +2587,13 @@ paths:
description: MAY include the costs for processing and downloading the data.
schema:
$ref: '#/components/schemas/money'
Link:
description: >-
The header MAY indicate a link to a log file generated by the request. If provided, the link MUST be serialized according to [RFC 8288](https://tools.ietf.org/html/rfc8288#section-3) and MUST use the relation type `monitor`. The link MUST follow the specifications for the links `GET /jobs/{job_id}/logs` and `GET /services/{service_id}/logs`, except that is MUST NOT accept any parameters (limit/offset). Therefore, the link MUST be accessible with HTTP GET, MUST be secured using a Bearer token and MUST follow the corresponding request body schema.
schema:
type: string
pattern: ^<[^>]+>;\s?rel="monitor"
example: <https://openeo.org/api/logs/258489231>; rel="monitor"
4XX:
$ref: '#/components/responses/client_error_auth'
5XX:
Expand Down Expand Up @@ -3144,6 +3151,25 @@ paths:
$ref: '#/components/responses/client_error_auth'
5XX:
$ref: '#/components/responses/server_error'
'/services/{service_id}/logs':
get:
summary: Logs for a secondary service
description: Logs for the secondary service.
tags:
- Secondary Services
security:
- Bearer: []
parameters:
- $ref: '#/components/parameters/service_id'
- $ref: '#/components/parameters/log_offset'
- $ref: '#/components/parameters/log_limit'
responses:
'200':
$ref: '#/components/responses/logs'
4XX:
$ref: '#/components/responses/client_error_auth'
5XX:
$ref: '#/components/responses/server_error'
/jobs:
get:
summary: List all batch jobs
Expand Down Expand Up @@ -3336,12 +3362,6 @@ paths:
minimum: 0
maximum: 100
example: 75.5
error:
oneOf:
- nullable: true
enum:
- null
- $ref: '#/components/schemas/job_error'
submitted:
$ref: '#/components/schemas/submitted'
updated:
Expand Down Expand Up @@ -3437,6 +3457,25 @@ paths:
$ref: '#/components/responses/client_error_auth'
5XX:
$ref: '#/components/responses/server_error'
'/jobs/{job_id}/logs':
get:
summary: Logs for a batch job
description: Logs for the batch job.
tags:
- Batch Jobs
security:
- Bearer: []
parameters:
- $ref: '#/components/parameters/job_id'
- $ref: '#/components/parameters/log_offset'
- $ref: '#/components/parameters/log_limit'
responses:
'200':
$ref: '#/components/responses/logs'
4XX:
$ref: '#/components/responses/client_error_auth'
5XX:
$ref: '#/components/responses/server_error'
'/jobs/{job_id}/results':
parameters:
- $ref: '#/components/parameters/job_id'
Expand Down Expand Up @@ -3492,16 +3531,14 @@ paths:
'424':
description: >-
The request can't be fulfilled as the batch job failed. This request
will deliver the error message that was produced by the batch job.
will deliver the last error message that was produced by the batch job.
This HTTP code MUST be sent only when the job `status` is `error`.
The response to this error mirrors the `error` field in the `GET
/jobs/{job_id}` response.
content:
application/json:
schema:
$ref: '#/components/schemas/job_error'
$ref: '#/components/schemas/log_entry'
4XX:
$ref: '#/components/responses/client_error_auth'
5XX:
Expand Down Expand Up @@ -5074,26 +5111,6 @@ components:
required: true
minimum: 0
maximum: 100
job_error:
title: Batch Job Error
description: >-
An error message that describes the problem during the batch job
execution. May only be available if the `status` is `error`. The error
MUST be cleared if the job is started again (i.e. the status changes to
`queue`).
type: object
required:
- code
- message
properties:
id:
$ref: '#/components/schemas/error_id'
code:
$ref: '#/components/schemas/error_code'
message:
$ref: '#/components/schemas/error_message'
links:
$ref: '#/components/schemas/error_links'
error:
title: General Error
description: >-
Expand All @@ -5108,42 +5125,32 @@ components:
- message
properties:
id:
$ref: '#/components/schemas/error_id'
type: string
description: >-
A back-end may add a unique identifier to the error response to be able
to log and track errors with further non-disclosable details. A client
could communicate this id to a back-end provider to get further
information.
example: 550e8400-e29b-11d4-a716-446655440000
code:
$ref: '#/components/schemas/error_code'
$ref: '#/components/schemas/log_code'
message:
$ref: '#/components/schemas/error_message'
type: string
description: >-
A message explaining what the client may need to change or what
difficulties the server is facing.
links:
$ref: '#/components/schemas/error_links'
error_id:
type: string
description: >-
A back-end may add a unique identifier to the error response to be able
to log and track errors with further non-disclosable details. A client
could communicate this id to a back-end provider to get further
information.
example: 550e8400-e29b-11d4-a716-446655440000
error_code:
$ref: '#/components/schemas/log_links'
log_code:
type: string
description: >-
The code is either one of the standardized error codes or a custom error
code.
The code is either one of the standardized error codes or a custom code,
for example specified by a user in the `debug` process.
example: SampleError
error_message:
type: string
description: >-
A message explaining what the client may need to change or what
difficulties the server is facing. By default the message must be sent
in English language. Content Negotiation is used to localize the error
messages: If an Accept-Language header is sent by the client and a
translation is available, the message should be translated accordingly
and the Content-Language header must be present in the response.
example: A sample error message.
error_links:
log_links:
description: >-
Additional links related to this error, e.g. a resource that is
explaining the error and potential solutions in-depth or a contact
e-mail address.
Additional links related to this error or log entry, e.g. a resource that
provides further explanations.
type: array
items:
$ref: '#/components/schemas/link'
Expand Down Expand Up @@ -5347,7 +5354,93 @@ components:
type: array
items:
$ref: '#/components/schemas/GeoJsonGeometry'
log_entry:
title: Log Entry
description: >-
An log message that communicates information about the processed data.
type: object
required:
- id
- level
- message
- path
properties:
id:
type: string
description: >-
An unique identifier for the log message, could simply be an incrementing number.
example: "1"
code:
$ref: '#/components/schemas/log_code'
level:
description: >-
The severity level of the log entry.
The level `error` usually stops processing the data.
type: string
enum:
- error
- warning
- info
- debug
example: error
message:
type: string
description: A message explaining the log entry.
example: >-
Can't load the UDF file from the URL `http://example.com/invalid/file.txt`.
Server responded with error 404.
data:
description: >-
Data of any type. It is the back-ends task to decide how to best
present passed data to a user.
nullable: true
path:
description: >-
Describes where the log entry has occured.
The first element of the array is the process that has triggered the log entry, the second element is the parent of the process that has triggered the log entry, etc. This pattern is followed until the root of the process graph.
type: array
items:
type: object
required:
- node_id
properties:
node_id:
type: string
example: runudf1
process_id:
type: string
example: run_udf
parameter:
type: string
nullable: true
example: udf
links:
$ref: '#/components/schemas/log_links'
responses:
logs:
description: Lists the requested log entries.
content:
application/json:
schema:
type: object
required:
- logs
- links
properties:
logs:
type: array
items:
$ref: '#/components/schemas/log_entry'
links:
type: array
description: >-
Related links, e.g. a page to the customer support.
items:
$ref: '#/components/schemas/link'
created:
description: >-
The resource has been created successfully and the location of the newly
Expand Down Expand Up @@ -5417,6 +5510,23 @@ components:
schema:
$ref: '#/components/schemas/error'
parameters:
log_limit:
name: limit
description: The maximum amount of log entries to include in the response. If not provided or empty, return all log entries.
in: query
allowEmptyValue: true
example: 10
schema:
type: integer
minimum: 1
log_offset:
name: offset
description: The last identifier (property `id` of a log entry) the client has received. If provided, the back-ends only sends the entries that occured after the specified identifier. If not provided or empty, start with the first entry.
in: query
allowEmptyValue: true
example: "log1234"
schema:
type: string
process_graph_id:
name: process_graph_id
in: path
Expand Down
2 changes: 1 addition & 1 deletion processes
Submodule processes updated 2 files
+3 −2 CHANGELOG.md
+26 −6 debug.json

0 comments on commit 514a259

Please sign in to comment.