-
Notifications
You must be signed in to change notification settings - Fork 8
data catalog API 0.7
Enables search, retrieval and storageof metadata describing data sets.
Version: 0.4.29
BasePath: /
- default: Default namespace
- rest/datasets: Data Catalog - enables search, retrieval and storage of metadata describing data sets.
- application/json
- application/json
GET /rest/datasets
Query should be in this format: { "query": SEARCH_TEXT, "filters":[ {FILTERED_FIELD_NAME: [FIELD_VALUE_1, FIELD_VALUE_1]} ], "from": FROM_HIT_NUMBER, "size": NUMBER_OF_HITS }
All query fields are optional. When filtering by time ranges, you must supply exactly two filter field values. -1 can be used as infinity.
"from" and "size" are used for pagination of search queries. If we get 20 hits for a query, we can set "from" and "size" to 10 to get the second half of hits.
Filter examples:
{"creationTime": [-1, "2015-02-24T14:56"]} <- all until 2015-02-24T14:56 {"format": ["csv", "json"]} <- all CSV and JSON data sets
Field 'orgs' should be in a form of a list of org uuids separated with a coma example: orguuid-01,oruuid-02
Fields 'onlyPublic' and 'onlyPrivate' should have boolean value (true or false). In addition to a query, they allow to choose only private data sets or only public ones. They are mutually exclusive!
Consumer of this endpoint must have a valid OAuth token. Also, user has to be a member of the organization owning the data sets. This doesn't concern admins (console.admin in token's scope) who always have access. Moreover an admin owning the data sets being targeted by this request receives data from all orgs.
Type | Name | Description | Required | Schema | Default |
---|---|---|---|---|---|
QueryParameter | onlyPrivate | Returns a list of the private data sets only | false | boolean | |
QueryParameter | query | A query JSON object. | false | string | |
QueryParameter | onlyPublic | Returns a list of the public data sets only. | false | boolean | |
QueryParameter | orgs | A list of org UUIDs. | false | null string array |
HTTP Code | Description | Schema |
---|---|---|
200 | Result queries returned. | SearchHits |
400 | Invalid or malformed query. | No Content |
500 | Internal error. | No Content |
- rest/datasets
GET /rest/datasets/count
Consumer of this endpoint must have a valid OAuth token. Also, user has to be a member of the organization owning the data sets. This doesn't concern admins (console.admin in token's scope) who always have access. Moreover an admin owning the data sets being targeted by this request receives data from all orgs.
Type | Name | Description | Required | Schema | Default |
---|---|---|---|---|---|
QueryParameter | onlyPrivate | Returns a list of the private data sets only | false | boolean | |
QueryParameter | orgs | A list of org UUIDs. | false | null string array | |
QueryParameter | onlyPublic | Returns a list of the public data sets only. | false | boolean |
HTTP Code | Description | Schema |
---|---|---|
200 | Data set count returned. | integer |
- rest/datasets
PUT /rest/datasets/{entry_id}
Consumer of this endpoint must have a valid OAuth token. Also, user has to be a member of the organization owning the data sets. This doesn't concern admins (console.admin in token's scope) who always have access.
Type | Name | Description | Required | Schema | Default |
---|---|---|---|---|---|
PathParameter | entry_id | ID of a metadata entry (data set). | true | string | |
BodyParameter | body | JSON-formatted metadata entry. | true | InputMetadataEntry |
HTTP Code | Description | Schema |
---|---|---|
200 | Entry updated. | No Content |
201 | Entry created. | No Content |
400 | Putting data set in index failed: malformed data in meta data fields. | No Content |
403 | Forbidden access to required organisation | No Content |
503 | Putting data set in index failed: failed to connect to ElasticSearch. | No Content |
- application/x-www-form-urlencoded
- multipart/form-data
- rest/datasets
DELETE /rest/datasets/{entry_id}
Consumer of this endpoint must have a valid OAuth token. Also, user has to be a member of the organization owning the data sets. This doesn't concern admins (console.admin in token's scope) who always have access.
Type | Name | Description | Required | Schema | Default |
---|---|---|---|---|---|
PathParameter | entry_id | ID of the metadata entry describing some data set. | true | string |
HTTP Code | Description | Schema |
---|---|---|
200 | Entry has been removed from Elastic Search. Status of deletion from external services is in response's body | DeleteResponse |
401 | Authorization header not found. | No Content |
403 | Forbidden access to the resource | No Content |
404 | No entry with the given ID found. | No Content |
503 | Problem connecting to ElasticSearch. | No Content |
- rest/datasets
GET /rest/datasets/{entry_id}
Consumer of this endpoint must have a valid OAuth token. Also, user has to be a member of the organization owning the data sets. This doesn't concern admins (console.admin in token's scope) who always have access.
Type | Name | Description | Required | Schema | Default |
---|---|---|---|---|---|
PathParameter | entry_id | ID of the metadata entry describing some data set. | true | string |
HTTP Code | Description | Schema |
---|---|---|
200 | Success | QueryHit |
403 | Forbidden access to the resource | No Content |
404 | No entry with the given ID found. | No Content |
503 | Problem while connecting to the index. | No Content |
- rest/datasets
POST /rest/datasets/{entry_id}
The body of the POST method should be formed in a following way:
{ "argumentName": ["value01", "value02"] }
The value of a given argument will replace current value for this argument in the specified metadata entry.
Example: { "title": "A new, better title for this data set!" }
Consumer of this endpoint must have a valid OAuth token. Also, user has to be a member of the organization owning the data set. This doesn't concern admins (console.admin in token's scope) who always have access.
Type | Name | Description | Required | Schema | Default |
---|---|---|---|---|---|
PathParameter | entry_id | ID of a metadata entry (data set). | true | string | |
BodyParameter | body | Attributes with values to change. | true | InputMetadataEntry |
HTTP Code | Description | Schema |
---|---|---|
200 | Data set attributes are updated. | No Content |
400 | Wrong input data. | No Content |
403 | Forbidden access to the resource | No Content |
404 | No entry with the given ID found. | No Content |
- rest/datasets
Name | Description | Required | Schema | Default |
---|---|---|---|---|
deleted_from_downloader | true | boolean | ||
deleted_from_publisher | true | boolean |
Name | Description | Required | Schema | Default |
---|---|---|---|---|
category | true | string | ||
recordCount | true | integer | ||
dataSample | true | string | ||
isPublic | true | boolean | ||
creationTime | false | string (date-time) | ||
targetUri | true | string | ||
format | true | string | ||
sourceUri | true | string | ||
title | true | string | ||
size | true | integer | ||
orgUUID | true | string |
Name | Description | Required | Schema | Default |
---|---|---|---|---|
category | true | string | ||
recordCount | true | integer | ||
dataSample | true | string | ||
isPublic | true | boolean | ||
id | true | string | ||
creationTime | false | string (date-time) | ||
targetUri | true | string | ||
format | true | string | ||
sourceUri | true | string | ||
title | true | string | ||
size | true | integer | ||
orgUUID | true | string |
Name | Description | Required | Schema | Default |
---|---|---|---|---|
_type | true | string | ||
_source | true | InputMetadataEntry | ||
_id | true | string | ||
found | true | boolean | ||
_index | true | string | ||
_version | true | integer |
Name | Description | Required | Schema | Default |
---|---|---|---|---|
total | true | integer | ||
hits | true | InputMetadataEntryWithID array | ||
formats | true | string array | ||
categories | true | string array |