| description |
|---|
This page details the types and organization of information recorded by Gravitee reporters |
The same payload can be sent to any of the Gravitee reporters to write the record of events to a particular output. Payload data can be converted to JSON, CSV, or Elasticsearch format, depending on the reporter type:
| Reporter | JSON | CSV | Elasticsearch |
|---|---|---|---|
| Elasticsearch | false | false | true |
| File | true | true | true |
| TCP | true | true | true |
| Datadog | false | false | false |
Each reporter writes particular payload data to files that share a common naming convention and structure, regardless of output format. JSON, CSV, and Elasticsearch formats each generate the following files, which pertain to different Gravitee execution engines:
{% tabs %} {% tab title="Common" %} The following file is common to both the legacy and reactive execution engines:
| File name | Description |
|---|---|
monitor.json(or monitor.csv) | Reports the state of a Gravitee node (Gateway, APIM) |
{% tab title="Legacy" %} The following files pertain to the legacy execution engine only:
| File name | Description |
|---|---|
endpoint-status.json(or endpoint-status.csv) | Pushed as the result of an API healthcheck |
metrics.json(or metrics.csv) | Common metrics related to a specific HTTP request |
log.json(or log.csv) | An in-depth report of an HTTP request, where the body can be appended to the data structure. This file content is configured from the UI (in the logs => configure logging section). |
{% tab title="Reactive" %} The following files pertain to the reactive execution engine only:
| File name | Description |
|---|---|
metrics.json (or metrics.csv) | Common metrics related to a specific HTTP request |
log.json (or log.csv) | An in-depth report of an HTTP request, where the body can be appended to the data structure. This file content is configured from the UI (in the logs => configure logging section). |
message-metrics.json(or message-metrics.csv) | Same as metrics.json but for an event-driven API |
message-log.json (or message-log.csv) | Same as log.json but for an event-driven API |
The metrics recorded for a given payload are similar for all reporters and formats. Below are the metrics for a sample payload in JSON, CSV, and Elasticsearch formats:
{% tabs %}
{% tab title="JSON" %}
Sample contents of metrics.json:
{% code title="Reactive engine" %}
{
"timestamp": 1692359213844,
"requestId": "076aea69-6024-4590-aaea-6960247590a0",
"transactionId": "076aea69-6024-4590-aaea-6960247590a0",
"apiId": "5f67b38f-0700-4557-a7b3-8f0700855779",
"apiType": "proxy",
"planId": "8463511c-fbed-4ca9-a351-1cfbed9ca99d",
"applicationId": "91f077b0-1204-49e4-b077-b0120419e4f6",
"subscriptionId": "318e47e5-349c-4fa4-8e47-e5349c3fa444",
"clientIdentifier": "318e47e5-349c-4fa4-8e47-e5349c3fa444",
"httpMethod": "GET",
"localAddress": "127.0.0.1",
"remoteAddress": "127.0.0.1",
"host": "localhost:8082",
"uri": "/test-v4",
"pathInfo": "",
"userAgent": "curl/7.88.1",
"requestContentLength": -1,
"requestEnded": true,
"endpoint": "https://api.gravitee.io/echo",
"endpointResponseTimeMs": 137,
"status": 200,
"responseContentLength": 274,
"gatewayResponseTimeMs": 144,
"gatewayLatencyMs": 7
}{% endcode %}
{% code title="Legacy engine" %}
{
"timestamp": 1692357381941,
"proxyResponseTimeMs": 150,
"proxyLatencyMs": 6,
"apiResponseTimeMs": 144,
"requestId": "13f5ae30-068b-4e2d-b5ae-30068bae2d2d",
"api": "ff3c6c48-53e0-41d6-bc6c-4853e011d656",
"application": "91f077b0-1204-49e4-b077-b0120419e4f6",
"transactionId": "13f5ae30-068b-4e2d-b5ae-30068bae2d2d",
"plan": "e115ea63-7cef-4646-95ea-637cef7646ec",
"localAddress": "127.0.0.1",
"remoteAddress": "127.0.0.1",
"httpMethod": "GET",
"host": "localhost:8082",
"uri": "/test",
"requestContentLength": 0,
"responseContentLength": 275,
"status": 200,
"endpoint": "https://api.gravitee.io/echo",
"path": "",
"userAgent": "curl/7.88.1",
"securityType": "API_KEY",
"securityToken": "21b560b2-59b8-4a4b-921a-32b3731fdec4",
"subscription": "04975880-f147-43bc-9758-80f147e3bcbb",
"customMetrics": {
"zone": "europe-north1-a"
}
}{% endcode %} {% endtab %}
{% tab title="CSV" %}
Sample contents of metrics.csv:
{% code title="Reactive engine" %}
"076aea69-6024-4590-aaea-6960247590a0";
"076aea69-6024-4590-aaea-6960247590a0";
1692359213844;
"127.0.0.1";
"127.0.0.1";
"5f67b38f-0700-4557-a7b3-8f0700855779";
"91f077b0-1204-49e4-b077-b0120419e4f6";
"8463511c-fbed-4ca9-a351-1cfbed9ca99d";
"318e47e5-349c-4fa4-8e47-e5349c3fa444";
"";
"";
"/test-v4";
"";
"";
"GET";
200;
"https://api.gravitee.io/echo";
"";
"";
"curl/7.88.1";
"localhost:8082";
-1;
274;
137;
144;
7;
"";
""
{% endcode %}
{% code title="Legacy engine" %}
"13f5ae30-068b-4e2d-b5ae-30068bae2d2d";
"13f5ae30-068b-4e2d-b5ae-30068bae2d2d";
1692357381941;
"127.0.0.1";
"127.0.0.1";
"ff3c6c48-53e0-41d6-bc6c-4853e011d656";
"91f077b0-1204-49e4-b077-b0120419e4f6";
"e115ea63-7cef-4646-95ea-637cef7646ec";
"04975880-f147-43bc-9758-80f147e3bcbb";
"";
"";
"/test";
"";
"";
"GET";
200;
"https://api.gravitee.io/echo";
"";
"";
"curl/7.88.1";
"localhost:8082";
0;
275;
144;
150;
6;
"API_KEY";
"ff3c6c48-53e0-41d6-bc6c-4853e011d656";
"europe-north1-a"
{% endcode %} {% endtab %}
{% tab title="Elasticsearch" %}
Sample contents of metrics.json:
{% code title="Reactive engine" %}
{
"type": "v4-metrics",
"date": "2023.08.18",
"_id": "076aea69-6024-4590-aaea-6960247590a0",
"gateway": "gateway-id",
"@timestamp": "2023-08-18T11:46:53.844Z",
"request-id": "076aea69-6024-4590-aaea-6960247590a0",
"client-identifier": "318e47e5-349c-4fa4-8e47-e5349c3fa444",
"transaction-id": "076aea69-6024-4590-aaea-6960247590a0",
"api-id": "5f67b38f-0700-4557-a7b3-8f0700855779",
"plan-id": "8463511c-fbed-4ca9-a351-1cfbed9ca99d",
"application-id": "91f077b0-1204-49e4-b077-b0120419e4f6",
"subscription-id": "318e47e5-349c-4fa4-8e47-e5349c3fa444",
"http-method": 3,
"local-address": "127.0.0.1",
"remote-address": "127.0.0.1",
"host": "localhost:8082",
"uri": "/test-v4",
"path-info": "",
"user-agent": "",
"request-ended": "true",
"endpoint": "https://api.gravitee.io/echo",
"endpoint-response-time-ms": 137,
"status": 200,
"response-content-length": 274,
"gateway-response-time-ms": 144,
"gateway-latency-ms": 7
}{% endcode %}
{% code title="Legacy engine" %}
{
"gateway": "gateway-id",
"@timestamp": "2023-08-18T11:16:21.941Z",
"type": "request",
"date": "2023.08.18",
"_id": "13f5ae30-068b-4e2d-b5ae-30068bae2d2d",
"transaction": "13f5ae30-068b-4e2d-b5ae-30068bae2d2d",
"method": 3,
"uri": "/test",
"status": 200,
"response-time": 150,
"api-response-time": 144,
"proxy-latency": 6,
"request-content-length": 0,
"response-content-length": 275,
"plan": "e115ea63-7cef-4646-95ea-637cef7646ec",
"api": "ff3c6c48-53e0-41d6-bc6c-4853e011d656",
"application": "91f077b0-1204-49e4-b077-b0120419e4f6",
"local-address": "127.0.0.1",
"remote-address": "127.0.0.1",
"endpoint": "https://api.gravitee.io/echo",
"path": "",
"host": "localhost:8082",
"user-agent": "",
"security-type": "API_KEY",
"security-token": "21b560b2-59b8-4a4b-921a-32b3731fdec4",
"subscription": "04975880-f147-43bc-9758-80f147e3bcbb",
"custom": {
"zone": "europe-north1-a"
}
}{% endcode %} {% endtab %} {% endtabs %}
Depending on which execution engine is used, equivalent fields observe slightly different naming conventions. The number of fields also differs slightly due to differences in execution engine.
The following table maps field names between JSON and Elasticsearch formats and provides a description for each.
Naming conventions are consistent within a certain format. Although there is significant overlap, the specific fields that are generated depend on which execution engine and format are used. The table below compares data recorded with the reactive engine.
| JSON | Elasticsearch | Description |
|---|---|---|
timestamp | @timestamp | The timestamp of the transaction in milliseconds. Elasticsearch formats the @timestamp field as an ISO 8601 string. |
date | This field is only added if the Elasticsearch format is used with the TCP or file reporter. It enables building the index name in your ingest pipeline (e.g., when using Logstash). | |
type | This field is only added if the Elasticsearch format is used with the TCP or file reporter. It enables building the index name in your ingest pipeline (e.g., when using Logstash). | |
requestID | request-id | Universally Unique Identifier (UUID) identifying the request. |
_id | If you are using Elasticsearch format, the content of the _id and request-id fields will be identical. | |
transactionID | transaction-id | This ID can be used to track end-to-end transactions spanning across multiple HTTP requests. The Gateway configuration allows defining an expected correlation ID header passed by a client request. If this header is set, the content of this field will be set to the value of the header. If no correlation header has been passed, the content of this field will be the same as the content of the request ID. This value will be propagated to the upstream service using the correlation header defined in the configuration (the default header is X-Gravitee-Transaction-Id). |
apiID | api-id | The API ID. |
apiType | type | The API type (can be either "proxy" or "message"). |
planID | plan-id | The plan ID. |
applicationID | application-id | The application ID. For a keyless plan, this value is "1". |
subscriptionID | subscription-id | The subscription ID. For a keyless plan, this value will be the same as the value of the remote address field. |
user | user | The authenticated user, if any type of security was used when processing the request. |
securityType | security-type | The security type, if security was used when processing the request (can be either API_KEY, OAUTH2 or JWT). |
securityToken | security-token | The security token, if any type of security was used when processing the request. |
clientIdentifier | client-identifier | This field identifies the client of the request. It is either the subscription ID (if any) or, for a keyless plan, a hash of the remote address. The Client-Identifier can be provided by the client using the header X-Gravitee-Client-Identifier; in this case, the value used by Gravitee will be the original inferred value suffixed with the overridden value. |
httpMethod | http-method | The HTTP method used to perform the client request. |
localAddress | local-address | The address used as a destination when the incoming request was issued by the client. |
remoteAddress | remote-address | The remote address used as a source when the incoming request was issued by the client. |
host | host | The content of the Host header, passed when the incoming request was issued by the client. |
uri | uri | The URI used by the client to perform its request (this includes the context path of the request and query parameters). |
path-info | path-info | The path used to perform the client request (starting from the context path of the API). |
mappedPath | mapped-path | If a path mapping has been defined to group requests in your analytics, this is the value of your mapping. |
userAgent | user-agent | The content of the User-Agent header, passed by the client when the incoming request was issued. |
requestContentLength | The size of the body, in bytes, of the incoming request issued by the Gateway client. | |
requestEnded | request-ended | Flag to indicate if the request completed. |
endpoint | endpoint | The URL used by the proxy to forward the request to the upstream service. |
endpointResponseTimeMs | endpoint-response-time-ms | The time (ms) it takes the upstream service to respond to the Gateway proxy. |
status | status | The HTTP status code of the transaction. |
responseContentLength | response-content-length | The size of the body, in bytes, of the response received by the Gateway client. |
gatewayResponseTimeMs | gateway-response-time-ms | The time (ms) it takes the Gateway to respond to the client (this includes the roundtrip between the Gateway and the upstream service). |
gatewayLatencyMs | gateway-latency-ms | The overhead added by the Gateway when forwarding the request upstream and the response back to the client. |
gateway | A UUID identifying the Gateway instance handling the request. | |
errorKey | error-key | If the policy chain was interrupted by an error, this key identifies the error type. |
errorMessage | error-message | A more detailed explanation of the error associated with the error key (if any). |
custom | custom | Custom metrics defined via the assign-metrics policy will be added to this dictionary. |
Files formatted as CSV do not include a key. Use the following table to map the offset of metrics data recorded with the reactive engine to the corresponding field:
| Offset | Field | Sample value |
|---|---|---|
| 0 | transactionID | 076aea69-6024-4590-aaea-6960247590a0 |
| 1 | requestID | 076aea69-6024-4590-aaea-6960247590a0 |
| 2 | timestamp | 1692359213844 |
| 3 | remoteAddress | 127.0.0.1 |
| 4 | localAddress | 127.0.0.1 |
| 5 | apiID | 5f67b38f-0700-4557-a7b3-8f0700855779 |
| 6 | applicationID | 91f077b0-1204-49e4-b077-b0120419e4f6 |
| 7 | planID | 8463511c-fbed-4ca9-a351-1cfbed9ca99d |
| 8 | subscriptionID | 318e47e5-349c-4fa4-8e47-e5349c3fa444 |
| 9 | user | 5f2dd42f-610b-4719-ae39-8ccf7243047e |
| 10 | tenant | |
| 11 | uri | /test-v4 |
| 12 | path | / |
| 13 | mappedPath | /:anyPath |
| 14 | httpMethod | GET |
| 15 | status | 200 |
| 16 | endpoint | https://api.gravitee.io/echo |
| 17 | errorKey | GATEWAY_OAUTH2_ACCESS_DENIED |
| 18 | errorMessage | |
| 19 | userAgent | curl/7.88.1 |
| 20 | host | api.example.com |
| 21 | requestContent | -1 |
| 22 | responseContent | 274 |
| 23 | endpointResponseTimeMs | 137 |
| 24 | gatewayResponseTimeMs | 144 |
| 25 | gatewayLatencyMs | 7 |
| 26 | securityType | OAUTH2 |
| 27 | securityToken | 6d8772c9-3336-4ede-8ffd-4852cfb85f95 |
| 28 | customMetrics[0] |