Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HTTP Log Plugin missed updates (new fields, merge for CE 2.3.x on 1/20) #2393

Merged
merged 18 commits into from
Jan 20, 2021
Merged
Show file tree
Hide file tree
Changes from 17 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions app/_data/extensions/kong-inc/http-log/versions.yml
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
- release: 2.0-x
- release: 1.0-x
- release: 0.1-x
227 changes: 227 additions & 0 deletions app/_hub/kong-inc/http-log/1.0-x.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,227 @@
---
name: HTTP Log
publisher: Kong Inc.
version: 1.0.x

desc: Send request and response logs to an HTTP server
description: |
Send request and response logs to an HTTP server.

<div class="alert alert-warning">
<strong>Note:</strong> The functionality of this plugin as bundled
with versions of Kong prior to 0.10.2 differs from
what is documented herein. Refer to the
<a href="https://github.com/Kong/kong/blob/master/CHANGELOG.md">CHANGELOG</a>
for details.
</div>

type: plugin
categories:
- logging

kong_version_compatibility:
community_edition:
compatible:
- 1.5.x
- 1.4.x
- 1.3.x
- 1.2.x
- 1.1.x
- 1.0.x
- 0.14.x
- 0.13.x
- 0.12.x
- 0.11.x
- 0.10.x
- 0.9.x
- 0.8.x
- 0.7.x
- 0.6.x
- 0.5.x
- 0.4.x
- 0.3.x
enterprise_edition:
compatible:
- 1.5.x
- 1.3-x
- 0.36-x
- 0.35-x
- 0.34-x
- 0.33-x
- 0.32-x
- 0.31-x

params:
name: http-log
service_id: true
route_id: true
consumer_id: true
protocols: ["http", "https", "grpc", "grpcs"]
dbless_compatible: yes
config:
- name: http_endpoint
required: true
default:
value_in_examples: http://mockbin.org/bin/:id
description: The HTTP endpoint (including the protocol to use) to which the data is sent.
- name: method
required: false
default: "`POST`"
value_in_examples: POST
description: |
An optional method used to send data to the HTTP server. Supported values are `POST` (default), `PUT`, and `PATCH`.
- name: timeout
required: false
default: "`10000`"
value_in_examples: 1000
description: An optional timeout in milliseconds when sending data to the upstream server.
- name: keepalive
required: false
default: "`60000`"
value_in_examples: 1000
description: An optional value in milliseconds that defines how long an idle connection will live before being closed.
extra: |
**NOTE:** If the `config.http_endpoint` contains a username and password (for example,
`http://bob:password@example.com/logs`), then Kong automatically includes
a basic-auth `Authorization` header in the log requests.

---

## Log Format

Every request is logged separately in a JSON object in the following format:

```json
{
"request": {
"method": "GET",
"uri": "/get",
"url": "http://httpbin.org:8000/get",
"size": "75",
"querystring": {},
"headers": {
"accept": "*/*",
"host": "httpbin.org",
"user-agent": "curl/7.37.1"
},
"tls": {
"version": "TLSv1.2",
"cipher": "ECDHE-RSA-AES256-GCM-SHA384",
"supported_client_ciphers": "ECDHE-RSA-AES256-GCM-SHA384",
"client_verify": "NONE"
}
},
"upstream_uri": "/",
"response": {
"status": 200,
"size": "434",
"headers": {
"Content-Length": "197",
"via": "kong/0.3.0",
"Connection": "close",
"access-control-allow-credentials": "true",
"Content-Type": "application/json",
"server": "nginx",
"access-control-allow-origin": "*"
}
},
"tries": [
{
"state": "next",
"code": 502,
"ip": "127.0.0.1",
"port": 8000
},
{
"ip": "127.0.0.1",
"port": 8000
}
],
"authenticated_entity": {
"consumer_id": "80f74eef-31b8-45d5-c525-ae532297ea8e",
"id": "eaa330c0-4cff-47f5-c79e-b2e4f355207e"
},
"route": {
"created_at": 1521555129,
"hosts": null,
"id": "75818c5f-202d-4b82-a553-6a46e7c9a19e",
"methods": null,
"paths": [
"/example-path"
],
"preserve_host": false,
"protocols": [
"http",
"https"
],
"regex_priority": 0,
"service": {
"id": "0590139e-7481-466c-bcdf-929adcaaf804"
},
"strip_path": true,
"updated_at": 1521555129
},
"service": {
"connect_timeout": 60000,
"created_at": 1521554518,
"host": "example.com",
"id": "0590139e-7481-466c-bcdf-929adcaaf804",
"name": "myservice",
"path": "/",
"port": 80,
"protocol": "http",
"read_timeout": 60000,
"retries": 5,
"updated_at": 1521554518,
"write_timeout": 60000
},
"workspaces": [
{
"id":"b7cac81a-05dc-41f5-b6dc-b87e29b6c3a3",
"name": "default"
}
],
"consumer": {
"username": "demo",
"created_at": 1491847011000,
"id": "35b03bfc-7a5b-4a23-a594-aa350c585fa8"
},
"latencies": {
"proxy": 1430,
"kong": 9,
"request": 1921
},
"client_ip": "127.0.0.1",
"started_at": 1433209822425
}
```

A few considerations on the above JSON object:

* `request` contains properties about the request sent by the client.
* `response` contains properties about the response sent to the client.
* `tries` contains the list of tries and retries (successes and failures) made by the load balancer for the request.
* `route` contains Kong properties about the specific Route requested.
* `service` contains Kong properties about the Service associated with the requested Route.
* `authenticated_entity` contains Kong properties about the authenticated credential if an authentication plugin has been enabled.
* `workspaces` contains Kong properties of the Workspaces associated with the requested Route. **Only in Kong Enterprise version >= 0.34**.
* `consumer` contains the authenticated Consumer if an authentication plugin has been enabled.
* `latencies` contains some data about the latencies involved:
* `proxy` is the time it took for the final service to process the request.
* `kong` is the internal Kong latency that it took to run all the plugins.
* `request` is the time elapsed between the first bytes were read from the client and after the last bytes were sent to the client. Useful for detecting slow clients.
* `client_ip` contains the original client IP address.
* `started_at` contains the UTC timestamp of when the request started to be processed.

----

## Kong Process Errors

This logging plugin only logs HTTP request and response data. If you are
looking for the Kong process error file (which is the nginx error file), then
you can find it at the following path:

`$KONG_PREFIX/logs/error.log`

where `$KONG_PREFIX` means
[prefix in the configuration](/{{site.data.kong_latest.release}}/configuration/#prefix).
73 changes: 43 additions & 30 deletions app/_hub/kong-inc/http-log/index.md
Original file line number Diff line number Diff line change
@@ -1,28 +1,21 @@
---
name: HTTP Log
publisher: Kong Inc.
version: 1.0.0
version: 2.0.x

desc: Send request and response logs to an HTTP server
description: |
Send request and response logs to an HTTP server.

<div class="alert alert-warning">
<strong>Note:</strong> The functionality of this plugin as bundled
with versions of Kong prior to 0.10.2 differs from
what is documented herein. Refer to the
<a href="https://github.com/Kong/kong/blob/master/CHANGELOG.md">CHANGELOG</a>
for details.
</div>

type: plugin
categories:
- logging

kong_version_compatibility:
community_edition:
compatible:
- 2.2.x
- 2.3.x
- 2.2.x
- 2.1.x
- 2.0.x
- 1.5.x
Expand Down Expand Up @@ -68,33 +61,53 @@ params:
required: true
default:
value_in_examples: http://mockbin.org/bin/:id
description: The HTTP endpoint (including the protocol to use) to which the data will be sent.
description: The HTTP endpoint (including the protocol to use) to which the data is sent.
- name: method
required: false
default: "`POST`"
value_in_examples: POST
description: |
An optional method used to send data to the http server, other supported values are `PUT`, `PATCH`
An optional method used to send data to the HTTP server. Supported values are `POST` (default), `PUT`, and `PATCH`.
- name: timeout
required: false
default: "`10000`"
value_in_examples: 1000
description: An optional timeout in milliseconds when sending data to the upstream server
description: An optional timeout in milliseconds when sending data to the upstream server.
- name: keepalive
required: false
default: "`60000`"
value_in_examples: 1000
description: An optional value in milliseconds that defines for how long an idle connection will live before being closed
description: An optional value in milliseconds that defines how long an idle connection will live before being closed.
- name: flush_timeout
required: false
default: "`2`"
value_in_examples: 2
description: |
Optional time in seconds. If `queue_size` > 1, this is the max idle time before sending a log with less than `queue_size` records.
- name: retry_count
required: false
default: 10
value_in_examples: 15
description: Number of times to retry when sending data to the upstream server.
- name: queue_size
required: false
default: 1
description: Max number of log entries to be sent on each message to the upstream sever.
javierguerragiraldez marked this conversation as resolved.
Show resolved Hide resolved
- name: headers
required: false
default: empty table
description: An optional table of headers added to the HTTP message to the upstream server.
extra: |
**NOTE:** If the `config.http_endpoint` contains a username and password (ex.
`http://bob:password@example.com/logs`), then Kong will automatically include
**NOTE:** If the `config.http_endpoint` contains a username and password (for example,
`http://bob:password@example.com/logs`), then Kong Gateway automatically includes
a basic-auth `Authorization` header in the log requests.

---

## Log Format

Every request will be logged separately in a JSON object, with the following format:
Every request is logged separately in a JSON object (or array of JSON objects if
`queue_size` argument > 1) in the following format:

```json
{
Expand Down Expand Up @@ -203,23 +216,23 @@ Every request will be logged separately in a JSON object, with the following for

A few considerations on the above JSON object:

* `request` contains properties about the request sent by the client
* `response` contains properties about the response sent to the client
* `tries` contains the list of (re)tries (successes and failures) made by the load balancer for this request
* `route` contains Kong properties about the specific Route requested
* `service` contains Kong properties about the Service associated with the requested Route
* `authenticated_entity` contains Kong properties about the authenticated credential (if an authentication plugin has been enabled)
* `workspaces` contains Kong properties of the Workspaces associated with the requested Route. **Only in Kong Enterprise version >= 0.34**.
* `consumer` contains the authenticated Consumer (if an authentication plugin has been enabled)
* `request` contains properties about the request sent by the client.
* `response` contains properties about the response sent to the client.
* `tries` contains the list of tries and retries (successes and failures) made by the load balancer for the request.
* `route` contains {{site.base_gateway}} properties about the specific Route requested.
* `service` contains {{site.base_gateway}} properties about the Service associated with the requested Route.
* `authenticated_entity` contains {{site.base_gateway}} properties about the authenticated credential if an authentication plugin has been enabled.
* `workspaces` contains {{site.ee_gateway_name}} properties of the Workspaces associated with the requested Route. **Only in {{site.ee_gateway_name}} version >= 0.34**.
* `consumer` contains the authenticated Consumer if an authentication plugin has been enabled.
* `latencies` contains some data about the latencies involved:
* `proxy` is the time it took for the final service to process the request
* `kong` is the internal Kong latency that it took to run all the plugins
* `proxy` is the time it took for the final service to process the request.
* `kong` is the internal {{site.base_gateway}} latency that it took to run all the plugins.
* `request` is the time elapsed between the first bytes were read from the client and after the last bytes were sent to the client. Useful for detecting slow clients.
* `client_ip` contains the original client IP address
* `started_at` contains the UTC timestamp of when the request has started to be processed.
* `client_ip` contains the original client IP address.
* `started_at` contains the UTC timestamp of when the request started to be processed.

----

## Kong Process Errors
## Kong Gateway Process Errors

{% include /md/plugins-hub/kong-process-errors.md %}