Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option do disable log payloads indexation #2952

Closed
brasseld opened this issue Nov 28, 2019 · 0 comments
Closed

Add option do disable log payloads indexation #2952

brasseld opened this issue Nov 28, 2019 · 0 comments

Comments

@brasseld
Copy link
Member

When the logging condition is true, the gateway is storing:

  • Request headers + payload
  • Response headers + payload

Since #2053 the payload is defined as a keyword for being able to search logs base on payloads.

This results with issues for large payloads:

09:19:30.692 [vert.x-eventloop-thread-0] [] ERROR i.g.e.client.http.HttpClient - An error occurs while indexing data into ES: indice[gravitee-log-2019.11.13] error[Document contains at least one immense term in field="client-response.body" (whose UTF8 encoding is longer than the max length 32766), all of which were skipped.  Please correct the analyzer to not produce such terms.  The prefix of the first immense term is: '[123, 10, 32, 34, 101, 120, 112, 101, 114, 116, 79, 112, 105, 110, 105, 111, 110, 115, 34, 58, 32, 91, 10, 32, 123, 10, 32, 34, 101, 120]...', original message: bytes can be at most 32766 in length; got 49023]

We should provide a way to enable / disable payload indexation

@brasseld brasseld added this to the APIM - 1.30.x milestone Nov 28, 2019
brasseld added a commit to gravitee-io/gravitee-common-elasticsearch that referenced this issue Dec 4, 2019
NicolasGeraud pushed a commit to gravitee-io/gravitee-common-elasticsearch that referenced this issue Dec 5, 2019
@NicolasGeraud NicolasGeraud modified the milestones: APIM - 1.30.x, APIM - 1.30.1 Dec 5, 2019
aelamrani added a commit to gravitee-io/gravitee-common-elasticsearch that referenced this issue May 6, 2020
* chore(): Prepare next version

* chore(): Prepare next version

* fix(template): index remote address

fix gravitee-io/issues#2895

* release(1.25.13)

* chore(): Prepare next version

* fix(elasticsearch): Update field type for body payload

Closes gravitee-io/issues#2952

* fix(httpClient): allow path in endpoints

fix gravitee-io/issues#3030

* fix(groupBy): aggregation by response-time doesnt work

fix gravitee-io/issues#3041

* release(1.30.1)

* chore(): Prepare next version

* fix: analytic widgets with latency displayed hits instead of latency

Closes gravitee-io/issues#3041

* release(1.30.2)

* chore(): Prepare next version

* fix: cannot index a cpu loadaverage as long

Closes gravitee-io/issues#2692

* release(1.25.14)

* release(1.30.3)

* chore(): Prepare next version

* fix(reporter): Do not stop if bulk indexer encounters error

Closes gravitee-io/issues#3630

* chore(): upgrade parent to fix gpg error

* release(1.30.4)

Co-authored-by: Gravitee.io Bot <contact@gravitee.io>
Co-authored-by: David BRASSELY <brasseld@gmail.com>
Co-authored-by: Azize Elamrani <azize.elamrani@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
API Management
Awaiting triage
Development

No branches or pull requests

2 participants