-
Notifications
You must be signed in to change notification settings - Fork 111
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add ability to capture non-indexed large content for spans #752
Comments
I added my 👍, but have some questions.
I am missing something. Couldn't we have a newer APM server version that guaranteed |
The mappings for these fields are not set by the APM Server, which only refers this field's mapping to ECS, where Because this is all too clear, there was a need to complicate it some more, so what we did is call those Bottom line: apparently we CAN use
So now the question is only whether we want to use labels for this type of data or is it not the proper field. I don't have any strong opinion towards one vs the other. |
Ignoring the technicalities and details on the apm-server and index mapping for a bit - if we purely relied on the |
I also don't have a strong opinion either way.
|
Please don't ignore, if any of what I wrote is inaccurate, please let us know as we will base our decision also on this information
This is not specific to labels though, is it? We map all sorts of attributes to our fields, so this is relevant to any limitation we have on agents, server and ES mappings for all mapped fields IIANM |
Just ment for the time being to focus on the conversation whether this is the path forward. |
Is your feature request related to a problem? Please describe.
We currently have a distinction between large custom context data that is not intended for indexing and such that is purposed for indexing. The former can be added through agents API, but only to transactions and errors. The latter, labels, are applicable for spans as well, but are limited in size.
There are cases where users want to add large textual data to spans' context, without getting it indexed, for example- as described in this forum request.
Describe the solution you'd like
We should consider adding the custom context API to spans as well. It will have the same limitations as the existing one has in terms of maximum allowed size and not being indexed.
In order to offer this capability through OTel API as well, we may decide on a specific attribute that will be mapped to the custom context field that has the proper ES mapping.
For now, this issue is only for 👍 and 👎 or any further discussion. A spec PR will follow if we decide that it is useful and important enough.
Describe alternatives you've considered
Another alternative is to use the labels API for custom context data as well. However, this requires a change to the current ECS definitions for the mapping of this field, adding the
ignore_above: 1024
to it. This will allow automatic mapping decision based on size. However, since sending very large labels to older APM indices may be problematic, it will require agents to be aware of the actual backend mapping, which would probably be an overkill. If it was only dependent on the APM Server version, that would be OK, as agents are already aware of that (required for the numeric and boolean labels), but this means also a change in APM Server to override the ECS mapping for this field.The text was updated successfully, but these errors were encountered: