-
Notifications
You must be signed in to change notification settings - Fork 111
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
More ECS field on spans: "destination" #115
Comments
For the RUM agent the raw address might be a relative url! Since ECS doesn't include relative urls as valid addresses we need to generate address using the page origin. capturing @roncohen , re. |
In OpenTracing, the |
Is it safe to say that when setting |
That depends: do we consider the message broker to be the destination or the source for a consumer span? If it's always the destination then yes, but otherwise we might also need to consider the |
Can we add |
@axw By |
@SergeyKleyman according to https://github.com/opentracing/specification/blob/master/semantic_conventions.md, |
elastic/apm-agent-go#664 is a POC which adds @roncohen if we're capturing the destination address for HTTP spans, we should be taking into account proxies in order to ensure we record the peer network address, for SIEM, right? |
Another POC, elastic/apm-agent-python#618. This adds the The branch can be tested with
|
@axw great! not sure how SIEM deals with proxies. Perhaps there's a separate set of proxy fields? |
Summarizing some discussion from the weekly meeting:
Summarizing open questions:
Proposal: go with the denormalized form, and leave opentracing translation for the next iteration. @elastic/apm-agent-devs please comment if you disagree with the above proposals, or 👍 if you're good. |
I don't really want to maintain a list of default ports if I can help it. Why not just send it? 4-5 digits are not going to add much to the gzipped payloads. |
Sorry, should have added some arguments to the proposals :D this isn't really about saving space, but about trying to harmonize data. If one service is configured to connect to |
thanks for pushing this forward @beniwohli! I've updated the description to your suggestion and added the check box matrix. @elastic/apm-agent-devs Please create your individual issues and link them in the list. |
@beniwohli not sure you are the right person to ask for, but could you summarize the fields that will be necessary on the Intake API, so we can create the according server issue and plan for it. |
right, thanks @simitt. |
IIUC, it's quite important that agents end up sending the same I think we should update the spec (https://github.com/elastic/apm/blob/master/docs/agent-development.md) and add some examples or acceptance tests for things like Postgres, Elasticsearch, MongoDB, Redis, etc. |
Will context.destination.address support more than 1 value? Background of my question is #107 - Proposal: add Database Link to span context. We connect to a database and jump to a second one using a db link. Therefore we would have 2 target DBs |
And just to clarify this should be added only to the context for spans right? Or are there any use cases where a |
@wolframhaussig it would not, and this is only intended to hold network addresses @simitt yes, only spans |
Seeing as the agents are only producing one field, the server will presumably need to explode the address into ip/domain/port. For IPv6 addresses, I think we should format them as in URLs, i.e. by surrounding the IPv6 address component in square brackets, regardless of whether there's a port included. e.g. given |
Can you expand on that? AFAIK, it's not necessary for the service map to have the components of the address split up, or is it? |
Not directly, but I presume we'll be storing the information in ECS |
From an APM/SIEM integration points of view we want to at least extract the |
Could you please clarify the following?
I was under impression that |
I had a chat with Ron offline. There's a few issues here. For service maps, we really want to be able to aggregate on the address and port as one field. That doesn't work if we stick to ECS definitions, as Later on we'll likely want to add other types of destination labels for service maps, such as queue or topic names when sending to a message queue or bus. We were deferring this, but given that we'll need a separate field to combine address and port, it probably makes sense for that field to hold these types of labels too. We've been talking about omitting the port number from the context due to a service maps requirement. This data will also be used for SIEM integration (#115 (comment)); the service maps requirement should not impact SIEM. Due to the interference caused by these, I'd like to refocus this issue specifically on recording network destination information for SIEM, and create a separate issue for capturing a more abstract, logical service destination label. That label is where we'll consider omitting ports and so on. So for this specific issue, I'd like to wind back the proposal half way: agents send
Example 1: client request to
|
The 👀 have it, I've updated the description. I'll follow up with a separate issue for the logical destination soon. |
Note: for IPv6 addresses, surrounding the address with square brackets is only relevant where you have (or may have) the port alongside, e.g. When recording an IPv6 address separately from a port, as we are in the case of |
this is done |
There are several areas that could benefit from having more data on spans. There's a lot we can do, but i suggest we start with simple things. destination is a good first candidate. This builds on and supersedes this proposal which suggests to introduce the
peer
namespace.I suggest we automatically set
destination.address
as the remote address for spans of typeext
. Also fordb
spans, this could be relevant if the data is available. There are probably other we can think of. Generally, i think we shouldn't constrain ourselves to certain types of spans, but set thedestination
fields everywhere they make sense. Would be great to get your ideas.outdated proposal 1
destination.address
is the raw address. That is, we use whatever we have. We can setdestination.port
if we can derive it, e.g. for http libraries where the user did not specify a port we can assume it's port 80 for http and 443 for https. Down the line we can look into also supplyingdestination.ip
ordestination.domain
, but i don't think it's necessary as a start. The rest likedestination.bytes
anddestination.packets
are less relevant in this context.If users create spans manually and set any of the following, we should also set the ECS fields:
peer.ipv4 -> destination.ip
peer.ipv6 -> destination.ip
peer.port -> destination.port
peer.hostname -> destination.domain
@elastic/apm-agent-devs Please have a look. When there are no more objections or additional clarification needed I'll post the checkbox matrix
outdated proposal 2
from @beniwohli:{url.protocol}://{url.domain}:{url.port}
, drop port component if it is the standard port for the given protocolAgents send
context.destination.address
andcontext.destination.port
, with the exact same meanings as defined by ECS:context.destination.address
: should hold either an IP (v4 or v6) or a host/domain name. The server can copy the value todestination.ip
ordestination.domain
, depending on the value, rather than duplicating that logic in each agent, which requires no local contextcontext.destination.port
: should hold a port number; agents should report default portsFor database queries, the destination information can be extracted from the connection string. For outgoing HTTP requests, the information should be extracted from the URI's authority.
We will leave translating OpenTracing
peer.*
tags todestination.
field(s) to a later time.Example 1: client request to https://elastic.co/foo/bar
context.destination.address: elastic.co
context.destination.port: 443 (default HTTPS port)
Example 2: client request to http://[::1]:8080/
context.destination.address: ::1
context.destination.port: 8080
Example 3: query to postgresql://postgres123.local/dbinst
context.destination.address: postgres123.local
context.destination.port: 5432 (default postgres port)
Example 4: query to user:pass@tcp(1.2.3.4:1234)/dbname (MySQL)
context.destination.address: 1.2.3.4
context.destination.port: 1234
The text was updated successfully, but these errors were encountered: