-
Notifications
You must be signed in to change notification settings - Fork 871
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Remote SDK Management #2207
Comments
I think having a standard configuration for SDKs would be wonderful. As we consider how to migrate the collector to use the Go SDK for self metrics we are confronted with the need to configure that SDK and the desire to expose options for that configuration to the end-user alongside the rest of the collector configuration. I'd really rather not define a configuration mechanism that is only useful to the collector. Once we have a configuration structure defined it seems like a logical step to enable retrieving (and perhaps updating) that configuration from a remote source. If we can leverage the work already being done to make that work for agents then I'm fully supportive of doing so. |
Would changing the trace sampler for specific components(/services) (e.g. from probability sampler to always on temporarily) or changing the log level (for a period) for all components be a use case? (If the SDKs/components could be set up to periodically retrieve the configuration from a remote source) |
Yes, provided that these values are settings in a configuration file that can be retrieved from a remote source. Generally the management solution / protocol we are working on doesn't care about what individual settings are manageable. From protocol's perspective the configuration is an opaque stream of bytes and the protocol's concern is only to deliver that configuration and then it is up to the Agent (or SDK in this case) to interpret and apply the configuration. Whether the particular value (sampling rate of log level) is updatable at runtime is up to the SDK implementation. (I would argue that they should be updatable). |
@open-telemetry/specs-approvers @open-telemetry/technical-committee I would love to know what you think about this. |
I added a draft proposal to support plain HTTP transport in OpAMP: open-telemetry/opamp-spec#70 The primary use case I have is for Otel SDK remote configuration, so it would be great to know what folks here think about it. |
Resolves open-telemetry#410 I am not aware of any OpenTelemetry or non-OpenTelemetry codebase that uses the experimental TraceConfig. I suggest to remove it. It can be added back if we see the need for it in the future. The SDK configuration may also be achieved by different means, see open-telemetry/opentelemetry-specification#2207 We need to keep this PR open for a while and circulate it widely to make sure we give enough opportunity to object the removal.
Resolves #410 I am not aware of any OpenTelemetry or non-OpenTelemetry codebase that uses the experimental TraceConfig. I suggest to remove it. It can be added back if we see the need for it in the future. The SDK configuration may also be achieved by different means, see open-telemetry/opentelemetry-specification#2207 We need to keep this PR open for a while and circulate it widely to make sure we give enough opportunity to object the removal.
We are looking for something similar. We would like the SDKs to pick up the Metrics Reader Interval from the Collector or a central point: https://opentelemetry.io/docs/reference/specification/sdk-environment-variables/#periodic-exporting-metricreader I think this would be perfect, has there been any movement here? |
Yes, 2 things happened since my last comment:
Once the config file format is finalized the decision needs to be made if we want a remote management capability and if we do we can work on defining how exactly it can use OpAMP to fetch the config file remotely. |
The Agent Management Workgroup is currently working on a protocol specification and prototype implementation of remote agent management capabilities. This will be likely used with OpenTelemetry Collector and possibly with other data collection agents. One element of capabilities is the ability for the Agents to receive their configuration from a remote server.
A few people expressed interest in having a similar capability for Otel SDKs, i.e. for the SDK to receives its configuration from a remote server.
We could potentially implement this in a way that uses the same communication protocol between Otel Collector and Management Server and between Otel SDKs and Management Server. The benefit of such approach would be that there is a single view of all SDK-instrumented applications and all Otel Collectors, all communicating and receiving their (different) configurations from one Management Server.
If we decide that we want such capability for SDKs we likely need to come up with a config file format for SDKs.
This issue is a request for comments. I would like to understand if there is sufficient interest in this so that the Agent Management Workgroup can take this into account in our designs.
Please comment or upvote the issue.
Relevant issue in OpAMP repo: open-telemetry/opamp-spec#16
The text was updated successfully, but these errors were encountered: