Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

trace: add OTEL initialization #10526

Merged
merged 1 commit into from Jun 8, 2023
Merged

trace: add OTEL initialization #10526

merged 1 commit into from Jun 8, 2023

Conversation

milas
Copy link
Member

@milas milas commented May 3, 2023

What I did
This is a bunch of OTEL initialization code. It's all in internal/ because there are re-usable parts here, but Compose isn't the right spot. Once we've stabilized the interfaces a bit and the need arises, we can move it to a separate module.

Currently, a single span is produced to wrap the root Compose command.

Compose will respect the standard OTEL environment variables as well as OTEL metadata from the Docker context. Both can be used simultaneously. The latter is intended for local system integration and is restricted to Unix sockets / named pipes.

None of this is enabled by default. It requires setting the COMPOSE_EXPERIMENTAL_OTEL=1 environment variable to gate it during development.

(not mandatory) A picture of a cute animal, if possible in relation to what you did
grey fox

@milas milas requested a review from a team May 3, 2023 23:00
@milas milas self-assigned this May 3, 2023
@milas milas requested review from nicksieger, ndeloof, StefanScherer, ulyssessouza, glours and laurazard and removed request for a team May 3, 2023 23:00

const maxUnixSocketPathSize = len(syscall.RawSockaddrUnix{}.Path)

func DialInMemory(ctx context.Context, addr string) (net.Conn, error) {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@thaJeztah I was thinking of PRing this to docker/go-connections later. Thoughts? It's inspired by the InmemSocket there

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@milas hmm. need to think about this; honest answer;

  • there's an issue with the latest (main) version of go-connections that we still need to sort out (moby started failing when we tried to update it to latest)
  • there's been some discussion about that repository (mainly around the TLS config specified in that repository, but also because it became a bit of a grab-bag of utilities that are not very related), and we need to re-think if we want to preserve it.

I wonder if we should have something like https://github.com/docker/go-metrics (either to provide abstractions, or to have a shared implementation for some of this).

(all just quick blurbs from my side! haven't dug in deep into all of this, but perhaps something we should discuss with more people)

Copy link
Member Author

@milas milas May 5, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahhh sorry, I should have clarified -- I was only referring to the DialInMemory function - the way that library is structured right now, you can:

  • create a listener type that works across OSes (impls behind build tags)
  • configure an HTTP transport across OSes

...but it does NOT expose a function to create a platform-agnostic in-memory net.Conn directly

(That said, yes -- the rest of the observability pieces should be shared! I purposefully put this in internal/tracing so it cannot become a dependency elsewhere & it doesn't depend on anything in the rest of the Compose code base. Will spin it out into its own repo/module later)

@codecov
Copy link

codecov bot commented May 3, 2023

Codecov Report

Patch coverage: 23.61% and project coverage change: -0.64 ⚠️

Comparison is base (e63ab14) 59.65% compared to head (f3a0668) 59.01%.

❗ Current head f3a0668 differs from pull request most recent head 91f3892. Consider uploading reports for the commit 91f3892 to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##               v2   #10526      +/-   ##
==========================================
- Coverage   59.65%   59.01%   -0.64%     
==========================================
  Files         107      111       +4     
  Lines        9515     9709     +194     
==========================================
+ Hits         5676     5730      +54     
- Misses       3257     3395     +138     
- Partials      582      584       +2     
Impacted Files Coverage Δ
internal/tracing/conn_unix.go 0.00% <0.00%> (ø)
internal/tracing/errors.go 0.00% <0.00%> (ø)
internal/tracing/mux.go 0.00% <0.00%> (ø)
internal/tracing/tracing.go 15.06% <15.06%> (ø)
internal/tracing/docker_context.go 24.59% <24.59%> (ø)
cmd/main.go 67.24% <63.63%> (-4.76%) ⬇️

... and 4 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

Copy link
Contributor

@glours glours left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most of the new files need the Copyright header.
e2e tests checking the console output seem to not be happy, maybe there is a trailing character added by the otel configuration? 🤔

internal/tracing/conn_windows.go Outdated Show resolved Hide resolved
cmd/main.go Outdated Show resolved Hide resolved
@milas milas marked this pull request as draft May 11, 2023 13:59
@milas milas force-pushed the otel-init branch 2 times, most recently from 39eae89 to 368e242 Compare June 1, 2023 21:08
@milas milas marked this pull request as ready for review June 6, 2023 13:48
@milas milas requested a review from glours June 6, 2023 13:55
Copy link
Contributor

@glours glours left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

internal/tracing/docker_context.go Outdated Show resolved Hide resolved
This is a bunch of OTEL initialization code. It's all in
`internal/` because there are re-usable parts here, but Compose
isn't the right spot. Once we've stabilized the interfaces a bit
and the need arises, we can move it to a separate module.

Currently, a single span is produced to wrap the root Compose
command.

Compose will respect the standard OTEL environment variables
as well as OTEL metadata from the Docker context. Both can be
used simultaneously. The latter is intended for local system
integration and is restricted to Unix sockets / named pipes.

None of this is enabled by default. It requires setting the
`COMPOSE_EXPERIMENTAL_OTEL=1` environment variable to gate it
during development.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
@milas milas merged commit 3c8a56d into docker:v2 Jun 8, 2023
23 checks passed
@milas milas deleted the otel-init branch June 8, 2023 20:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants