Auto SDK loader mechanism PoC

[jmacd]

With this change, the global trace.Tracer, metric.Meter, and stats.Recorder instances are auto-initialized using an SDK that is loaded from a plugin named by OPENTELEMETRY_LIB.

To avoid an import cycle, the global setter and getter methods (e.g., trace.GlobalTracer) are moved into dedicated global packages. The SDK may not (transitively) depend on the global state via an import dependency, or it will cause a linker error. The linker error is caused because the plugin load is attempted while its dependencies are not finished initializing.

[tigrannajaryan]

are auto-initialized using an SDK that is loaded from a plugin named by OPENTELEMETRY_LIB.

Is dynamically loading the SDK from the plugin the only option or I can still link with the SDK statically?

[tigrannajaryan]

plugin package which is supported on Linux/MacOS only. Are we OK with such limitation?

[jmacd]

| "Is dynamically loading the SDK from the plugin the only option or I can still link with the SDK statically?"

I feel that we need to decide which are / are not requirements:

1. We arrange so that the global SDK object is initialized exactly once before first use.

• It's a nice property, good for "zero touch"
• It's a difficult property, complicates deployment and/or build and/or customization

2. We support replacing the global SDK at runtime

• It means some applications will get a Noop SDK
• It means an SDK could be registered and quickly unregistered

3. Require an explicit Init() call

• Good for linking statically, good for customization
• Bad for zero-touch instrumentation

I've found that there are utilities that support building Go binaries with resources built in, presumably we could use something like this. https://github.com/go-bindata/go-bindata

The issue for plugin support on Windows is just waiting for a contribution: golang/go#19282

One sort of compromise would be to develop a kind of "meta" SDK that can be registered once on first use and supports dynamically attaching and detaching other SDKs. Such an SDK could be configured to buffer and replay events until the first real SDK is registered, for example. Such a "meta" SDK could be a good default implementation, supposing that generally the user will link the SDK they want and initialize it when the time is right.

[tigrannajaryan]

@jmacd good thoughts, thanks for posting.

I feel that we need to decide which are / are not requirements:

I think this warrants a wider discussion (perhaps cross-post on Gitter). You are right that the requirements should drive the implementation. It looks like there is not enough clarity on the requirements (at least I don't have that clarity myself).

I can only state my opinion on the issues your listed:

We arrange so that the global SDK object is initialized exactly once before first use.

This would be a "nice-to-have" for me, provided that it does not cause problems elsewhere. Otherwise as an application developer I think I can have enough discipline and control over my codebase to take care of this myself.

We support replacing the global SDK at runtime
It means some applications will get a Noop SDK

Yes. This has been explicitly stated previously and IMO is a "must-have". The rationale is that we want 3rd party libraries to instrument their code with OpenTelemetry and distribute one version of the library, not 2 versions: one instrumented, the other non-instrumented.

It means an SDK could be registered and quickly unregistered

This should be a non-goal IMO. I don't see why I would need to unregister an SDK at runtime. The decision to use or not use OpenTelemetry I believe happens when the developer builds or deploys their application. There is no need to allow such decision at arbitrary moment of application runtime. If the developer wants to enable/disable telemetry during runtime let them do this via a (possibly custom) exporter that has an on/off switch.

Require an explicit Init() call

I believe that even if it is not "required" then at least it should be an option that I can use. There are situations when I do not want to use dynamic loading at all (e.g. I want to produce a distribution that is a single executable).

The issue for plugin support on Windows is just waiting for a contribution

This is good to know but given that how long the issue has been open and still not resolved I wouldn't place short-term bets on this.

One sort of compromise would be to develop a kind of "meta" SDK that can be registered once on first use and supports dynamically attaching and detaching other SDKs. Such an SDK could be configured to buffer and replay events until the first real SDK is registered, for example.

This sounds like a possible feature of the default implementation that is provided with API package. Instead of throwing away telemetry data accumulate it in memory until the SDK is loaded.