Export metrics to a working endpoint when is OTEL_REALM set

[pellared]

Fixes #2582

Initially I tried reusing https://pkg.go.dev/go.opentelemetry.io/contrib/exporters/autoexport, but it resulted in too many changes that I was not comfortable with. The changes were not only related to "code", but in code that we would have to add to not break existing users (e.g. by default our distro uses insecure channel).

Testing

Here is a list manual E2E test scenarios using our example that I did to ensure that the functionality is working properly:

Via Collector

✔️

Default scenario. Setting only OTEL_SERVICE_NAME and OTEL_RESOURCE_ATTRIBUTES on the application. Use collector to send telemetry to Splunk o11y.

Direct Ingest

✔️

Additionally set SPLUNK_REALM and SPLUNK_ACCESS_TOKEN to send telemetry directly to Splunk o11y.

[codecov-commenter]

Codecov Report

Attention: 18 lines in your changes are missing coverage. Please review.

Comparison is base (28c00a5) 82.77% compared to head (5c2cff3) 82.43%.

Files	Patch %	Lines
distro/exporter.go	50.00%	16 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2637      +/-   ##
==========================================
- Coverage   82.77%   82.43%   -0.34%     
==========================================
  Files          83       83              
  Lines        3228     3246      +18     
==========================================
+ Hits         2672     2676       +4     
- Misses        483      497      +14     
  Partials       73       73              

Flag	Coverage Δ
Linux	82.13% <50.00%> (-0.34%)	⬇️
Windows	78.36% <50.00%> (-0.38%)	⬇️
macOS	77.84% <50.00%> (-0.32%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.