[jaeger-v2] Add remotesampling extension

[Pushkarm029]

Description of the changes

• Part of [jaeger-v2] Support sampling strategies #5531
• adds static sampling support for otel-based jaeger-v2.
• supports hot reload
• added unit tests

How was this change tested?

• go run -tags=ui ./cmd/jaeger --config ./cmd/jaeger/config-badger.yaml
• make test

Checklist

• I have read https://github.com/jaegertracing/jaeger/blob/master/CONTRIBUTING_GUIDELINES.md
• I have signed all commits
• I have added unit tests for the new functionality
• I have run lint and test steps successfully
  • for jaeger: make lint test
  • for jaeger-ui: yarn lint and yarn test

[codecov]

Codecov Report

Attention: Patch coverage is 86.15917% with 40 lines in your changes missing coverage. Please review.

Project coverage is 96.70%. Comparing base (882ae45) to head (ed06321).

Files	Patch %	Lines
...ger/internal/extension/remotesampling/extension.go	74.66%	19 Missing and 19 partials ⚠️
.../internal/processors/adaptivesampling/processor.go	94.59%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #5389      +/-   ##
==========================================
- Coverage   96.88%   96.70%   -0.18%     
==========================================
  Files         335      341       +6     
  Lines       16156    16444     +288     
==========================================
+ Hits        15652    15902     +250     
- Misses        335      354      +19     
- Partials      169      188      +19     

Flag	Coverage Δ
badger_v1	8.05% <ø> (ø)
badger_v2	1.90% <0.00%> (ø)
cassandra-3.x-v1	16.62% <ø> (ø)
cassandra-3.x-v2	1.82% <0.00%> (ø)
cassandra-4.x-v1	16.62% <ø> (ø)
cassandra-4.x-v2	1.82% <0.00%> (ø)
elasticsearch-6.x-v1	18.78% <ø> (-0.02%)	⬇️
elasticsearch-7.x-v1	18.86% <ø> (ø)
elasticsearch-8.x-v1	19.05% <ø> (ø)
elasticsearch-8.x-v2	1.90% <0.00%> (ø)
grpc_v1	9.53% <ø> (+0.01%)	⬆️
grpc_v2	7.48% <0.00%> (ø)
kafka	9.75% <ø> (ø)
opensearch-1.x-v1	18.91% <ø> (+0.01%)	⬆️
opensearch-2.x-v1	18.91% <ø> (+0.01%)	⬆️
opensearch-2.x-v2	1.90% <0.00%> (+0.01%)	⬆️
unittests	95.12% <86.15%> (-0.15%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[Pushkarm029]

I am thinking of this approach.:

• create a gRPC sampling server
• then, register api_v2.RegisterSamplingManagerServer(server, sampling.NewGRPCHandler(SamplingStore))
• similarly, create an HTTP server and register a sampling API.

That's how it works in the current Jaeger collector, but we are serving this API in the collector server. Here, we have to create a new one.

suggestions

[Pushkarm029]

One more question:

How can I run hotrod with jaeger-v2 binary?
I tried this, but Jaeger UI was not able to detect Hotrod service.
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 go run ./main.go all
go run -tags=ui ./cmd/jaeger --config ./cmd/jaeger/config-elasticsearch.yaml

[yurishkuro]

We need a remotesampling extension which performs the following duties:

• creates HTTP server that returns sampling strategies as JSON
• allows user a choice of a sampling strategy store
  • file: a static file with auto-reload
  • adaptive: a backend storage that implements interfaces in sampling/strategystore package

We also need an adaptivesampling processor that

• uses remotesampling to obtain storage backend (similar how jaegerexporter uses jaegerextension to get storage)
• process spans and updates strategies in that storage (which then will be read by the extension's server)

All of this should require very minimal implementation, primarily just wiring up the existing components already implemented in app/collector/main

[yurishkuro]

The existing component (from jaeger v1) are in blue:

flowchart LR
    Receiver --> AdaptiveSamplingProcessor --> BatchProcessor --> Exporter
    Exporter -->|"(1) get storage"| JaegerStorageExension
    Exporter -->|"(2) write trace"| TraceStorage
    AdaptiveSamplingProcessor -->|"getStorage()"| StorageConfig

    OTEL_SDK[OTEL
             SDK]
    OTEL_SDK -->|"(1) GET /sampling"| HTTP_endpoint
    HTTP_endpoint -->|"(2) getStrategy()"| StrategiesProvider
    style HTTP_endpoint fill:blue,color:white

    subgraph Jaeger Collector
        Receiver
        BatchProcessor[Batch
                       Processor]
        Exporter        
        TraceStorage[(Trace
                      Storage)]
        AdaptiveSamplingProcessor[Adaptive
                                  Sampling
                                  Processor]
        AdaptiveSamplingProcessorV1[Adaptive
                                    Sampling
                                    Processor_v1]
        style AdaptiveSamplingProcessorV1 fill:blue,color:white
        AdaptiveSamplingProcessor -->|"[]*model.Span"| AdaptiveSamplingProcessorV1
        AdaptiveSamplingProcessorV1 ---|use| SamplingStorage

        subgraph JaegerStorageExension[Jaeger Storage Exension]
            Storage[[Storage
                     Config]]
        end
        subgraph RemoteSamplingExtension[Remote Sampling Extension]
            StrategiesProvider -->|"(3b) getStrategy()"| AdaptiveProvider
            StrategiesProvider -->|"(3a) getStrategy()"| FileProvider
            FileProvider --> FileConfig
            AdaptiveProvider --> StorageConfig
    
            HTTP_endpoint[HTTP
                          endpoint]
            StrategiesProvider[Strategies
                               Provider]
            FileProvider[File
                         Provider]
            AdaptiveProvider[Adaptive
                             Provider]
            style StrategiesProvider fill:blue,color:white
            style FileProvider fill:blue,color:white
            style AdaptiveProvider fill:blue,color:white
            subgraph Config
                FileConfig[[File Config]]
                StorageConfig[[Storage Config]]
            end
            StorageConfig --- SamplingStorage
            SamplingStorage[(Sampling
                             Storage)]
            style SamplingStorage fill:blue,color:white
        end
    end

Loading

[Pushkarm029]

i haven;t pushed latest changes yet.

[yurishkuro]

looks good, in the right direction

[yurishkuro]

how do you plan to test it?

[yurishkuro]

maybe we should default to file, not to more complex adaptive?

[yurishkuro]

please fix the DCO and keep it green

[yurishkuro]

@Pushkarm029 this looks to be in a good shape, we just need to bump test coverage. I verified that the server starts and serves sampling strategies over HTTP.

[Pushkarm029]

@Pushkarm029 this looks to be in a good shape, we just need to bump test coverage. I verified that the server starts and serves sampling strategies over HTTP.

thanks, I will start writing tests now.

[Pushkarm029]

Halfway🛣️

[Pushkarm029]

80% Done, few more to go

[yurishkuro]

@Pushkarm029 I added a few more test and am going to merge it, because the PR has been open for a long time.

The coverage is almost there, we can write extra tests (mostly for remotesampling/extension) separately.

@@            Coverage Diff             @@
##             main    #5389      +/-   ##
==========================================
- Coverage   96.88%   96.70%   -0.18%