Add config transport endpoint

[lloeki]

What does this PR do?

Add /v0-7/config endpoint and corresponding transport.

Motivation

Remote configuration.

Additional Notes

A dedicated transport is required, see #2676

How to test the change?

The code is isolated and uncalled, so CI should stay green.

Manual testing:

• Step 1: start the agent

docker run --rm -it -e DD_APM_ENABLED=true -e DD_ENV=${YOUR_ENV} -e DD_HOSTNAME=${YOUR_HOSTNAME} -e DD_BIND_HOST=0.0.0.0 -e DD_APM_NON_LOCAL_TRAFFIC=true -e DD_API_KEY=${YOUR_API_KEY}-e DD_REMOTE_CONFIGURATION_ENABLED=true -p 8126:8126 datadog/agent:7.42.0

• Step 2: Make sure your API has the RC enable on the Org settings page
• Step 3: hit it with the following code

require 'ddtrace'

Datadog.configure do |c|
    c.agent.host = "#{YOUR_AGENT_HOST}   
end

transport_options = {
  agent_settings: Datadog::Core::Configuration::AgentSettingsResolver.call(Datadog.configuration)
}

require 'datadog/core/transport/http'

transport_v7 = Datadog::Core::Transport::HTTP.v7(**transport_options.dup)

CAP_ASM_ACTIVATION                = 1 << 1 # Remote activation via ASM_FEATURES product
CAP_ASM_IP_BLOCKING               = 1 << 2 # accept IP blocking data from ASM_DATA product
CAP_ASM_DD_RULES                  = 1 << 3 # read ASM rules from ASM_DD product
CAP_ASM_EXCLUSIONS                = 1 << 4 # exclusion filters (passlist) via ASM product
CAP_ASM_REQUEST_BLOCKING          = 1 << 5 # can block on request info
CAP_ASM_RESPONSE_BLOCKING         = 1 << 6 # can block on response info
CAP_ASM_USER_BLOCKING             = 1 << 7 # accept user blocking data from ASM_DATA product
CAP_ASM_CUSTOM_RULES              = 1 << 8 # accept custom rules
CAP_ASM_CUSTOM_BLOCKING_RESPONSE  = 1 << 9 # supports custom http code or redirect sa blocking response

capabilities = [
  CAP_ASM_IP_BLOCKING,
  CAP_ASM_USER_BLOCKING,
  CAP_ASM_CUSTOM_RULES,
  CAP_ASM_EXCLUSIONS,
  CAP_ASM_REQUEST_BLOCKING,
  CAP_ASM_RESPONSE_BLOCKING,
  CAP_ASM_DD_RULES,
].reduce(&:|)

capabilities_binary = capabilities
  .to_s(16)
  .tap { |s| s.size.odd? && s.prepend('0') }
  .scan(/\h\h/)
  .map { |e| e.to_i(16) }
  .pack('C*')

products = [
  'ASM_DD',       # Datadog employee issued configuration
  'ASM',          # customer issued configuration (rulesets, passlist...)
  'ASM_FEATURES', # capabilities
  'ASM_DATA',     # config files (IP addresses or users for blocking)
]

state = OpenStruct.new(
  {
    root_version: 1,              # unverified mode, so 1
    targets_version: 0,           # from scratch, so zero
    config_states: [],            # from scratch, so empty
    has_error: false,             # from scratch, so false
    error: '',                    # from scratch, so blank
    opaque_backend_state: '',     # from scratch, so blank
  }
)

id = SecureRandom.uuid # client id

payload = {
  client: {
    state: {
      root_version:         state.root_version,
      targets_version:      state.targets_version,
      config_states:        state.config_states,
      has_error:            state.has_error,
      error:                state.error,
      backend_client_state: state.opaque_backend_state,
    },
    id: id,
    products: products,
    is_tracer: true,
    is_agent: false,
    client_tracer: {
      runtime_id:      Datadog::Core::Environment::Identity.id,
      language:        Datadog::Core::Environment::Identity.lang,
      tracer_version:  Datadog::Core::Environment::Identity.tracer_version,
      service:         Datadog.configuration.service,
      env:             Datadog.configuration.env,
    # app_version:     app_version,   # TODO: I don't know where this is in the tracer
      tags:            [],            # TODO: add nice tags!
    },
    # base64 is needed otherwise the Go agent fails with an unmarshal error
    capabilities: Base64.encode64(capabilities_binary).chomp,
  },
  cached_target_files: [
  # {
  #   path: '',
  #   length: 0,
  #   hashes: '';
  # }
  ],
}


res = transport_v7.send_config(payload)
puts res.target_files
puts res.client_configs
puts res.targets
puts res.roots

[lloeki]

Pending:

• specs
• RBS typing

[marcotc]

Feel free to skip this then, no worries.

[GustavoCaso]

We can probably extract some methods for parsing roots, targets and target_files. Also, error handling could improve we could create helper methods for json parsing and decode

[codecov-commenter]

Codecov Report

Merging #2678 (775bde1) into add-info-transport-endpoint (b4a4815) will decrease coverage by 0.03%.
The diff coverage is 89.65%.

@@                       Coverage Diff                       @@
##           add-info-transport-endpoint    #2678      +/-   ##
===============================================================
- Coverage                        98.04%   98.01%   -0.03%     
===============================================================
  Files                             1178     1180       +2     
  Lines                            64556    64799     +243     
  Branches                          2884     2904      +20     
===============================================================
+ Hits                             63294    63513     +219     
- Misses                            1262     1286      +24     

Impacted Files	Coverage Δ
lib/datadog/core/transport/http.rb	79.66% <75.00%> (-1.20%)	⬇️
lib/datadog/core/transport/http/config.rb	84.55% <84.55%> (ø)
spec/datadog/core/transport/integration_spec.rb	91.89% <85.00%> (-8.11%)	⬇️
lib/datadog/core/transport/config.rb	96.00% <96.00%> (ø)
spec/datadog/core/transport/http_spec.rb	98.92% <98.70%> (+1.86%)	⬆️
lib/datadog/core/transport/http/api.rb	100.00% <100.00%> (ø)

... and 3 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more