cloudflare_logpush: Add SSL configuration to aws-s3 input

[kcreddy]

Proposed commit message

[cloudflare_logpush] Add SSL configuration to aws-s3 input

Add an ssl variable (type yaml) to the aws-s3 input so users can
specify custom certificate authorities and other TLS settings. This
is needed when the Elastic Agent connects to S3-compatible storage
(such as Cloudflare R2) through a TLS-intercepting corporate proxy
that presents its own CA certificate.

Changes:
- Add ssl variable to manifest.yml under the aws-s3 policy template,
  with a commented-out certificate_authorities example as the default.
- Render ssl conditionally in all 21 aws-s3.yml.hbs templates, after
  proxy_url.
- Update policy test inputs and expected outputs to exercise the new
  variable with a certificate_authorities value.

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.
• I have verified that any added dashboard complies with Kibana's Dashboard good practices

How to test this PR locally

Policy tests run successfully (updated with ssl option):

--- Test results for package: cloudflare_logpush - START ---
╭────────────────────┬───────────────────────┬───────────┬────────────────────────┬────────┬───────────────╮
│ PACKAGE            │ DATA STREAM           │ TEST TYPE │ TEST NAME              │ RESULT │  TIME ELAPSED │
├────────────────────┼───────────────────────┼───────────┼────────────────────────┼────────┼───────────────┤
│ cloudflare_logpush │ access_request        │ policy    │ test-abs.yml           │ PASS   │   17.0388905s │
│ cloudflare_logpush │ access_request        │ policy    │ test-aws-s3.yml        │ PASS   │ 15.088884667s │
│ cloudflare_logpush │ access_request        │ policy    │ test-gcs.yml           │ PASS   │ 17.041074375s │
│ cloudflare_logpush │ access_request        │ policy    │ test-http-endpoint.yml │ PASS   │ 13.026250292s │
│ cloudflare_logpush │ audit                 │ policy    │ test-abs.yml           │ PASS   │ 14.025424416s │
│ cloudflare_logpush │ audit                 │ policy    │ test-aws-s3.yml        │ PASS   │ 15.037789125s │
│ cloudflare_logpush │ audit                 │ policy    │ test-gcs.yml           │ PASS   │ 12.040894666s │
│ cloudflare_logpush │ audit                 │ policy    │ test-http-endpoint.yml │ PASS   │ 13.034491666s │
│ cloudflare_logpush │ casb                  │ policy    │ test-abs.yml           │ PASS   │ 13.042496583s │
│ cloudflare_logpush │ casb                  │ policy    │ test-aws-s3.yml        │ PASS   │ 14.019103083s │
│ cloudflare_logpush │ casb                  │ policy    │ test-gcs.yml           │ PASS   │ 12.027385833s │
│ cloudflare_logpush │ casb                  │ policy    │ test-http-endpoint.yml │ PASS   │ 12.027828417s │
│ cloudflare_logpush │ device_posture        │ policy    │ test-abs.yml           │ PASS   │ 12.032256292s │
│ cloudflare_logpush │ device_posture        │ policy    │ test-aws-s3.yml        │ PASS   │ 12.042376958s │
│ cloudflare_logpush │ device_posture        │ policy    │ test-gcs.yml           │ PASS   │ 12.037872667s │
│ cloudflare_logpush │ device_posture        │ policy    │ test-http-endpoint.yml │ PASS   │ 13.037599125s │
│ cloudflare_logpush │ dlp_forensic_copies   │ policy    │ test-abs.yml           │ PASS   │   13.0430555s │
│ cloudflare_logpush │ dlp_forensic_copies   │ policy    │ test-aws-s3.yml        │ PASS   │ 12.027563167s │
│ cloudflare_logpush │ dlp_forensic_copies   │ policy    │ test-gcs.yml           │ PASS   │ 13.040601125s │
│ cloudflare_logpush │ dlp_forensic_copies   │ policy    │ test-http-endpoint.yml │ PASS   │ 12.054906084s │
│ cloudflare_logpush │ dns                   │ policy    │ test-abs.yml           │ PASS   │ 12.040246375s │
│ cloudflare_logpush │ dns                   │ policy    │ test-aws-s3.yml        │ PASS   │ 12.035916792s │
│ cloudflare_logpush │ dns                   │ policy    │ test-gcs.yml           │ PASS   │ 12.033162209s │
│ cloudflare_logpush │ dns                   │ policy    │ test-http-endpoint.yml │ PASS   │    12.049102s │
│ cloudflare_logpush │ dns_firewall          │ policy    │ test-abs.yml           │ PASS   │ 12.035723417s │
│ cloudflare_logpush │ dns_firewall          │ policy    │ test-aws-s3.yml        │ PASS   │  12.09046475s │
│ cloudflare_logpush │ dns_firewall          │ policy    │ test-gcs.yml           │ PASS   │ 13.991325583s │
│ cloudflare_logpush │ dns_firewall          │ policy    │ test-http-endpoint.yml │ PASS   │ 11.035655708s │
│ cloudflare_logpush │ email_security_alerts │ policy    │ test-abs.yml           │ PASS   │   12.0530255s │
│ cloudflare_logpush │ email_security_alerts │ policy    │ test-aws-s3.yml        │ PASS   │ 11.044571125s │
│ cloudflare_logpush │ email_security_alerts │ policy    │ test-gcs.yml           │ PASS   │ 12.039533083s │
│ cloudflare_logpush │ email_security_alerts │ policy    │ test-http-endpoint.yml │ PASS   │ 11.020521542s │
│ cloudflare_logpush │ firewall_event        │ policy    │ test-abs.yml           │ PASS   │ 12.024359042s │
│ cloudflare_logpush │ firewall_event        │ policy    │ test-aws-s3.yml        │ PASS   │ 12.031645666s │
│ cloudflare_logpush │ firewall_event        │ policy    │ test-gcs.yml           │ PASS   │ 11.033856833s │
│ cloudflare_logpush │ firewall_event        │ policy    │ test-http-endpoint.yml │ PASS   │ 12.023206333s │
│ cloudflare_logpush │ gateway_dns           │ policy    │ test-abs.yml           │ PASS   │ 11.042641541s │
│ cloudflare_logpush │ gateway_dns           │ policy    │ test-aws-s3.yml        │ PASS   │ 12.040402208s │
│ cloudflare_logpush │ gateway_dns           │ policy    │ test-gcs.yml           │ PASS   │  12.03895925s │
│ cloudflare_logpush │ gateway_dns           │ policy    │ test-http-endpoint.yml │ PASS   │ 12.020703709s │
│ cloudflare_logpush │ gateway_http          │ policy    │ test-abs.yml           │ PASS   │   12.0370675s │
│ cloudflare_logpush │ gateway_http          │ policy    │ test-aws-s3.yml        │ PASS   │ 13.023173958s │
│ cloudflare_logpush │ gateway_http          │ policy    │ test-gcs.yml           │ PASS   │ 12.036669083s │
│ cloudflare_logpush │ gateway_http          │ policy    │ test-http-endpoint.yml │ PASS   │ 13.036766959s │
│ cloudflare_logpush │ gateway_network       │ policy    │ test-abs.yml           │ PASS   │ 13.023036583s │
│ cloudflare_logpush │ gateway_network       │ policy    │ test-aws-s3.yml        │ PASS   │ 13.041155708s │
│ cloudflare_logpush │ gateway_network       │ policy    │ test-gcs.yml           │ PASS   │ 13.021887792s │
│ cloudflare_logpush │ gateway_network       │ policy    │ test-http-endpoint.yml │ PASS   │ 15.038958875s │
│ cloudflare_logpush │ http_request          │ policy    │ test-abs.yml           │ PASS   │ 14.049000875s │
│ cloudflare_logpush │ http_request          │ policy    │ test-aws-s3.yml        │ PASS   │ 12.040091042s │
│ cloudflare_logpush │ http_request          │ policy    │ test-gcs.yml           │ PASS   │ 13.045613459s │
│ cloudflare_logpush │ http_request          │ policy    │ test-http-endpoint.yml │ PASS   │ 14.044511084s │
│ cloudflare_logpush │ magic_ids             │ policy    │ test-abs.yml           │ PASS   │ 13.035139916s │
│ cloudflare_logpush │ magic_ids             │ policy    │ test-aws-s3.yml        │ PASS   │ 12.051750208s │
│ cloudflare_logpush │ magic_ids             │ policy    │ test-gcs.yml           │ PASS   │ 12.049005875s │
│ cloudflare_logpush │ magic_ids             │ policy    │ test-http-endpoint.yml │ PASS   │ 13.039760833s │
│ cloudflare_logpush │ nel_report            │ policy    │ test-abs.yml           │ PASS   │ 12.036917542s │
│ cloudflare_logpush │ nel_report            │ policy    │ test-aws-s3.yml        │ PASS   │ 12.033160667s │
│ cloudflare_logpush │ nel_report            │ policy    │ test-gcs.yml           │ PASS   │ 12.049409833s │
│ cloudflare_logpush │ nel_report            │ policy    │ test-http-endpoint.yml │ PASS   │ 12.038336584s │
│ cloudflare_logpush │ network_analytics     │ policy    │ test-abs.yml           │ PASS   │ 13.057086625s │
│ cloudflare_logpush │ network_analytics     │ policy    │ test-aws-s3.yml        │ PASS   │ 12.027675959s │
│ cloudflare_logpush │ network_analytics     │ policy    │ test-gcs.yml           │ PASS   │ 12.031166375s │
│ cloudflare_logpush │ network_analytics     │ policy    │ test-http-endpoint.yml │ PASS   │ 14.041302875s │
│ cloudflare_logpush │ network_session       │ policy    │ test-abs.yml           │ PASS   │ 16.039804625s │
│ cloudflare_logpush │ network_session       │ policy    │ test-aws-s3.yml        │ PASS   │ 14.046845416s │
│ cloudflare_logpush │ network_session       │ policy    │ test-gcs.yml           │ PASS   │ 12.039516458s │
│ cloudflare_logpush │ network_session       │ policy    │ test-http-endpoint.yml │ PASS   │ 12.036659084s │
│ cloudflare_logpush │ page_shield_events    │ policy    │ test-abs.yml           │ PASS   │ 12.050023917s │
│ cloudflare_logpush │ page_shield_events    │ policy    │ test-aws-s3.yml        │ PASS   │ 12.043762792s │
│ cloudflare_logpush │ page_shield_events    │ policy    │ test-gcs.yml           │ PASS   │ 13.038787958s │
│ cloudflare_logpush │ page_shield_events    │ policy    │ test-http-endpoint.yml │ PASS   │ 12.039916666s │
│ cloudflare_logpush │ sinkhole_http         │ policy    │ test-abs.yml           │ PASS   │ 14.051900333s │
│ cloudflare_logpush │ sinkhole_http         │ policy    │ test-aws-s3.yml        │ PASS   │ 12.036101792s │
│ cloudflare_logpush │ sinkhole_http         │ policy    │ test-gcs.yml           │ PASS   │ 13.051122208s │
│ cloudflare_logpush │ sinkhole_http         │ policy    │ test-http-endpoint.yml │ PASS   │ 12.033695792s │
│ cloudflare_logpush │ spectrum_event        │ policy    │ test-abs.yml           │ PASS   │ 12.046756709s │
│ cloudflare_logpush │ spectrum_event        │ policy    │ test-aws-s3.yml        │ PASS   │ 13.042612875s │
│ cloudflare_logpush │ spectrum_event        │ policy    │ test-gcs.yml           │ PASS   │ 14.051788333s │
│ cloudflare_logpush │ spectrum_event        │ policy    │ test-http-endpoint.yml │ PASS   │ 13.050008875s │
│ cloudflare_logpush │ workers_trace         │ policy    │ test-abs.yml           │ PASS   │ 13.027968292s │
│ cloudflare_logpush │ workers_trace         │ policy    │ test-aws-s3.yml        │ PASS   │ 13.045356834s │
│ cloudflare_logpush │ workers_trace         │ policy    │ test-gcs.yml           │ PASS   │ 12.045042917s │
│ cloudflare_logpush │ workers_trace         │ policy    │ test-http-endpoint.yml │ PASS   │ 11.054454666s │
╰────────────────────┴───────────────────────┴───────────┴────────────────────────┴────────┴───────────────╯
--- Test results for package: cloudflare_logpush - END   ---
Done

Related issues

• Closes [Cloudflare Logpush] Add SSL configuration to aws-s3 input #18588

[elasticmachine]

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

[elastic-vault-github-plugin-prod]

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 5ea943c

cc @kcreddy

[ShourieG]

LGTM

[elastic-vault-github-plugin-prod]

Package cloudflare_logpush - 1.44.0 containing this change is available at https://epr.elastic.co/package/cloudflare_logpush/1.44.0/