Properly formated User-Agent should be sent in all HTTP Requests

[endersonmaia]

Recently I sent a small patch so that the User-Agent would be sent to in_prometheus_scraper HTTP request, but then I thought that this could be the default behavior for every HTTP requests from fluent-bit.

I'll try to map on this issue all the places we could have this added, and if it's of interest of the maintainers, I can start working on PRs to tackle each of them individually.

Proposal

According to HTTP reference from Mozilla¹, we should have something like:

User-Agent: Fluent-Bit/3.0.6 (input: prometheus_scrape, Git commit: 9af65e2c36a2c36e75b93cab6a92bf29c307ec9e)

Where to add this change

• ./plugins/filter_ecs/ecs.c
• ./plugins/filter_kubernetes/kube_meta.c
• ./plugins/filter_nightfall/nightfall_api.c
• ./plugins/in_calyptia_fleet/in_calyptia_fleet.c
• ./plugins/in_kubernetes_events/kubernetes_events.c
• ./plugins/in_nginx_exporter_metrics/nginx.c
• ./plugins/in_prometheus_scrape/prom_scrape.c
• ./plugins/out_azure/azure.c
• ./plugins/out_azure_blob/azure_blob_blockblob.c
• ./plugins/out_azure_blob/azure_blob.c
• ./plugins/out_azure_kusto/azure_kusto.c
• ./plugins/out_azure_kusto/azure_kusto_ingest.c
• ./plugins/out_azure_logs_ingestion/azure_logs_ingestion.c
• ./plugins/out_bigquery/bigquery.c
• ./plugins/out_calyptia/calyptia.c
• ./plugins/out_chronicle/chronicle.c
• ./plugins/out_datadog/datadog.c
• ./plugins/out_es/es.c
• ./plugins/out_http/http.c
• ./plugins/out_influxdb/influxdb.c
• ./plugins/out_kafka_rest/kafka.c
• ./plugins/out_logdna/logdna.c
• ./plugins/out_loki/loki.c
• ./plugins/out_nrlogs/newrelic.c
• ./plugins/out_opensearch/opensearch.c
• ./plugins/out_opentelemetry/opentelemetry.c
• ./plugins/out_oracle_log_analytics/oci_logan.c
• ./plugins/out_prometheus_remote_write/remote_write.c
• ./plugins/out_skywalking/skywalking.c
• ./plugins/out_slack/slack.c
• ./plugins/out_splunk/splunk.c
• ./plugins/out_stackdriver/gce_metadata.c
• ./plugins/out_stackdriver/stackdriver.c
• ./plugins/out_td/td.c
• ./plugins/out_websocket/websocket.c
• ./src/aws/flb_aws_util.c
• ./src/flb_http_client.c
• ./src/flb_oauth2.c

---

Ideally, we should have a well formed User-Agent like:

User-Agent: Fluent-Bit/3.0.6 (input: in_prometheus_scrape, Git commit: 9af65e2c36a2c36e75b93cab6a92bf29c307ec9e)

And that would change based on the plugin used to make the requests.

Originally posted by @endersonmaia in #8880 (comment)

Footnotes

1. https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/User-Agent ↩

[endersonmaia]

Already inside flb_http_client module we could have a default User-Agent like:

Fluent-Bit/3.0.6 (Linux/amd64, Git commit: 9af65e2c36a2c36e75b93cab6a92bf29c307ec9e)

And then, from inside each plugin/module before calling flb_http_do we call a flb_http_enrich_user_agent (or a better name) that would receive a ex.: (input: in_prometheus_scrape) part to be added, and ending like:

Fluent-Bit/3.0.6 (Linux/amd64, Git commit: 9af65e2c36a2c36e75b93cab6a92bf29c307ec9e) (input: in_prometheus_scrape)