[Meta][Telemetry] Reduce telemetry footprint

[Bamieh]

Reducing the telemetry footprint ensures future scalability of our telemetry and puts a system in place to enable performance optimizations for our collection methods. This also helps reduce the number of people opting out of telemetry in cases where telemetry is causing significant spikes in resources.

• Exponentially backs off retry attempts for sending usage data Browser retries failed telemetry requests every 60s #115221
• Cache stats to avoid Cache usage data to prevent expensive collection queries #117084
• [Telemetry] Use lastReported timestamp on the 'browser' as well [Telemetry] Use lastReported timestamp on the 'browser' as well #87846
• Performance improvements on collectors:
  • Usage tracking increments should be non-blocking #123151
  • [Maps] Usage collector is refreshing the Data Views on every run #123154
• [Telemetry] Report collector success/fail rate #122636
• [Telemetry] Use only the active tab to send telemetry from the browser #123837
• Add telemetry around the time it is taking for grabbing the telemetry stats [Telemetry] Add telemetry around the time it is taking for grabbing the telemetry stats #119468
• [Telemetry] Add execution_context to identify expensive queries coming from specific collectors #122516
• [Telemetry] Share promise for concurrent telemetry report generation #122572
• Audit Usage collection with high Kibana/ES performance cost for some clusters Usage collection has a high Kibana/ES performance cost for some clusters #117489
• Add load-test suites to identify our performance limitations
  • Concurrent users TelemetryAPIJourney - Retrieving the telemetry payload (cached vs. fresh) kibana-load-testing#211
  • [Telemetry] Load testing with large datasets #122637
• Expensive queries are causing unnecessary load and delays on Elasticsearch  #93770
• Lens using a separate UI collection [Lens] Bake lens ui metric type into @kbn/analytics #48352
• [Telemetry] Custom logger appender? #89839
• [Telemetry] Instrument APM around snapshot telemetry generation #135922

[elasticmachine]

Pinging @elastic/kibana-core (Team:Core)

[rudolf]

There's some overlap in this issue and #117489

[Bamieh]

@rudolf ++. I've consolidated the meta issue you've created in Audit Usage collection with high Kibana/ES performance cost for some clusters. Does that sound reasonable?

[afharo]

Should we also tackle #87846? IMHO, it will reduce the number of requests to only 1 every 24h (no matter how many users the cluster has).

[Bamieh]

@afharo good one! Added to the list

[afharo]

Another idea that would improve our visibility of telemetry is #89839. What do you think?