Add metrics for number of segments generated per task in MSQ

[YongGang]

Description

Report ingest/tombstones/count and ingest/segments/count metrics in MSQ when finish publishing segments.

Following are two examples:

{"taskIngestionMode":"NONE","feed":"metrics","taskType":"query_controller","metric":"ingest/tombstones/count","service":"druid/indexer","groupId":"query-481a0c2d-1f04-4a5b-bc30-f343240850d3","host":"localhost:8091","version":"","value":0,"dataSource":"wikipedia","taskId":"query-481a0c2d-1f04-4a5b-bc30-f343240850d3","timestamp":"2023-09-14T17:01:47.951Z"}

{"taskIngestionMode":"NONE","feed":"metrics","taskType":"query_controller","metric":"ingest/segments/count","service":"druid/indexer","groupId":"query-481a0c2d-1f04-4a5b-bc30-f343240850d3","host":"localhost:8091","version":"","value":1,"dataSource":"wikipedia","taskId":"query-481a0c2d-1f04-4a5b-bc30-f343240850d3","timestamp":"2023-09-14T17:01:47.954Z"}

Release note

Add ingest/tombstones/count and ingest/segments/count metrics in MSQ.

---

Key changed/added classes in this PR

• In ControllerImpl report the new metrics

---

This PR has:

• been self-reviewed.
  • using the concurrency checklist (Remove this item if the PR doesn't have any relation to concurrency.)
• added documentation for new or modified features or behaviors.
• a release note entry in the PR description.
• added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
• added or updated version, license, or notice information in licenses.yaml
• added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
• added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
• added integration tests.
• been tested in a test Druid cluster.

[LakshSingla]

1. Is there a way to distinguish between the metrics generated by MSQ and the native ingestion? Since we are selectively adding a few metrics, it would be nice to be able to filter by the mode of ingestion.
2. We'd wanna document this in the MSQ docs and/or the metrics docs somehow

[kfaraz]

Is there a way to distinguish between the metrics generated by MSQ and the native ingestion? Since we are selectively adding a few metrics, it would be nice to be able to filter by the mode of ingestion.

@LakshSingla , I suppose you could filter by the type of task that has reported that metric.

[YongGang]

Is there a way to distinguish between the metrics generated by MSQ and the native ingestion? Since we are selectively adding a few metrics, it would be nice to be able to filter by the mode of ingestion.

@LakshSingla , I suppose you could filter by the type of task that has reported that metric.

yup, can use task type to distinguish. I added two examples of the metrics generated in the PR description.

[YongGang]

Hi @LakshSingla @kfaraz , does this change look good to you?

[kfaraz]

LGTM

[YongGang]

Hi @LakshSingla , does this change look good to you? The main purpose for these two metrics is for alerting: when a task creates way many segments, it will make the cluster unstable, we want to be alerted in this case.

[YongGang]

fyi @cryptoe

[LakshSingla]

Merging the PR, cc @cryptoe if we require doc changes for the same.