Introduce TaskContextReport for reporting task context #16041

YongGang · 2024-03-05T05:57:29Z

Description

This PR initiates enhancements in Druid's task management and monitoring framework. It integrates task context into a new TaskContextReport and aligns task context tags with metrics reporting for a unified overview.
Additionally, the introduction of the TaskContextEnricher interface focuses on enriching tasks with detailed context information, thereby enhancing monitoring, reporting, and providing deeper insights into task dynamics. A potential future implementation of this interface could, for example, utilize the group-id prefix coordinator-issued_compact to classify tasks as part of auto-compaction, subsequently generating appropriate tags for easy identification.

Release note

Introducing TaskContextReport for reporting task context.
Introduce an experimental TaskContextEnricher interface which can be used in extensions to enrich taskContext.

Key changed/added classes in this PR

TaskContextEnricher.java an interface helps provide extra context info about tasks
TaskContextReport.java for reporting task context

This PR has:

suneet-s · 2024-03-05T19:09:55Z

indexing-service/src/main/java/org/apache/druid/indexing/common/task/Task.java

+  default String getLabel()
+  {
+    return getType();
+  }


Since a label is a subjective thing, I can imagine Druid operators wanting to implement their own custom labeling strategy.

Have you considered adding an interface that is extensible to allow Druid operators to define their own labeling? I think this will give operators the flexibility to define their own labels by implementing an extension. Druid can have a default labeling strategy that does something like what is being proposed in this change.

Can you please update the PR with a description of how the interface is expected to work, and any design decisions that were made in choosing the interface as well as docs that describe where the labels are emitted and how they are set.

Some things in particular I'm interested in knowing more about

This label appears to be a system defined label. Is this change meant to support system and user defined labels eventually?

Why is there just one label? Many times there is a reason to have multiple labels for something - why not have a list of labels?

How are Druid operators meant to customize this?

Where is the label expected to be emitted? Are there any future changes needed so that the label is in all the places it needs to be

+1 on this but instead of a label, can we just make it a generic Map<String, Object> that can be enriched however one feels?

@arunramani , if we want to have a generic Map<String, Object>, wouldn't the existing task context suffice for that?

As @suneet-s suggests, an extensible labeling mechanism is worth a thought. For simpler use cases though, we could simply try to add a new task context parameter as it is already a flexible structure that allows Druid operators complete control over their tasks. It has the added benefit of not having to educate operators on new mechanisms.

@kfaraz copying the task context over to the report would be an immediate win and might actually take care of the issue. I was thinking more about any context that needs to be injected after a spec has been planned, which will require some kind of extension point.

indexing-service/src/main/java/org/apache/druid/indexing/common/task/Tasks.java

indexing-service/src/main/java/org/apache/druid/guice/IndexingServiceModuleHelper.java

arunramani · 2024-03-19T21:48:09Z

...ns-core/multi-stage-query/src/main/java/org/apache/druid/msq/indexing/MSQControllerTask.java

@@ -240,13 +241,14 @@ public TaskStatus runTask(final TaskToolbox toolbox) throws Exception
        injector.getInstance(Key.get(ServiceClientFactory.class, EscalatedGlobal.class));
    final OverlordClient overlordClient = injector.getInstance(OverlordClient.class)
                                                  .withRetryPolicy(StandardRetryPolicy.unlimited());
+    final TaskIdentitiesProvider taskIdentitiesProvider = injector.getInstance(TaskIdentitiesProvider.class);


We should use the provider to get ALL instances of TaskIdentitiesProvider and then provide that as a list to the controller.

There should be only one type of provider configured and in use, otherwise the usage will become too complicated.

It could be implemented in a way that isn't so complicated. This is a tagging provider and it makes sense to have multiple providers depending on what the implemented finds useful. We do this with loggers and emitters and such. Does this need to be any different?

arunramani · 2024-03-19T21:50:40Z

...xing-service/src/main/java/org/apache/druid/indexing/common/task/TaskIdentitiesProvider.java

+ * performance and characteristics clearer in reports and summaries.
+ */
+@ExtensionPoint
+public interface TaskIdentitiesProvider


I think TaskInformationProvider a simpler and cleaner name. Did we consider that?

I believe Information might be too broad and doesn't capture the specific intent of this ExtensionPoint, which is focused on delivering task identity details for monitoring and management purposes.

The javadoc here talks about adding tags to tasks. Based on that statement alone - I think this should be called TaskTagsProvider. This can then build on top of an existing concept of user provided tags for tasks that was added in #13760

Design this as an ExtensionPoint, it will be used for areas other than tags, so don't want to limit what it can do in the name of the class.

arunramani · 2024-03-19T21:53:05Z

...rvice/src/main/java/org/apache/druid/indexing/common/task/DefaultTaskIdentitiesProvider.java

+  @Override
+  public Map<String, Object> getTaskMetricTags(Task task)
+  {
+    String taskIdentifier = task.getType();


Should we consider adding a groupType to the task object? Parsing the group IDs name is pretty fragile.

That's a valuable suggestion, however, implementing it could require modifications across numerous task types. I'd like to keep the scope of this PR focused, and consider that as potential follow-up work.

suneet-s

Thanks for this feature @YongGang. I like the concept and can see some potential uses. I'm -1 on this current implementation. The biggest ~~challenges~~ changes I'd like to see are:

Updates to the interface so that it is focused on system generated tags instead of all tags
It looks like in this current form, the tags are only applied on MSQ tasks. I think this should be hooked into all tasks given what the interface talks about.
Integration tests are missing so it is possible this functionality could break
Docs are missing indicating where the system tags could show up.

suneet-s · 2024-03-21T17:43:45Z

...xing-service/src/main/java/org/apache/druid/indexing/common/task/TaskIdentitiesProvider.java

+ * performance and characteristics clearer in reports and summaries.
+ */
+@ExtensionPoint
+public interface TaskIdentitiesProvider


The javadoc here talks about adding tags to tasks. Based on that statement alone - I think this should be called TaskTagsProvider. This can then build on top of an existing concept of user provided tags for tasks that was added in #13760

suneet-s · 2024-03-21T17:44:25Z

...xing-service/src/main/java/org/apache/druid/indexing/common/task/TaskIdentitiesProvider.java

+import java.util.Map;
+
+/**
+ * The TaskIdentitiesProvider interface helps add metric tags to tasks.


Suggested change

* The TaskIdentitiesProvider interface helps add metric tags to tasks.

* The TaskIdentitiesProvider interface provides tags for a task that are reported in metrics and task reports.

...xing-service/src/main/java/org/apache/druid/indexing/common/task/TaskIdentitiesProvider.java

suneet-s · 2024-03-21T19:03:32Z

...rvice/src/main/java/org/apache/druid/indexing/common/task/DefaultTaskIdentitiesProvider.java

+
+    Map<String, Object> tags = task.getContextValue(DruidMetrics.TAGS);
+    Map<String, Object> metricTags = tags == null ? new HashMap<>() : new HashMap<>(tags);
+    metricTags.put(TASK_IDENTIFIER, taskIdentifier);


I don't think we should add this for tag for tasks that do not need it

suneet-s · 2024-03-21T19:03:57Z

...rvice/src/main/java/org/apache/druid/indexing/common/task/DefaultTaskIdentitiesProvider.java

+      }
+    }
+
+    Map<String, Object> tags = task.getContextValue(DruidMetrics.TAGS);


See comment in interface about making this just about system generated tags.

extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/exec/ControllerImpl.java

indexing-service/src/main/java/org/apache/druid/guice/IndexingServiceModuleHelper.java

YongGang · 2024-03-26T03:57:27Z

Thanks for this feature @YongGang. I like the concept and can see some potential uses. I'm -1 on this current implementation. The biggest ~~challenges~~ changes I'd like to see are:

Updates to the interface so that it is focused on system generated tags instead of all tags

It looks like in this current form, the tags are only applied on MSQ tasks. I think this should be hooked into all tasks given what the interface talks about.

Integration tests are missing so it is possible this functionality could break

Docs are missing indicating where the system tags could show up.

Make some design changes:

The interface is refined to enrich tags in place
All tasks are hooked to the interface.
Now checking tags are in task report from integration test
Docs are updated

kfaraz

@YongGang , I took another look at the PR, please find my notes below.

Feedback on the current implementation

As it currently stands, I concur with @suneet-s on the following points:

TaskIdentitiesProvider is not an appropriate name as the information has nothing to do with the identity of the task. It is only meant to add some extra optional information to the Task (most likely in the task context). Something likeTaskContextEnricher or something or even TaskTagsProvider would be more appropriate.
taskIdentifier is not an appropriate name if it is going to take values such as compact or msq_select. ingestionType or something else would be more apt.

Also,

Why do tags need to be a part of the task reports? User specified tags would already be present in the task payload (in the context).
If there is a clear use case of system-generated tags, they should probably go under a separate header and not inside ingestionStatsAndErrors.

Suggested implementation to meet the current requirements

I am not entirely convinced on why we need the new interface. It seems a bit of an overkill, at least for the current requirements.

The only usage of the new interface right now is to populate taskIdentifier as a dimension while emitting the heartbeat metric.
Why not do just that? Essentially the following logic would live in CliPeon (with more appropriate dimension name and values):

if (!Strings.isNullOrEmpty(groupId)) {
      if (groupId.startsWith("coordinator-issued_compact")) {
        taskIdentifier = "compact";
      } else if (groupId.startsWith("coordinator-issued_kill")) {
        taskIdentifier = "kill";
      }
    }

It is already hardcoded in the proposed implementation, so we might as well just hard-code it in CliPeon for the time being.

Note on design

I think it is a little premature for us to be designing the TaskIdentitiesProvider extension right now as we don't have clarity on a real world use case, other than adding dimensions to a metric.

Once we get some clarity on a use case (a reasonable way to justify this would be an implementation of TaskIdentitiesProvider other than the default impl), we would be able to get consensus on the design. Until then, we should simply add a new dimension to the heartbeat metric.

This is especially important since you intend to expose this as an extension point. Extension points need a little more thought when designing as they cannot be changed freely once defined due to backwards compatibility reasons.

kfaraz · 2024-03-26T13:51:43Z

extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/exec/ControllerImpl.java

-          TaskReport.buildTaskReports(new MSQTaskReport(id(), taskReportPayload))
+          TaskReport.buildTaskReports(new MSQTaskReport(
+              id(),
+              task.getContextValue(DruidMetrics.TAGS),


Why does the report need the tags? Aren't these meant only for metrics?

I guess you can put tags as well here. But the correct location should be
MSQTaskReportPayload

Since the tags would be in the context, and the report already has the task context so I think no change should be required here no ?

But it would be already part of context so this should not be required.

The tags can contain some specific info about the task such as msq_async_download can be a tag, it will be generated from code based on task spec not provided by user through context.
Tags are also included in task report as external systems can read task report and rely on the tags to know the purpose of the task and take actions based on it.

Since the tags would be in the context, and the report already has the task context so I think no change should be required here no ?

I think you mean task query context field, it's different from Task.context which all task types can have on the top level.

YongGang · 2024-03-26T17:09:04Z

@kfaraz , thanks for your input.

The design of TaskIdentitiesProvider is to help identify the purpose of the task, as existing task type information falls short (for instance, MSQ task types like query_controller and query_worker offer limited clarity on the tasks' specific functions).
Incorporating these tags into both metrics reporting and task reports enables us to address numerous monitoring and analysis questions, such as:

"how many async download task/Peon are running in the cluster?"
"how many tasks are reading from external data source (e.g. S3) vs internal Druid data transfer?"
"if we want prioritize/limit task running based on its purpose, which info should look at?"

These are real questions we have and it's tightly coupled with the business requirements, that's why it's designed as an ExtensionPoint so it's not bound to a default implementation.
This is the first step we tackle in this field, for instance to answer/solve the third question in the above list there are more works can do regarding to task tags/labels/laning, this interface will be setting the groundwork for subsequent enhancements.

indexing-service/src/main/java/org/apache/druid/indexing/common/task/TaskContextEnricher.java

kfaraz

Left some feedback.

...ing-service/src/main/java/org/apache/druid/indexing/common/task/NoopTaskContextEnricher.java

indexing-service/src/main/java/org/apache/druid/indexing/common/task/TaskContextEnricher.java

indexing-service/src/main/java/org/apache/druid/indexing/common/IngestionStatsAndErrors.java

kfaraz · 2024-04-06T04:55:41Z

extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/exec/ControllerImpl.java

+          TaskReport.buildTaskReports(new MSQTaskReport(
+              id(),
+              taskReportPayload
+          ))


Formatting:

Suggested change

TaskReport.buildTaskReports(new MSQTaskReport(

id(),

taskReportPayload

))

TaskReport.buildTaskReports(

new MSQTaskReport(id(), taskReportPayload)

)

extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/exec/ControllerImpl.java

...lti-stage-query/src/main/java/org/apache/druid/msq/indexing/report/MSQTaskReportPayload.java

suneet-s

Thanks for the updates @YongGang I have removed my -1, but there are still some changes that I think would be good.

Reading the PR, it seems like the main change now is that task contexts are now being passed from the user into the task report. Can you please update the title of the PR and the description to call this out as the key change.

The fact that the TaskContextEnricher interface can be leveraged in future patches is good to add in the description, but should not be part of the release notes as the interface is not yet marked with an ExtensionPoint annotation. Instead, you could describe an example of task context enriching that will be possible in future - like the example implementation done in your previous patch (adding a tag to identify all auto-compaction tasks instead of relying on matching the start of the group-id with coordinator-issued_compact)

Also, can we please add docs to https://github.com/apache/druid/blob/master/docs/ingestion/tasks.md#completion-report documenting the fact that tags are now an expected part of the task report.

docs/operations/metrics.md

extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/exec/ControllerImpl.java

indexing-service/src/main/java/org/apache/druid/indexing/common/IngestionStatsAndErrors.java

docs/operations/metrics.md

suneet-s · 2024-04-08T22:31:17Z

integration-tests/src/test/java/org/apache/druid/tests/indexer/AbstractITBatchIndexTest.java

+      Assert.assertTrue(
+          reportData.getTaskContext() != null && !reportData.getTaskContext().isEmpty(),
+          "Report data does not contain task context. Ensure that TaskContextEnricher is correctly bound."
+      );


Instead of just checking that the context is non empty - can we validate that certain fields we expect are there and contain the expected values?

Similar changes to existing integration tests for streaming, compaction and MSQ tasks please. I also noticed there are 2 AbstractITBatchIndexTest classes, this one, and another one in the druid-it-cases sub module. Please make a similar change there so that we get good integration test coverage that the task context is being included in the report for all the different test cases.

indexing-service/src/main/java/org/apache/druid/indexing/common/task/TaskContextEnricher.java

+   *             supplement existing context entries or introduce new ones, thereby refining the
+   *             task's narrative and operational details.
+   */
+  void enrichContext(Task task);


kfaraz

Some minor pending comments, otherwise looks good.

docs/operations/metrics.md

indexing-service/src/main/java/org/apache/druid/indexing/common/task/TaskContextEnricher.java

kfaraz · 2024-04-10T03:46:51Z

indexing-service/src/test/java/org/apache/druid/indexing/common/task/TaskReportSerdeTest.java

@@ -81,6 +82,22 @@ public void testSerdeOfKillTaskReport() throws Exception
    Assert.assertEquals(originalReport, deserializedReport);
  }

+  @Test
+  public void testSerdeOfTaskContextReport() throws Exception


Thanks for adding the test!

kfaraz · 2024-04-10T03:48:25Z

...-tests-ex/cases/src/test/java/org/apache/druid/testsEx/indexer/AbstractITBatchIndexTest.java

+
+      TaskContextReport taskContextReport = (TaskContextReport) indexer.getTaskReport(taskID).get(TaskContextReport.REPORT_KEY);
+
+      Assert.assertFalse(taskContextReport.getPayload().isEmpty());


This assertion doesn't seem to add much value.

kfaraz · 2024-04-10T03:49:02Z

integration-tests-ex/cases/src/test/java/org/apache/druid/testsEx/msq/ITMultiStageQuery.java

    if (taskReport == null) {
      throw new ISE("Unable to fetch the status report for the task [%]", resultTaskStatus.getTaskId());
    }
+    TaskContextReport taskContextReport = (TaskContextReport) statusReport.get(TaskContextReport.REPORT_KEY);
+    Assert.assertFalse(taskContextReport.getPayload().isEmpty());


Can we have some more meaningful assertions? If not relevant here, then we can just skip this assertion too.

This is to assure TaskContextReport is generated for the task. Since the payload is task context which is hard to know the exact value for comparison so here it just check whether the field is populated.

kfaraz · 2024-04-10T03:50:17Z

integration-tests/src/test/java/org/apache/druid/tests/indexer/AbstractITBatchIndexTest.java

+      TaskContextReport taskContextReport =
+          (TaskContextReport) indexer.getTaskReport(taskID).get(TaskContextReport.REPORT_KEY);
+
+      Assert.assertFalse(taskContextReport.getPayload().isEmpty());


Please add some more concrete assertions or skip this one too.

extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/exec/ControllerImpl.java

Co-authored-by: Kashif Faraz <kashif.faraz@gmail.com>

suneet-s

I will be unavailable to review further updates to this PR for the next 2 weeks, so I am marking my review as +1 so that my review is not blocking.

I do agree with @kfaraz's suggestion that more specific checks in the ITs for a few specific key value pairs in the context would be helpful.

docs/operations/metrics.md

Co-authored-by: Suneet Saldanha <suneet@apache.org>

kfaraz

The tests need to be refined, but that can be addressed later too.

cryptoe · 2024-04-08T09:18:34Z

docs/operations/metrics.md

-field in the `context` field of the ingestion spec. `tags` is expected to be a map of string to object.
+The `tags` dimension enhances metrics, task reports, and the Peon service/heartbeat metric associated with ingestion tasks. These tags are derived from both the ingestion specification—when the `tags` field is specified within the task's context—and system-generated sources. The tags field within a task's `context` should be a map with string keys and object values, facilitating the inclusion of custom metadata. However, tags can also be automatically generated and added by the system, providing a comprehensive set of metadata for monitoring and analysis purposes.
+
+To further customize and enrich task metadata, developers can implement the `TaskContextEnricher` interface. By implementing custom logic within this interface, additional context fields can be introduced. This capability allows for enhanced observability and management of tasks, as these additional context fields can offer deeper insights into task execution, performance, and outcomes.


We should also mention that TaskContextProvider runs in the critical path. So use carefully.

extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/exec/ControllerImpl.java

cryptoe · 2024-04-12T08:33:44Z

...s-core/multi-stage-query/src/main/java/org/apache/druid/msq/util/MultiStageQueryContext.java

+   * Facilitates sharing the controller's execution environment
+   * and configurations with its associated worker tasks.
+   */
+  public static final String CTX_OF_CONTROLLER = "controllerCtx";


This should not be configurable.
We should revert this. Having another configuration parameter does not make sense to me.

My thinking on this:
#16041 (comment)
My follow up work will need to read the Controller's ctx when managing/scheduling Worker task as only Controller task has the full task info (e.g. hard to know if it's an async download task from the Worker task itself), these labels/tags need to be generated based on Controller task then pass on to Worker to guide the task execution.

Just pass only the labels/tag to the worker task always. No need to add a new configuration parameter.

Yup, after thinking this again, this new param in Worker cxt is better to be removed for now.
Will put up a PR to do it on Monday. Thanks @cryptoe

PR submitted #16285 @cryptoe

kfaraz · 2024-04-17T07:22:57Z

indexing-service/src/main/java/org/apache/druid/indexing/overlord/TaskQueue.java

@@ -512,6 +516,8 @@ public boolean add(final Task task)
        SinglePhaseParallelIndexTaskRunner.DEFAULT_USE_LINEAGE_BASED_SEGMENT_ALLOCATION
    );

+    taskContextEnricher.enrichContext(task);


@YongGang , I think we should have put this in a try-catch, log a proper message and rethrow a DruidException. What do you think?

cc: @cryptoe

Yes this is a very critical flow for druid. We cannot allow extensions to break this.
I would go one step further and add a future return type to the interface like .

class TaskEnrichContext{ Future<Void> enrichContext(Task task) }

and then this code waits upto a max of 20 MS to proceed else timeout.

future.get(20MS)

This is a normal interface (instead of an extension point), the default implementation is Noop.
Seems to me it will be overly protected by using try-catch and Future for such code.

cryptoe · 2024-05-22T05:25:40Z

@YongGang I have updated the release notes in the description to better depict the change. Please review it once.

YongGang · 2024-05-27T18:58:24Z

@YongGang I have updated the release notes in the description to better depict the change. Please review it once.

Looks good to me, thanks @cryptoe

github-actions bot added Area - Batch Ingestion Area - Ingestion Area - MSQ For multi stage queries - https://github.com/apache/druid/issues/12262 labels Mar 5, 2024

YongGang mentioned this pull request Mar 5, 2024

Task concurrency management in K8s Task Scheduling extension #15976

Closed

10 tasks

suneet-s reviewed Mar 5, 2024

View reviewed changes

suneet-s added the Design Review label Mar 5, 2024

YongGang marked this pull request as draft March 11, 2024 04:42

YongGang changed the title ~~Task Labeling for Improved Monitoring and Analysis~~ Improve Task Monitoring and Analysis Mar 14, 2024

YongGang marked this pull request as ready for review March 14, 2024 22:09

arunramani reviewed Mar 14, 2024

View reviewed changes

indexing-service/src/main/java/org/apache/druid/indexing/common/task/Tasks.java Outdated Show resolved Hide resolved

arunramani reviewed Mar 14, 2024

View reviewed changes

indexing-service/src/main/java/org/apache/druid/guice/IndexingServiceModuleHelper.java Outdated Show resolved Hide resolved

arunramani reviewed Mar 14, 2024

View reviewed changes

indexing-service/src/main/java/org/apache/druid/guice/IndexingServiceModuleHelper.java Outdated Show resolved Hide resolved

arunramani reviewed Mar 19, 2024

View reviewed changes

suneet-s requested changes Mar 21, 2024

View reviewed changes

YongGang force-pushed the task-labelling branch from aa1cbf1 to fb77fc7 Compare March 25, 2024 17:43

github-actions bot added the Area - Documentation label Mar 26, 2024

kfaraz requested changes Mar 26, 2024

View reviewed changes

YongGang added 9 commits April 3, 2024 15:51

Task concurrency management in K8s Task Scheduling extension

a185df0

more docs

aac937d

update Peon metric to use task label

e9126fa

move task lane config classes to core Druid

f03fdc5

Update getLabel to use MSQDestination's label

d1ebcca

add more test cases

03368d1

switch to use StringUtils

3dec860

add more tests and fix build

1ab4b23

fix checkstyle

6025d70

refine design based on feedback

b7f6b52

YongGang force-pushed the task-labelling branch from 3e44c2b to b7f6b52 Compare April 5, 2024 19:12

YongGang added 2 commits April 5, 2024 12:40

revert TaskToolbox change

4040aba

fix checkstyle

9a8886c

github-advanced-security bot found potential problems Apr 5, 2024

View reviewed changes

indexing-service/src/main/java/org/apache/druid/indexing/common/task/TaskContextEnricher.java Fixed Show fixed Hide fixed

fix checkstyle

a4fbf21

kfaraz reviewed Apr 6, 2024

View reviewed changes

suneet-s self-requested a review April 8, 2024 21:52

suneet-s reviewed Apr 8, 2024

View reviewed changes

introduce TaskContextReport

594d6e7

github-advanced-security bot found potential problems Apr 9, 2024

View reviewed changes

YongGang added 2 commits April 9, 2024 15:08

doc and integration test refine

cdf6b58

fix IT

6a1b3cd

kfaraz reviewed Apr 10, 2024

View reviewed changes

YongGang and others added 3 commits April 9, 2024 21:54

Update docs/operations/metrics.md

17748f8

Co-authored-by: Kashif Faraz <kashif.faraz@gmail.com>

code refine

05740f7

remove unused import

12347a5

YongGang changed the title ~~Improve Task Monitoring and Analysis~~ Introduce TaskContextReport for reporting task context Apr 10, 2024

suneet-s approved these changes Apr 11, 2024

View reviewed changes

docs/operations/metrics.md Outdated Show resolved Hide resolved

Update docs/operations/metrics.md

5cac463

Co-authored-by: Suneet Saldanha <suneet@apache.org>

kfaraz approved these changes Apr 12, 2024

View reviewed changes

kfaraz merged commit da9feb4 into apache:master Apr 12, 2024
85 checks passed

cryptoe reviewed Apr 12, 2024

View reviewed changes

YongGang mentioned this pull request Apr 15, 2024

Remove the unused MSQ Controller context reference for Worker task #16285

Merged

10 tasks

kfaraz reviewed Apr 17, 2024

View reviewed changes

adarshsanjeev added this to the 30.0.0 milestone May 6, 2024

adarshsanjeev mentioned this pull request May 28, 2024

[DRAFT] 30.0.0 release notes #16505

Open

	* The TaskIdentitiesProvider interface helps add metric tags to tasks.
	* The TaskIdentitiesProvider interface provides tags for a task that are reported in metrics and task reports.


		TaskContextReport taskContextReport = (TaskContextReport) indexer.getTaskReport(taskID).get(TaskContextReport.REPORT_KEY);

		Assert.assertFalse(taskContextReport.getPayload().isEmpty());

Introduce TaskContextReport for reporting task context #16041

Introduce TaskContextReport for reporting task context #16041

Conversation

YongGang commented Mar 5, 2024 • edited by cryptoe

Description

Release note

Key changed/added classes in this PR

suneet-s Mar 5, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

suneet-s left a comment • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

YongGang commented Mar 26, 2024

kfaraz left a comment • edited

Choose a reason for hiding this comment

Feedback on the current implementation

Suggested implementation to meet the current requirements

Note on design

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

YongGang commented Mar 26, 2024

kfaraz left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

suneet-s left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kfaraz left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

suneet-s left a comment

Choose a reason for hiding this comment

kfaraz left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cryptoe Apr 20, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cryptoe commented May 22, 2024

YongGang commented May 27, 2024

YongGang commented Mar 5, 2024 •

edited by cryptoe

suneet-s Mar 5, 2024 •

edited

suneet-s left a comment •

edited

kfaraz left a comment •

edited

cryptoe Apr 20, 2024 •

edited