Fetch active task payloads from memory #15377
Conversation
AmatyaAvadhanula
changed the title
| } | ||
|
|
||
| @Nullable | ||
| Task getActiveTask(String taskId) |
There was a problem hiding this comment.
Instead of Nullable, this can return an Optional.
| @@ -104,7 +104,14 @@ public List<TaskStatusPlus> getTaskStatusPlusList( | |||
|
|
|||
| public Optional<Task> getTask(final String taskid) | |||
| { | |||
| return storage.getTask(taskid); | |||
| // Try to fetch active task from memory | |||
| final Task activeTask = taskLockbox.getActiveTask(taskid); | |||
There was a problem hiding this comment.
I thought this should always be called under the giant lock no ?
There was a problem hiding this comment.
Thanks, I've ensured that the call happens within a lock in the new TaskQueue method
|
@kfaraz thank you for your feedback. Instead of maintaining a separate map which could increase the memory usage of the Overlord, this patch now utilizes the map of active tasks that already exists in the TaskQueue |
| @@ -947,6 +947,25 @@ public CoordinatorRunStats getQueueStats() | |||
| return stats; | |||
| } | |||
|
|
|||
| public Optional<Task> getTask(String id) | |||
There was a problem hiding this comment.
I think it would be better for this method to return only active tasks, i.e. it shouldn't have to query the underlying storage on behalf of the caller. As such, this method should be renamed to getActiveTask
The TaskQueue maintains a map of active task ids to tasks, which can be utilized to get active task payloads, before falling back to the metadata store.
The TaskQueue maintains a map of active task ids to tasks, which can be utilized to get active task payloads, before falling back to the metadata store.
The TaskQueue maintains a map of active task ids to tasks, which can be utilized to get active task payloads, before falling back to the metadata store.
This PR has: