YUNIKORN-184. Update YuniKorn-Core Design Documentation #158
Conversation
|
|
||
| ### Components: | ||
|
|
||
| ``` |
There was a problem hiding this comment.
Can we use a chart for this?
This is a bit hard to read.
| | | ||
| |Write Ops +----------------+ | ||
| +-------------+ V ++Scheduler | | ||
| |ConfigWacher + +---------------+ Allocate || And | |
There was a problem hiding this comment.
Guess we are missing PartitionManager, placement rules etc here.
And the rest server.
| | | ||
| +-------------------+-------------------------+ | ||
| |--------+ +------+ +----------+ +----------+ | | ||
| ||Node | |Queue | |Allocation| |Requests | | + |
|
|
||
| #### Scheduler Cache | ||
|
|
||
| Caches all data related to scheduler state, such as used resources of each queues, nodes, allocations. Relationship between allocations and nodes, etc. |
|
Hey @wangdatan, TravisBuddy Request Identifier: a71ce1a0-9c86-11ea-8472-279b0b07e116 |
Codecov Report
@@ Coverage Diff @@
## master #158 +/- ##
=======================================
Coverage 58.62% 58.62%
=======================================
Files 62 62
Lines 6096 6096
=======================================
Hits 3574 3574
Misses 2381 2381
Partials 141 141 Continue to review full report at Codecov.
|
| ``` | ||
|
|
||
| +---------------+ +--------------+ | ||
| |K8s Shim | |YARN Shim | |
There was a problem hiding this comment.
Is it a good idea to mention the YARN shim here, if we don't have it implemented yet? I think it would be better to display only the K8s one and add a note that it can be extended with others, such as a YARN Shim
There was a problem hiding this comment.
May be we can give a reference to SI here than shim
And cover more that in respective shim's as each shim may differ in impl
|
|
||
| **Allocation by application** | ||
|
|
||
| When it goes to Application, see (`scheduler_application.go: func (sa *SchedulingApplication) tryAllocate`), It first sort the pending resource requests belong to the application (based on requests' priority). And based on the selected request, and configured node-sorting policy, it sorts nodes belong to the partition and try to allocate resources on the sorted nodes. |
There was a problem hiding this comment.
I think it worths to mention somewhere what are the available sorting policies and how to configure and create a custom one. (or a link to a configuration page, when we can add this info)
| +-------------+ V ++Scheduler | | ||
| |ConfigWacher + +---------------+ Allocate || And | | ||
| +-------------+ |Scheduler Cache| <-----------------| | | ||
| +----------> +---------------+ Preempt ++Preemptor | |
There was a problem hiding this comment.
Its better we keep config out of core area? as anyway it will be a CRD etc later.
So it may confuse if we put it more tightly coupled.
|
based on the discussion, let's merge this PR first. Later on, we can do more improvements based on this. cc @kingamarton @sunilgovind feel free to create follow up JIRAs to improve this doc. |
Updated initial draft.