[USMP] Update RFC with constants pools #81
Conversation
This commit introduces the notion of constant memory pools and removes the need to define access on each of the targets. Change-Id: I96822cd905716fc13efa6fe58d13bcf7a9da8dee
|
cc : @areusch |
There was a problem hiding this comment.
thanks @manupa-arm , just a question here which i might have missed before
| ``` | ||
| The input IRModule is expected to have "candidate_memory_pools" annotation populated with a orderered list of PoolInfo objects. The ordering will indicate to the planner the order of preference each allocate will be pinned to each Pool. The core compiler will run a pass to assign each tir.allocate with candidate_memory_pools based on the target each PrimFunc is executed, prior to invoking the USMP. | ||
| The input IRModule is expected to have "candidate_memory_pools" annotation populated with a orderered list of WorkspacePoolInfo objects. The ordering will indicate to the planner the order of preference each allocate will be pinned to each Pool. The core compiler will run a pass to assign each tir.allocate with candidate_memory_pools based on the target each PrimFunc is executed, prior to invoking the USMP. |
There was a problem hiding this comment.
not sure if we should wade into the ordering/prioritization problem here. i mean we could, but it does seem like a bit of a can of worms. we could also solve this by customizing the allocator. just curious if you have a use case in mind?
There was a problem hiding this comment.
thanks @areusch for taking a look.
The ordering/prioritization is not in the scope of this change.
In fact, it is something already supported :
However, the USMP will only try to solve the allocation problem not the multi-objective problem of memory and performance. In other words, the scheduling stage is responsible for assigning the right pool for each allocate node. However, if the scheduling does not care about assigning a singular pool (i.e. scheduling is fine with a set of pools), USMP will try to do a prioritized/ordered consumption of the pools to assign all allocated nodes to memory pools.
There was a problem hiding this comment.
ok, that makes sense to me
There was a problem hiding this comment.
only request--shall we document the above somewhere? maybe in a file-level docstring for USMP?
There was a problem hiding this comment.
Sure, happy to do that in a follow up -- I ll add it to the main pass.
|
i have no concerns here, @manupa-arm feel free to submit this when you're ready to start |
This commit introduces the notion of constant memory pools
and removes the need to define access on each of the targets.