[GraphExecutor] Shared Memory bind and runtime reshaping

[piiswrong]

No description provided.

[tqchen]

nullptr

[tqchen]

Please add a few test cases on this, also suggest the following improvments.

• Make executor take a shared_ptr of the StorageAllocator (this helps cleanup the code reference issue).
• Being able to call release and re-aquire space during runtime, so an executor can optionally release its requested memory, and re-aquire it during imperative execution (optional, can do it later).

[piiswrong]

I thought about sharing StorageAllocator at the start, but that actually doesn't work. Suppose ExecA has 2 chunks of memory, Rebind ExecB reuse 1 chunk and allocate 1 new chunk. Then ExecA destructs. Now The un-reused 1 chunk of memory stays there until all derived executors are deleted.

[tqchen]

Understand this case, however, I guess the point of shared memory execs is that all the shared execs destructs together. We can discuss a bit on this offline

[piiswrong]

Not really. Usually you have one base exec. Then for each batch you derive a new exec with different size. After this batch the new exec destructs. Then if it allocated any new memory they will stay there unused before base destructs.

[piiswrong]

@tqchen Done. Lets do a final pass and merge this.

[tqchen]

make dummy color a constant, so it is easier to expect what is going on