[core][rdt] implement sender side memory pool + perf improvements

[Sparks0219]

Rewrite of the RDT NIXL memory pool + a couple performance improvements.
The original RDT NIXL memory pool had a couple key issues, most notably:
1.) It copied the entire storage tensor backing the views
2.) It did not collapse the transfer descriptors resulting in the transfer speed scaling with the number of tensors stored into the pool, even though they were backed by contiguous memory

The new pool solves both of these issues, and only copies the user provided views into the memory pool. It does this in two phases:

• it calls packed_offsets on the tensor sizes + their dtypes to figure out their packed size. This packed size is because the views could have different dtypes, but to construct a view from a tensor they need to be byte aligned by a multiple of the dtype size
• it then calls allocate_group to find a block (happy path) or multiple non contiguous blocks to copy the packed representation into.
• we then collapse the xfer_descs in _merged_desc to minimize the number of xfer descs and reconstruct the views on the other side.

[gemini-code-assist]

Code Review

This pull request refactors the NIXL memory pool management to pack tensors into contiguous pool blocks based on object IDs rather than individual storage pointers, and introduces several microbenchmarks to measure weight-syncing performance. The review feedback highlights critical bugs, including an AttributeError from accessing dtype.itemsize (which is not a valid PyTorch attribute), RuntimeErrors when calling get_device() on CPU tensors, and a potential cleanup failure when handling partially initialized tensor lists during exceptions. Additionally, performance optimizations are suggested to reduce $O(N^2)$ and $O(K^2)$ complexity in the packing and splitting algorithms to linear time.

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.

^{Reviewed by Cursor Bugbot for commit ea81e8b. Configure here.}

[stephanie-wang]

Looks good, just had some comments on simplifying it a bit. Still reviewing tests.

[stephanie-wang]

I think this is a bit of an over-optimization, not clear this is a common use case.

[Sparks0219]

I think this mainly matters if you do a ray.put on a bunch of contiguous views for the memory pool. I found that for the 10k view experiment that I did it added around 200ms to ray.put , and with this optimization it was around 2ms. I could put this in a separate PR if it's a bit easier to review.

[stephanie-wang]

I think the chance of this case where you are using packed tensors and the user's buffers match the packed tensors exactly isn't very high and it makes the code harder to follow, so I would remove this optimization for now.

[Sparks0219]

hmmm yea that makes sense, I think my only hesitancy would be that in the 10k view case we would then always do 10k NIXL READ calls which would increase the latency by like 3x then doing it all in one pull. It was a bit of an engineered example though. Should I just leave it as a follow up for now?

[stephanie-wang]

Yes, I think we should leave out the optimizations that I commented on earlier. I'm not sure if it makes sense to merge them at all, even in a follow-up PR. We can sync about that offline.

[Sparks0219]

Yes, I think we should leave out the optimizations that I commented on earlier. I'm not sure if it makes sense to merge them at all, even in a follow-up PR. We can sync about that offline.

Sounds good, removed them