Skip to content

.Net: [MEVD] Use MemoryMarshal.TryGetArray to efficiently get arrays out of ReadOnlyMemory<float> #11770

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
roji opened this issue Apr 28, 2025 · 1 comment
Open

.Net: [MEVD] Use MemoryMarshal.TryGetArray to efficiently get arrays out of ReadOnlyMemory<float> #11770

roji opened this issue Apr 28, 2025 · 1 comment
Labels
msft.ext.vectordata Related to Microsoft.Extensions.VectorData .NET Issue or Pull requests regarding .NET code

Comments

@roji
Copy link
Member

roji commented Apr 28, 2025

In most of our connectors, we currently do ToArray() on incoming ReadOnlyMemory<float> when serializing, since lower layers/SDKs frequently don't support ROM. We should at least use MemoryMarshal.TryGetArray() to unwrap the ROM when its wraps an entire array, rather than performing the extra allocation and copy. We can add an internal extension of ReadOnlyMemory<T> which does this or falls back to ToArray().
@roji roji added .NET Issue or Pull requests regarding .NET code msft.ext.vectordata Related to Microsoft.Extensions.VectorData labels Apr 28, 2025
@roji roji moved this to Backlog in Semantic Kernel Apr 28, 2025
@roji roji mentioned this issue Apr 28, 2025
Merged
@github-actions github-actions bot changed the title [MEVD] Use MemoryMarshal.TryGetArray to efficiently get arrays out of ReadOnlyMemory<float> .Net: [MEVD] Use MemoryMarshal.TryGetArray to efficiently get arrays out of ReadOnlyMemory<float> Apr 28, 2025
roji added a commit to roji/semantic-kernel that referenced this issue May 16, 2025
@roji
Enable embedding generation for hybrid search
97c125e 
Also use MemoryMarshal.TryGetArray() when possible in various places
to avoid extra allocations/copying.

Closes microsoft#11755
Contributes to microsoft#11770
@roji
Copy link
Member Author

roji commented May 16, 2025

Significant work done in #12090

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
msft.ext.vectordata Related to Microsoft.Extensions.VectorData .NET Issue or Pull requests regarding .NET code
Projects
Semantic Kernel
Status: Backlog
Milestone
No milestone
Development

No branches or pull requests
2 participants
@roji @markwallace-microsoft