Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate how other .NET client libraries deal with data from applications #1446

Open
lukebakken opened this issue Dec 15, 2023 · 2 comments
Assignees
Labels
enhancement next-gen-todo If a rewrite happens, address this issue.

Comments

@lukebakken
Copy link
Contributor

lukebakken commented Dec 15, 2023

Reference:

#1445

In that PR, BasicPublishAsync is modified to conditionally copy data from the client application. @paulomorgado then followed up with this comment.

Someone should take a look at libraries like these to see what is done in this scenario:

@paulomorgado
Copy link
Contributor

paulomorgado commented Dec 15, 2023

We need to be careful with comparisons here.

If you think about stream operations, they don't copy the whole message (or any part of it). It's the responsibility of the caller to not modify that memory.

Microsoft.Data.SqlClient allows some columns to be read as Stream. (see SqlDataReader.GetStream(Int32) Method). No way to write parameter values as Stream, though.

But both SqlDataReader and SqlParameter can work with SqlBytes that provide direct access to different byte sources/destinations.

APIs are becoming more mindful of memory usage and taking advantage of the new types introduced to avoid memory allocation and copying.

If other libraries are not doing the same, maybe they should.

@stebet
Copy link
Contributor

stebet commented Apr 3, 2024

I really like how the new NATS .NET Client v2 does things for (de)serializing data. Take a look at this: https://nats-io.github.io/nats.net.v2/documentation/serialization.html

By default they both accept and return a NatsMemory object which is an IDisposable object that takes care of renting/returning memory, which is then automatically done when the objects go out of scope if it hasn't been manually dispose with for example a using block. This is similar to how MemoryOwner<T> works in the core runtime and makes it easy to use memory efficiently. They also do not buffer commands to the server with a Channel, but just write struct objects/commands directly to the Pipelines.

It's pretty impressive in it's low memory overheads without sacrificing usability or making it easy to hang on to memory too long.

@lukebakken lukebakken removed this from the 7.0.0 milestone May 12, 2024
@lukebakken lukebakken added the next-gen-todo If a rewrite happens, address this issue. label May 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement next-gen-todo If a rewrite happens, address this issue.
Projects
None yet
Development

No branches or pull requests

3 participants