Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Okio-based convenience APIs to thrifty-runtime #408

Merged
merged 1 commit into from
Dec 15, 2020

Conversation

benjamin-bader
Copy link
Collaborator

@benjamin-bader benjamin-bader commented Dec 15, 2020

This adds extensions to BufferedSource and BufferedSink, making it easy to convert them into thrifty Transport objects in a fluent style. It also adds fluent builders for making Protocol instances out of Transport instances. This lets us do the following:

val someBigThriftObject = ...

File("/tmp/example.bin")
    .sink()
    .buffer()
    .transport()
    .binaryProtocol()
    .use(someBigThriftObject::write)

In contrast, before this PR one would have to either write a custom transport, or buffer the entire serialized message in memory:

val someBigThriftObject = ...
val buffer = Buffer()
someBigThriftObject.write(BinaryProtocol(BufferTransport(buffer)))
// buffer now holds the entire serialized 

File("/tmp/example.bin")
    .sink()
    .use() { it.write(buffer, buffer.size) }

The new code is about as brief as the old code, is more easily readable (IMO), and has the potential to be much more efficient in that it can partake in okio's segment-sharing and stream reads and writes rather than buffering entire messages into memory at once. In some real-world use cases this is a savings of MB per message of working memory.

It's clearly not hard to write these custom transports, but there's no need for everyone to have to do it themselves.

@benjamin-bader benjamin-bader merged commit cefa407 into microsoft:master Dec 15, 2020
@benjamin-bader benjamin-bader deleted the okio-convenience branch December 15, 2020 19:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant