New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Copy buffers when reading a URL from an InputStream #1202

Merged
merged 1 commit into from May 27, 2017

Conversation

Projects
None yet
3 participants
@rossabaker
Member

rossabaker commented May 27, 2017

readInputStream recycles the buffer that backs the chunk. If we do any storage or concurrent processing down stream, the chunk mutates out from underneath us.

When reading an inputStream, we'll upgrade to ByteVectorChunk, which is immutable and safe for downstream consumption.

Fixes #1201 /cc @jarreds

@jarreds

This comment has been minimized.

Show comment
Hide comment
@jarreds

jarreds May 27, 2017

Unfortunate to have to copy the bytes, but sounds like there is no other option at the moment.

+1

jarreds commented May 27, 2017

Unfortunate to have to copy the bytes, but sounds like there is no other option at the moment.

+1

@rossabaker

This comment has been minimized.

Show comment
Hide comment
@rossabaker

rossabaker May 27, 2017

Member

We could also solve this on the writer side by copying only when we buffer. But then we'd be copying for all sources, not just those derived from an InputStream. Furthermore, anything else that consumes from readInputStream in any complicated way could reintroduce the bug. I thought it was best to cut of the mutability at the source.

I think changes to fs2 could make this more efficient.

Member

rossabaker commented May 27, 2017

We could also solve this on the writer side by copying only when we buffer. But then we'd be copying for all sources, not just those derived from an InputStream. Furthermore, anything else that consumes from readInputStream in any complicated way could reintroduce the bug. I thought it was best to cut of the mutability at the source.

I think changes to fs2 could make this more efficient.

@jarreds

This comment has been minimized.

Show comment
Hide comment
@jarreds

jarreds May 27, 2017

I'm with you on stopping near source. Thank you.

jarreds commented May 27, 2017

I'm with you on stopping near source. Thank you.

@ChristopherDavenport

Good fix to the problem at hand. 👍

Has a test reproducing the bug, solves said problem, copying is done once, and has Latin words to be extra fancy.

@rossabaker rossabaker merged commit fb5b61a into http4s:master May 27, 2017

1 check failed

continuous-integration/travis-ci/pr The Travis CI build failed
Details

@rossabaker rossabaker deleted the rossabaker:master branch May 27, 2017

aeons added a commit to aeons/http4s that referenced this pull request Jun 15, 2017

rossabaker added a commit to rossabaker/http4s that referenced this pull request Dec 17, 2017

Remove the buffer copying hack from http4s#1202
It seems we had a bad merge after http4s#1197 and it came back.

aeons added a commit that referenced this pull request Dec 18, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment