Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Copy buffers when reading a URL from an InputStream #1202

Merged
merged 1 commit into from May 27, 2017

Conversation

@rossabaker
Copy link
Member

@rossabaker rossabaker commented May 27, 2017

readInputStream recycles the buffer that backs the chunk. If we do any storage or concurrent processing down stream, the chunk mutates out from underneath us.

When reading an inputStream, we'll upgrade to ByteVectorChunk, which is immutable and safe for downstream consumption.

Fixes #1201 /cc @jarreds

@jarreds
Copy link

@jarreds jarreds commented May 27, 2017

Unfortunate to have to copy the bytes, but sounds like there is no other option at the moment.

+1

@rossabaker
Copy link
Member Author

@rossabaker rossabaker commented May 27, 2017

We could also solve this on the writer side by copying only when we buffer. But then we'd be copying for all sources, not just those derived from an InputStream. Furthermore, anything else that consumes from readInputStream in any complicated way could reintroduce the bug. I thought it was best to cut of the mutability at the source.

I think changes to fs2 could make this more efficient.

@jarreds
Copy link

@jarreds jarreds commented May 27, 2017

I'm with you on stopping near source. Thank you.

Copy link
Member

@ChristopherDavenport ChristopherDavenport left a comment

Good fix to the problem at hand. 👍

Has a test reproducing the bug, solves said problem, copying is done once, and has Latin words to be extra fancy.

@rossabaker rossabaker merged commit fb5b61a into http4s:master May 27, 2017
1 check failed
1 check failed
continuous-integration/travis-ci/pr The Travis CI build failed
Details
@rossabaker rossabaker deleted the rossabaker:master branch May 27, 2017
aeons added a commit to aeons/http4s that referenced this pull request Jun 15, 2017
rossabaker added a commit to rossabaker/http4s that referenced this pull request Dec 17, 2017
It seems we had a bad merge after http4s#1197 and it came back.
aeons added a commit that referenced this pull request Dec 18, 2017
Remove the buffer copying hack from #1202
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants
You can’t perform that action at this time.