Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

plumbing: packfile, Add a buffer to crc writer #724

Merged
merged 2 commits into from Jan 21, 2018

Conversation

Projects
None yet
2 participants
@jfontan
Copy link
Member

jfontan commented Jan 19, 2018

crc update with block smaller than 16 bytes uses a slower version of the
function. ReadByte is heavily used by zlib inflate so most of the time
crc is updated byte by byte.

A new Flush method is added to the scanner to flush this crc writer
cache. It is only called when the Scanner reader is a teeReader.

Times to push localy from different repositories:

repo unbuffered buffered
cangallo 139.21ms 152.13ms
octoprint-tft 2.84s 2.92s
numpy 47.29s 44.94s
tensorflow 2m17.94s 1m56.92s
bismuth 8m13.79s 7m01.46s

I will try to make more tests with small repos if needed. The numbers in these repos changed a lot when using the laptop to test.

@jfontan jfontan requested review from ajnavarro and mcuadros Jan 19, 2018

@mcuadros

This comment has been minimized.

Copy link
Member

mcuadros commented Jan 20, 2018

@jfontan please read CONTRIBUTING.md

@jfontan jfontan force-pushed the jfontan:improvement/buffered-tee-reader branch from 429f9eb to 2da5738 Jan 21, 2018

@jfontan jfontan changed the title Add a buffer to crc writer plumbing: packfile, Add a buffer to crc writer Jan 21, 2018

@jfontan

This comment has been minimized.

Copy link
Member Author

jfontan commented Jan 21, 2018

I've updated commit message to conform to contributing guidelines. An extra commit was added with explicit checking of CRC calculated by the scanner.

jfontan added some commits Jan 19, 2018

plumbing: packfile, Add a buffer to crc writer.
crc update with block smaller than 16 bytes uses a slower version of the
function. ReadByte is heavily used by zlib inflate so most of the time
crc is update byte by byte.

A new Flush method is added to the scanner to flush this crc writer
cache. It is only called when the Scanner reader is a teeReader.

Signed-off-by: Javi Fontan <jfontan@gmail.com>
plumbing: packfile, Add crc check to scanner test.
Signed-off-by: Javi Fontan <jfontan@gmail.com>

@jfontan jfontan force-pushed the jfontan:improvement/buffered-tee-reader branch from 2da5738 to 1cb896b Jan 21, 2018

@mcuadros mcuadros merged commit 834cd6f into src-d:master Jan 21, 2018

3 checks passed

codecov/project 77.19% (+0.02%) compared to f6aca08
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.