storage: use sha256-simd from minio #347
Conversation
Signed-off-by: Ramkumar Chinchani <rchincha@cisco.com>
Codecov Report
@@ Coverage Diff @@
## main #347 +/- ##
=======================================
Coverage 80.28% 80.28%
=======================================
Files 46 46
Lines 6294 6296 +2
=======================================
+ Hits 5053 5055 +2
Misses 873 873
Partials 368 368
Continue to review full report at Codecov.
|
|
Linux kernel already has included support for this. |
There was a problem hiding this comment.
LGTM
I did not have CPU support for this new feature on my VM (both locally and in the lab). The improvement is working only on bare metal? Did you have chance to try this on a machine that supports the feature and does the benchmarking shows a huge improvement for large blobs? Currently it takes between 3-4 sec to compute hash for blob with size 1000MB (includes the reading of the file which I guess takes a lot) using crypto/sha256 (fallback from sha256-simd without CPU SHA features support). Maybe there is still room for improvements here like caching the digest when the blob is first uploaded into .uploads (for systems that does not have CPU support for sha acceleration).
adodon@adodon-cs-bld:/data/ssd/work/benchmark$ go test -bench=. -benchtime=60s
goos: linux
goarch: amd64
pkg: github.com/benchmark
cpu: Intel(R) Xeon(R) Gold 5218 CPU @ 2.30GHz
BenchmarkNewSha100MB-20 205 350024972 ns/op
BenchmarkNewSha1000MB-20 19 3503338046 ns/op
PASS
ok github.com/benchmark 177.248sadodon@adodon-cs-bld:/data/ssd/work/benchmark$ go test -bench=. -benchtime=60s
goos: linux
goarch: amd64
pkg: github.com/benchmark
cpu: Intel(R) Xeon(R) Gold 5218 CPU @ 2.30GHz
BenchmarkNewSha100MB-20 205 350024972 ns/op
BenchmarkNewSha1000MB-20 19 3503338046 ns/op
PASS
ok github.com/benchmark 177.248s
|
#347 (review) |
…roject-zot#347) * test(test-data): add layers information to the image metadata json Signed-off-by: Andrei Aaron <aaaron@luxoft.com> * fix(tests): fix username userd as password, fix prerequisite validation Signed-off-by: Andrei Aaron <aaaron@luxoft.com> * fix(tests): auto-confirm cosign upload to private registry Signed-off-by: Andrei Aaron <aaaron@luxoft.com> --------- Signed-off-by: Andrei Aaron <aaaron@luxoft.com>
Signed-off-by: Ramkumar Chinchani rchincha@cisco.com
What type of PR is this?
performance enchancement
Which issue does this PR fix:
#306
What does this PR do / Why do we need it:
Uses the sha_ni extensions for CPUs that support it.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.