Uses deprecated OpenSSL APIs

[rossburton]

OpenSSL 3 has deprecated a fair chunk of the EVF APIs. For example:

elf/icf.cc:234:18: warning: ‘int SHA256_Update(SHA256_CTX*, const void*, size_t)’ is deprecated: Since OpenSSL 3.0 [-Wdeprecated-declarations]
  234 |     SHA256_Update(&sha, &val, sizeof(val));
      |     ~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~

The non-deprecated APIs existed in OpenSSL 1.1, so you can migrate to the new API without breaking the build.

[rui314]

It feels like they are deprecating functions that don't have to be deprecated... Anyways, I'll migrate to the new API.

[rui314]

They deprecated SHA256_CTX struct, so we now have to allocate a context object using EVP_MD_CTX_new. I.e. we can't allocate a context object to the stack anymore. It looks like it significantly degrades the mold's Identical Comdat Folding pass. So we can't migrate to the new API.

If we have to migrate, we should consider switching to other library to compute SHA256 hashes.

[kleunen]

If you just use SHA256, maybe just incorporate this calculation.

Maybe library such as:
https://github.com/okdshin/PicoSHA2

[rui314]

@kleunen SHA256 performance is critical for us. x86 processors have special-purpose instructions for SHA1/SHA256 (https://en.wikipedia.org/wiki/Intel_SHA_extensions), but that implementation doesn't use it. It doesn't seem to even use SIMD instructions. So it is probably not as fast as the OpenSSL's implementation.

[matu3ba]

@rui314 You may want to keep a look onto HACL or contact them about the status of the (SHA-256) implementations ie asking for an estimate when it becomes usable.

Their benchmarks in the publication showed a 5x speedup against the openssl implementation, though it can vary significantly.

[mikejsavage]

FYI blake2b is quite a bit faster than SHA2 with multiple standalone vectorised implementations available

[rui314]

@mikejsavage Is there a benchmark comparison? I tried blake3 before but it wasn't as fast as hardware-accelerated sha256 on my AMD Threadripper machine.

[mikejsavage]

I only know about the benchmarks on the blake website

blake3 is supposed to be faster again than blake2b, so it's odd that you saw the opposite!

[matu3ba]

Some Intel and ARM processors support native SHA-2 instructions, and using these instructions can provide much better performance than vectorization. On Amazon Graviton, for instance, OpenSSL assembly uses hardware SHA instructions and is by far the fastest implementation.

@mikejsavage Benchmarks for blake3 show only amd64 and in folder benches/bench.rs only target arch x86_64 is explicitly given. So to me it looks like they did not optimize or measure other architectures yet (even for the Rust version with multithreading support).

Take here a user comment on performance of blake3 on AMD being on pair (slightly slower than sha256) due to no vectorization.

[mikejsavage]

blake2b has a NEON implementation: https://github.com/BLAKE2/BLAKE2/tree/master/neon

For blake3 all the implementations are in C and they have a NEON impl too, the rust implementation is just a wrapper around the C API

[rui314]

Note that I'm not actively looking for faster crypto hash algorithms at this moment. Faster algorithms are always welcome, but for now I just want to silence the warning they introduced to OpenSSL 3. We can continue ignoring these deprecation warnings though.

[cryptomilk]

Maybe just use nettle. It is a low level crypto library and is used by GnuTLS, which means it is available on every distribution already:

https://www.lysator.liu.se/~nisse/nettle/nettle.html#SHA256

[joshcangit]

@matu3ba There's another also actively developed library other than HACL that is blst.

I thought maybe libsodium could be used but then it's software based. Whether it's blake2b or blake3, I doubt it would be fast enough.

libcrypto++/cryptopp should be better.
Version 8.6.0, the latest as of now, is available on Ubuntu 22.04 which is scheduled to release April 21^st this year. That is also available on Termux.
As for benchmarks, it's a bit slower than OpenSSL.

[oconnor663]

the rust implementation is just a wrapper around the C API

Just quickly chiming in to mention that this isn't entirely correct. In its default x86-64 build configuration, the Rust implementation of BLAKE3 depends on assembly files that are shared with the C implementation, but it doesn't depend on any C code. (We do currently use C code for NEON and for some less common build configurations.)