Add support for SHA3 (Keccak) #20342
Comments
|
It's been on the watch list since it was announced. Since we don't implement cryptographic algorithms within .NET we're waiting on support from the underlying platforms (Windows CNG, Apple Security.framework, and OpenSSL). As far as speed, I'm given to understand that SHA-3 can never be as fast as SHA-2, due to an inability to hardware optimize it. |
|
Hello @bartonjs： Thanks for your quick reply, and I get the complete code from here: http://keccak.noekeon.org/KeccakReferenceAndOptimized-3.2.zip. For more about speed and other things, please read here：http://www.drdobbs.com/security/keccak-the-new-sha-3-encryption-standard/240154037?pgno=1 |
|
Yeah, Keccak is (per your linked article) 12.5 cycles/byte. SHA-2-512 with 64-bit instructions on an Intel processor is 8.5 cycles/byte. That said, we're just at the mercy of
|
|
Well……I think you can rename it as "Keccak256,384 or 512", because Keccak is just the algorithm of SHA3 as the standard one. A persudo code may look like this following:
Also something like for these providers: Keccak256CrytoServiceProvider |
|
Sorry to jump in the thread, but I would suggest to consider the SHAKE extendable-output functions (XOF) from FIPS 202, or the cSHAKE from SP 800-185. They are more flexible than the plain SHA-3 hash functions and have about the same speed as SHA-2. For higher speed, there is ParallelHash [SP 800-185] or KangarooTwelve. https://twitter.com/KeccakTeam/status/834789451708628995 Kind regards, |
|
|
|
Based on @gvanas 's points, I'd be interested to see an API design that keeps those other functions in mind. Maybe that would even affect the naming question. |
|
@JonHanna：Agree. The name is something like the inner class that isn't publicly published to the public to be used, it's really a bit strange for us. Maybe this can be used as a wrapper or something else like this dynamically generated. And for us, the published one can be something like what I've mentioned above in my post. |
danmoseley
changed the title
|
The naming The proper name according to the capitalization convention would be |
|
My port of Keccak (including both original and FIPS 202 SHA3 padding) integrates with HashAlgorithm as the others in that namespace. I would be happy to contribute it. |
|
@jdluzen Nice! Ideally, your code should also support the SHAKE and cSHAKE functions. Not much extra work, the underlying function is the same. |
|
@jdluzen, we appreciate the offer, but we have a strict policy of only using cryptographic algorithms provided by underlying platforms (CNG on Windows, OpenSSL on Linux, Apple Crypto on Mac). That ensures we don't have to worry about provable correctness or avoiding side channels. Right now, I think OpenSSL has SHA3, but not the other two platforms. |
|
Well, we could implement the SHA3 API and just throw a "PlatformNotSupportedException" exception, when running on Mac or Windows. |
@Spacefish we generally try to avoid this, as it makes it harder to write cross platform libraries if an API that seems relevant to all platforms only works on some. |
|
As far as I can see, there are several free implementations (at least BouncyCastle and the one by @jdluzen mentioned above), so users have alternatives as long as it's not supported in the Framework. |
|
@bartonjs Is SHA3 realistically something that we're going to be able to support in the near-to-intermediate future? Should we resolve this issue for now and reopen it once there's a critical mass of support? |
maryamariyan
added
the
untriaged
|
(Just a heads up; I don't feel as strongly as the wording I've written here implies, this one's a little cynical.) You might also want to consider the impact on driving adoption in the direction opposite vendor support to encourage vendor support by creating adopters by creating the API. When SHA3 is suggested and no implementation is present the framework, at present (emphasis on it's 2020) many architects opt for SHA2 and concrete their architectures around it. Those that do opt to use SHA3 end up using OpenSSL/BouncyCastle/etc. or rolling their own, which is perfectly fine, and 'demand' is met and none is perceived by the OS vendors or framework. OS vendor implementation appears to be driven by demand, of which dependent frameworks are the prime mover. So then .NET and Windows CNG are both waiting on each other. (Or do you think there are relatively many that use Windows CNG directly instead of as a supplment or fallback?) Microsoft has their own PQC initiative, sans hashing algo at present. Wait much longer, the question won't be "Where's SHA3?" it'll be "Where's PQSHA1?" Microsoft added XTS mode for AES in Windows 10, though I honestly have no idea if you can use it via System.Security.Cryptography. 🤷 I think the general consensus of those still using SHA2 is vaguely "the world didn’t move to SHA3 because almost none of the world’s software or hardware supported it" ... which isn't true in the technical sense of software and hardware, only true in the business sense of software and hardware. Since you're both (OS vendor and framework) waiting on demand that isn't measured by uses of the algo. in general or whether the algo. should be used, or as some sort of leading example w.r.t ASP.NET Core, you may as well do the industry a favor just declare the whole namespace legacy support and encourage the use of some 3rd party solutions for legitimate up-to-date solutions or create a 1st party .NET crypto lib solution, but you probably can't for some reason that involves recursion. We'll call it the SHA3 pickle. edit: So no Keccak256Managed/SHA3b256Managed then? |
|
.NET 5.0 is here! SHA3 has been added? Linked to: |
|
If anyone is interested, I have implemented SHA3 + SHAKE variants here... https://github.com/MrMatthewLayton/CORE/tree/master/Core/Security/Cryptography |
|
@Xyncgas srsly, stop spamming, it is helping no-one.. |
|
@Xyncgas Disparaging remarks about other contributors are not welcome. Please review our code of conduct (https://dotnetfoundation.org/about/code-of-conduct) for more information. |
|
With Windows SHA-3 Support now announced, we will move forward with this proposal and include it in .NET 8. |
|
@jeffhandley Excellent news! Do you know whether the same APIs may be available for Linux and macOS? |
One would assume that, as it would be quite trivial to support it there. The main reason new Crypto is hold back in the .NET standard libs is native platform support in windows. The .NET devs have been hesitant to support Crypto Algorithms in the .NET standard libs which would return "Plaform Not Supported" on Windows. Probably for obvious reasons.. You can always use bouncy castle and or openssl via NuGet packages on windows as well to get SHA3 today. |
|
@Spacefish or, I could just use my own ;) #20342 (comment) That is, until they're provided in .NET, in which case I will probably refactor my own APIs to use the platform provided API, and fall back to my own in the event that platform not supported exceptions are thrown. |
|
The top post is updated with the proposal for SHA3. |
bartonjs
added
api-ready-for-review
This comment was marked as outdated.
This comment was marked as outdated.
bartonjs
added
api-approved
vcsjones
added
api-ready-for-review
|
This is going back to review just to further discuss the |
|
We discussed removing the attributes and treating it similar to how we do the intrinsics... if there's an IsSupported it's up to you to know to check it (or to not check it because you have no fallback). We should similarly remove the OS-support attributes from Ed25519 types. namespace System.Security.Cryptography;
public partial struct HashAlgorithmName {
public static HashAlgorithmName SHA3_256 { get; }
public static HashAlgorithmName SHA3_384 { get; }
public static HashAlgorithmName SHA3_512 { get; }
}
public sealed partial class RSAEncryptionPadding {
public static RSAEncryptionPadding OaepSHA3_256 { get; }
public static RSAEncryptionPadding OaepSHA3_384 { get; }
public static RSAEncryptionPadding OaepSHA3_512 { get; }
}
public abstract partial class SHA3_256 : HashAlgorithm {
public const int HashSizeInBits = 256;
public const int HashSizeInBytes = 32;
protected SHA3_256();
public static bool IsSupported { get; }
public static new SHA3_256 Create();
public static byte[] HashData(byte[] source);
public static byte[] HashData(ReadOnlySpan<byte> source);
public static int HashData(ReadOnlySpan<byte> source, Span<byte> destination);
public static bool TryHashData(ReadOnlySpan<byte> source, Span<byte> destination, out int bytesWritten);
public static byte[] HashData(Stream source);
public static int HashData(Stream source, Span<byte> destination);
public static ValueTask<byte[]> HashDataAsync(Stream source, CancellationToken cancellationToken = default);
public static ValueTask<int> HashDataAsync(
Stream source,
Memory<byte> destination,
CancellationToken cancellationToken = default);
}
public abstract partial class SHA3_384 : HashAlgorithm {
public const int HashSizeInBits = 384;
public const int HashSizeInBytes = 48;
protected SHA3_384();
public static bool IsSupported { get; }
public static new SHA3_384 Create();
public static byte[] HashData(byte[] source);
public static byte[] HashData(ReadOnlySpan<byte> source);
public static int HashData(ReadOnlySpan<byte> source, Span<byte> destination);
public static bool TryHashData(ReadOnlySpan<byte> source, Span<byte> destination, out int bytesWritten);
public static byte[] HashData(Stream source);
public static int HashData(Stream source, Span<byte> destination);
public static ValueTask<byte[]> HashDataAsync(Stream source, CancellationToken cancellationToken = default);
public static ValueTask<int> HashDataAsync(
Stream source,
Memory<byte> destination,
CancellationToken cancellationToken = default);
}
public abstract partial class SHA3_512 : HashAlgorithm {
public const int HashSizeInBits = 512;
public const int HashSizeInBytes = 64;
protected SHA3_512();
public static bool IsSupported { get; }
public static new SHA3_512 Create();
public static byte[] HashData(byte[] source);
public static byte[] HashData(ReadOnlySpan<byte> source);
public static int HashData(ReadOnlySpan<byte> source, Span<byte> destination);
public static bool TryHashData(ReadOnlySpan<byte> source, Span<byte> destination, out int bytesWritten);
public static byte[] HashData(Stream source);
public static int HashData(Stream source, Span<byte> destination);
public static ValueTask<byte[]> HashDataAsync(Stream source, CancellationToken cancellationToken = default);
public static ValueTask<int> HashDataAsync(
Stream source,
Memory<byte> destination,
CancellationToken cancellationToken = default);
}
public partial class HMACSHA3_256 : HMAC {
public const int HashSizeInBits = 256;
public const int HashSizeInBytes = 32;
public HMACSHA3_256();
public HMACSHA3_256(byte[] key);
public static bool IsSupported { get; } // New
public static byte[] HashData(byte[] key, byte[] source);
public static byte[] HashData(ReadOnlySpan<byte> key, ReadOnlySpan<byte> source);
public static int HashData(ReadOnlySpan<byte> key, ReadOnlySpan<byte> source, Span<byte> destination);
public static bool TryHashData(
ReadOnlySpan<byte> key,
ReadOnlySpan<byte> source,
Span<byte> destination,
out int bytesWritten);
public static byte[] HashData(byte[] key, Stream source);
public static byte[] HashData(ReadOnlySpan<byte> key, Stream source);
public static int HashData(ReadOnlySpan<byte> key, Stream source, Span<byte> destination);
public static ValueTask<byte[]> HashDataAsync(byte[] key, Stream source, CancellationToken cancellationToken = default);
public static ValueTask<byte[]> HashDataAsync(ReadOnlyMemory<byte> key, Stream source, CancellationToken cancellationToken = default);
public static ValueTask<int> HashDataAsync(
ReadOnlyMemory<byte> key,
Stream source,
Memory<byte> destination,
CancellationToken cancellationToken = default);
}
public partial class HMACSHA3_384 : HMAC {
public const int HashSizeInBits = 384;
public const int HashSizeInBytes = 48;
public HMACSHA3_384();
public HMACSHA3_384(byte[] key);
public static bool IsSupported { get; } // New
public static byte[] HashData(byte[] key, byte[] source);
public static byte[] HashData(ReadOnlySpan<byte> key, ReadOnlySpan<byte> source);
public static int HashData(ReadOnlySpan<byte> key, ReadOnlySpan<byte> source, Span<byte> destination);
public static bool TryHashData(
ReadOnlySpan<byte> key,
ReadOnlySpan<byte> source,
Span<byte> destination,
out int bytesWritten);
public static byte[] HashData(byte[] key, Stream source);
public static byte[] HashData(ReadOnlySpan<byte> key, Stream source);
public static int HashData(ReadOnlySpan<byte> key, Stream source, Span<byte> destination);
public static ValueTask<byte[]> HashDataAsync(byte[] key, Stream source, CancellationToken cancellationToken = default);
public static ValueTask<byte[]> HashDataAsync(ReadOnlyMemory<byte> key, Stream source, CancellationToken cancellationToken = default);
public static ValueTask<int> HashDataAsync(
ReadOnlyMemory<byte> key,
Stream source,
Memory<byte> destination,
CancellationToken cancellationToken = default);
}
public partial class HMACSHA3_512 : HMAC {
public const int HashSizeInBits = 512;
public const int HashSizeInBytes = 64;
public HMACSHA3_512();
public HMACSHA3_512(byte[] key);
public static bool IsSupported { get; } // New
public static byte[] HashData(byte[] key, byte[] source);
public static byte[] HashData(ReadOnlySpan<byte> key, ReadOnlySpan<byte> source);
public static int HashData(ReadOnlySpan<byte> key, ReadOnlySpan<byte> source, Span<byte> destination);
public static bool TryHashData(
ReadOnlySpan<byte> key,
ReadOnlySpan<byte> source,
Span<byte> destination,
out int bytesWritten);
public static byte[] HashData(byte[] key, Stream source);
public static byte[] HashData(ReadOnlySpan<byte> key, Stream source);
public static int HashData(ReadOnlySpan<byte> key, Stream source, Span<byte> destination);
public static ValueTask<byte[]> HashDataAsync(byte[] key, Stream source, CancellationToken cancellationToken = default);
public static ValueTask<byte[]> HashDataAsync(ReadOnlyMemory<byte> key, Stream source, CancellationToken cancellationToken = default);
public static ValueTask<int> HashDataAsync(
ReadOnlyMemory<byte> key,
Stream source,
Memory<byte> destination,
CancellationToken cancellationToken = default);
}
public sealed partial class Shake128 : IDisposable {
public Shake128();
public static bool IsSupported { get; }
public void AppendData(byte[] data);
public void AppendData(ReadOnlySpan<byte> data);
public byte[] GetCurrentHash(int outputLength);
public void GetCurrentHash(Span<byte> destination);
public byte[] GetHashAndReset(int outputLength);
public void GetHashAndReset(Span<byte> destination);
public void Dispose();
public static byte[] HashData(byte[] source, int outputLength);
public static byte[] HashData(ReadOnlySpan<byte> source, int outputLength);
public static void HashData(ReadOnlySpan<byte> source, Span<byte> destination);
public static byte[] HashData(Stream source, int outputLength);
public static void HashData(Stream source, Span<byte> destination);
public static ValueTask HashDataAsync(Stream source, Memory<byte> destination, CancellationToken cancellationToken = default);
public static ValueTask<byte[]> HashDataAsync(Stream source, int outputLength, CancellationToken cancellationToken = default);
}
public sealed partial class Shake256 : IDisposable {
public Shake256();
public static bool IsSupported { get; }
public void AppendData(byte[] data);
public void AppendData(ReadOnlySpan<byte> data);
public byte[] GetCurrentHash(int outputLength);
public void GetCurrentHash(Span<byte> destination);
public byte[] GetHashAndReset(int outputLength);
public void GetHashAndReset(Span<byte> destination);
public void Dispose();
public static byte[] HashData(byte[] source, int outputLength);
public static byte[] HashData(ReadOnlySpan<byte> source, int outputLength);
public static void HashData(ReadOnlySpan<byte> source, Span<byte> destination);
public static byte[] HashData(Stream source, int outputLength);
public static void HashData(Stream source, Span<byte> destination);
public static ValueTask HashDataAsync(Stream source, Memory<byte> destination, CancellationToken cancellationToken = default);
public static ValueTask<byte[]> HashDataAsync(Stream source, int outputLength, CancellationToken cancellationToken = default);
} |
bartonjs
added
api-approved
|
SHA-3 has been merged, except for SHAKE128 and SHAKE256. I hope to get a pull request open for those in the next week or two. |
ghost
added
the
in-pr
ghost
removed
the
in-pr
|
@vcsjones: Thanks a lot for your work! Linked to: |
ghost
locked as resolved and limited conversation to collaborators
dotnet-policy-service
bot
added
the
in-pr
and others
Since SHA1 has been known as an "unsafe" algorithm, but now a safer algorithm called "SHA3" is created. So this algorithm should also be included here. Any plans or options for Microsoft now? As far as we see, SHA3 is a little faster than SHA2 and what's more——It's SAFER.
API Proposal v2
The original proposal was outfitted with many annotations for
[SupportedOSPlatform{Guard}]. We discussed this in the original review and made it a point to revisit this topic. The current thinking now is that we should simply omit the attributes.We have
IsSupportedproperties that folks should be using to determine if the platform is supported or not.The platform support may change over time. None of these APIs are inherently OS-specific. If macOS gives us SHA3 APIs we can use later, then we will do so, and the
IsSupportedis the source of truth for this information, not the platform name.The API below is the new proposal. It is the same as the previously approved API, but
[SupportedOSPlatform("XYZ")]and[SupportedOSPlatformGuard("XYZ")]have been removed.Old proposal
API Proposal
The text was updated successfully, but these errors were encountered: