curl_sha512_256: add workaround for NetBSD bug

[Karlson2k]

Based on @mkauf analysis and suggestion.

The details are here and here.

The macro EVP_MAX_MD_SIZE is not available (on my machine), so I used SHA512_256_DIGEST_SIZE * 2 as a size of the temporal array which should be enough.

[Karlson2k]

@mkauf, please check this PR.

[Karlson2k]

PR has been updated with fixed code formatting.

[mkauf]

Thanks! The code compiles on NetBSD 9.3 and test 1615 works.

If the NetBSD team fixes the bug, how long should curl keep the workaround?

[Karlson2k]

Thanks! The code compiles on NetBSD 9.3 and test 1615 works.

If the NetBSD team fixes the bug, how long should curl keep the workaround?

If/When NetBSD fixed the bug, we can identify NetBSD version by __NetBSD_Version__ (defined in <sys/param.h>) and limit the workaround only for earlier versions. If it would be fixed in ported OpenSSL, then fixed version can be detected by OPENSSL_VERSION_NUMBER

[mkauf]

@Karlson2k : Can you please change the commit message so that it does not mention me? e.g. just remove the "@".

Last time this happened I got a lot of notification mails from GitHub... many projects copy commits and when they push, mentioned users get notifications.

[Karlson2k]

@Karlson2k : Can you please change the commit message so that it does not mention me? e.g. just remove the "@".

Done.
No change in the code, only commit message updated.

[dfandrich]

I'm not sure we need to work around this issue. It's a security issue in NetBSD and (presumably), it will be fixed fairly quickly. Even if we keep it, I wouldn't keep it around past the next curl release.

[Karlson2k]

Failed tests 661 and 1541 with VS2010 seem to be unrelated.

[Karlson2k]

https://gnats.netbsd.org/cgi-bin/query-pr-single.pl?number=58039

This bug was fixed in NetBSD, but so far hasn't been pushed as update, so existing NetBSD installations still have this bug.
I think it worth to merge this PR. It is not large, not intrusive, limited to NetBSD only and ensures that curl and libcurl will not trigger segmentation failures on NetBSD.

[bagder]

Since this works around a crash in existing netbsd I think it is worth merging.

[riastradh]

FYI, we fixed this in the NetBSD 10.0 release that went out today, but it still affects all NetBSD 9.x versions.

You may want to explicit_memset to zero the part of tmp_digest that you don't use -- some of the noteworthy security properties of SHA-512/256 rely on keeping the other half of the output secret.