SHA-1 x86

[MarekKnapek]

Hi @sjaeckel , not sure where to open a discussion. Issue or pull request? I implemented SHA-1 accelerated by x86 intrinsics.

But, for that I would need some support from LibTomCrypt architecture:

• How to detect at compile-time what processor architecture are we compiling for? Is it x86 or x64 or some other one?
• How to detect compiler used? Is it GCC? Or clang? Or MSVC? Or something else which I do not know how to write intrinsics for?
• How to detect SHA-1 (and SSE2 and SSSE3 and SSE4.1) instruction set presence at run-time? Use the cpuid instruction and cache its result. But where? Is there a precedent already in the lib?
• How to detect the ability to use alignas or pragma pack(n) or something to force variable alignment to 16 bytes?
• How to add a new file into all those make, cmake and other build systems? Because I have no idea.

My idea is to have two implementations of SHA-1 ("sha-1-portable" and "sha-1-x86") in parallel at once in the lib (if the user wants to). Then have third, generic "sha-1" one, which "just" dispatches its logic into one of the previous ones, depending on CPUID results.

My code is at: https://github.com/MarekKnapek/libtomcrypt/commits/SHA-1x86/.

[sjaeckel]

Hey hey,

did you already have a look at #714 (and maybe also #557, but that PR itself was not optimal and some parts, especially regarding the instruction set checks etc., were heavily changed after it has been merged)? Both combined should contain the answers to most of your questions.

How to add a new file into all those make, cmake and other build systems?

First you add the file to the tree, then you call ./helper.pl --update-makefiles. This will update all the makefiles etc. (There's also ./updatemakes.sh which has a -c option to commit only the makefiles. NB: Please first commit your source code and commit the makefiles in a separate step afterwards.

You should also call ./helper.pl -a which will run some sanity checks around the code which would also be flagged in CI, so you can catch those cases early.

My idea is to have two implementations of SHA-1 ("sha-1-portable" and "sha-1-x86") in parallel at once in the lib (if the user wants to). Then have third, generic "sha-1" one, which "just" dispatches its logic into one of the previous ones, depending on CPUID results.

That's roughly also what we did in the past. The name in all of the descriptors should be "sha1" and there should be three instances of ltc_hash_descriptor.

sha1_desc.c:
/* This file provides the old `sha1_xx` API and does the multiplexing to the correct implementation */
const struct ltc_hash_descriptor sha1_desc;
sha1.c:
/* This file provides the old `sha1` implementation, but under a new API, which I'd say should be prefixed as sha1_c_* */
const struct ltc_hash_descriptor sha1_portable_desc;
sha1_x86.c:
/* This file provides the new x86 instrinsics based `sha1` implementation under a new API, which you already prefixed as sha1_x86_* */
const struct ltc_hash_descriptor sha1_x86_desc

This mimics roughly what we did in #557.

The most complete runtime check of CPU support comes from #714

libtomcrypt/src/encauth/gcm/gcm_gf_mult.c

Lines 21 to 46 in 3223b87

	static LTC_INLINE int s_pclmul_is_supported(void)
	{
	static int initialized = 0, is_supported = 0;
	if (initialized == 0) {
	/* Test CPUID.1.0.ECX[1]
	* EAX = 1, ECX = 0 */
	#if defined(_MSC_VER)
	int cpuInfo[4];
	__cpuid(cpuInfo, 1);
	is_supported = ((cpuInfo[2] >> 1) & 1);
	#else
	int a = 1 , b, c = 0, d;
	asm volatile ("cpuid"
	:"=a"(a), "=b"(b), "=c"(c), "=d"(d)
	:"a"(a), "c"(c)
	);
	is_supported = ((c >> 1) & 1);
	initialized = 1;
	#endif
	}
	return is_supported;
	}

I'd suggest you copy that once more and modify it up to your requirements. In a second step then, after this PR has been merged, we check if this can somehow be generalized into a shared implementation that isn't too weird to use ;)

I'm also going to add some comments to your code, which already looks pretty nice :)

Please feel free to ask more if something is unclear!

[sjaeckel]

I'm also going to add some comments to your code, which already looks pretty nice :)

Sorry, but after the first comment I got reminded that GitHub is now a part of Microslop and we're happy if anything works, but comments to commits apparently don't work anymore ... Please open a PR with your changes as they currently are and I'll add them directly to the PR.

not sure where to open a discussion. Issue or pull request?

Therefore: pull request

[MarekKnapek]

Created #717.