Add XMSS/XMSSMT wolfCrypt hooks. #6840

philljj · 2023-10-05T14:27:44Z

Description

This is a companion PR to:

Adds XMSS/XMSS^MT hooks to wolfCrypt. This is achieved by integration with a patched xmss-reference.

This PR adds:

XMSS/XMSS^MT hooks support, built with --enable-xmss --with-libxmss=<path to patched xmss src>.
XMSS/XMSS^MT verify only build, enabled with --enable-xmss=verify-only instead.
Added xmss tests to wolfcrypt/test/test.c for both normal and verify-only builds.
Added xmss benchmark to wolfcrypt/benchmark/benchmark.c.
Updated INSTALL item with instructions for building xmss-hooks (see item 20: 20. Building with xmss-reference lib for XMSS/XMSS^MT support [EXPERIMENTAL]).

Supported xmss parameters

This supports all SHA256 parameter sets from RFC 8391 and NIST SP 800-208.

A table showing the XMSS parameter sets was added in the documentation PR here:

wolfSSL/src/appendix07.md#supported-parameters-1
Add XMSS/XMSS^MT hooks documentation. documentation#110

Also, see wolfssl/wolfcrypt/xmss.h for additional info.

Building

See item 20 of the INSTALL file:

20. Building with xmss-reference lib for XMSS/XMSS^MT support [EXPERIMENTAL]

    Experimental support for XMSS/XMSS^MT has been achieved by integration
    with the xmss-reference implementation from RFC 8391 (XMSS: eXtended
    Merkle Signature Scheme). We support a patched version of xmss-reference
    based on this git commit
    ...

Documentation

Opened a XMSS/XMSS^MT documentation pull request:

Add XMSS/XMSS^MT hooks documentation. documentation#110

Benchmark

The patch updates xmss-reference to use wolfCrypt SHA256 for hashing, and benefits from asm speedups. See the documentation PR for full x86_64 and aarch64 benchmarking.

Benchmark with --enable-intelasm:

$./wolfcrypt/benchmark/benchmark -xmss_xmssmt
------------------------------------------------------------------------------
 wolfSSL version 5.6.3
------------------------------------------------------------------------------
Math: 	Multi-Precision: Wolf(SP) word-size=64 bits=4096 sp_int.c
wolfCrypt Benchmark (block bytes 1048576, min 1.0 sec each)
XMSS-SHA2_10_256  2500     sign       300 ops took 1.650 sec, avg 5.501 ms, 181.774 ops/sec
XMSS-SHA2_10_256  2500   verify      1600 ops took 1.035 sec, avg 0.647 ms, 1545.452 ops/sec
XMSSMT-SHA2_20/2_256  4963     sign       200 ops took 1.152 sec, avg 5.760 ms, 173.617 ops/sec
XMSSMT-SHA2_20/2_256  4963   verify       900 ops took 1.099 sec, avg 1.222 ms, 818.595 ops/sec
XMSSMT-SHA2_20/4_256  9251     sign       300 ops took 1.151 sec, avg 3.836 ms, 260.674 ops/sec
XMSSMT-SHA2_20/4_256  9251   verify       500 ops took 1.239 sec, avg 2.478 ms, 403.509 ops/sec
XMSSMT-SHA2_40/4_256  9893     sign       200 ops took 1.343 sec, avg 6.715 ms, 148.916 ops/sec
XMSSMT-SHA2_40/4_256  9893   verify       500 ops took 1.155 sec, avg 2.309 ms, 433.027 ops/sec
XMSSMT-SHA2_40/8_256 18469     sign       300 ops took 1.119 sec, avg 3.729 ms, 268.175 ops/sec
XMSSMT-SHA2_40/8_256 18469   verify       300 ops took 1.494 sec, avg 4.980 ms, 200.801 ops/sec
XMSSMT-SHA2_60/6_256 14824     sign       200 ops took 1.473 sec, avg 7.364 ms, 135.796 ops/sec
XMSSMT-SHA2_60/6_256 14824   verify       300 ops took 1.107 sec, avg 3.690 ms, 271.002 ops/sec
XMSSMT-SHA2_60/12_256 27688     sign       300 ops took 1.086 sec, avg 3.619 ms, 276.343 ops/sec
XMSSMT-SHA2_60/12_256 27688   verify       200 ops took 1.464 sec, avg 7.321 ms, 136.585 ops/sec
Benchmark complete

Benchmark without intelasm:

$./wolfcrypt/benchmark/benchmark -xmss_xmssmt
------------------------------------------------------------------------------
 wolfSSL version 5.6.3
------------------------------------------------------------------------------
Math: 	Multi-Precision: Wolf(SP) word-size=64 bits=4096 sp_int.c
wolfCrypt Benchmark (block bytes 1048576, min 1.0 sec each)
XMSS-SHA2_10_256  2500     sign       200 ops took 1.317 sec, avg 6.586 ms, 151.843 ops/sec
XMSS-SHA2_10_256  2500   verify      1100 ops took 1.037 sec, avg 0.943 ms, 1060.798 ops/sec
XMSSMT-SHA2_20/2_256  4963     sign       200 ops took 1.669 sec, avg 8.344 ms, 119.854 ops/sec
XMSSMT-SHA2_20/2_256  4963   verify       600 ops took 1.114 sec, avg 1.857 ms, 538.545 ops/sec
XMSSMT-SHA2_20/4_256  9251     sign       200 ops took 1.103 sec, avg 5.515 ms, 181.319 ops/sec
XMSSMT-SHA2_20/4_256  9251   verify       300 ops took 1.039 sec, avg 3.464 ms, 288.694 ops/sec
XMSSMT-SHA2_40/4_256  9893     sign       200 ops took 1.945 sec, avg 9.723 ms, 102.853 ops/sec
XMSSMT-SHA2_40/4_256  9893   verify       300 ops took 1.095 sec, avg 3.651 ms, 273.872 ops/sec
XMSSMT-SHA2_40/8_256 18469     sign       200 ops took 1.100 sec, avg 5.500 ms, 181.826 ops/sec
XMSSMT-SHA2_40/8_256 18469   verify       200 ops took 1.391 sec, avg 6.956 ms, 143.768 ops/sec
XMSSMT-SHA2_60/6_256 14824     sign       100 ops took 1.092 sec, avg 10.924 ms, 91.544 ops/sec
XMSSMT-SHA2_60/6_256 14824   verify       200 ops took 1.084 sec, avg 5.421 ms, 184.459 ops/sec
XMSSMT-SHA2_60/12_256 27688     sign       200 ops took 1.136 sec, avg 5.679 ms, 176.102 ops/sec
XMSSMT-SHA2_60/12_256 27688   verify       100 ops took 1.096 sec, avg 10.958 ms, 91.258 ops/sec
Benchmark complete

philljj · 2023-10-06T19:51:26Z

The way I'm building xmss with wolfcrypt is breaking our dependency tracking logic. I will fix it to be more like LMS.

anhu · 2023-10-11T19:54:59Z

INSTALL

+    To build verify-only patched xmss-reference:
+      $ make xmss_verify_lib.a
+
+    Note that this patch adds wolfCrypt SHA256 hashing to xmss-reference, and


Note that this patch changes xmss-reference to use wolfCrypt SHA256 hashing, and ....

anhu · 2023-10-11T19:55:33Z

INSTALL

+    Note that this patch adds wolfCrypt SHA256 hashing to xmss-reference, and
+    thus benefits from all the same asm speedups as wolfCrypt SHA hashing.
+    Depending on architecture you may build with --enable-intelasm, or
+    and --enable-armasm, and see 30-40% speedups in XMSS/XMSS^MT.


first "and" should be removed.

anhu · 2023-10-11T20:02:23Z

INSTALL

+
+    For full keygen, signing, verifying, and benchmarking support, build
+    wolfSSL with:
+      $ ./configure \


wait...if xmss uses wolfssl for sha, then why doesn't it need to have wolfSSL around first? Is it done by registering callbacks? If so, just a short sentence about how its done would be nice.

Correct, by registering a sha callback. Good point, I'll update these comments.

anhu · 2023-10-11T20:05:05Z

wolfcrypt/benchmark/benchmark.c

+    /* All NIST SP 800-208 approved SHA256 XMSS/XMSS^MT parameter
+     * sets.
+     *
+     * note: not testing "XMSS-SHA2_16_256", "XMSS-SHA2_20_256",


N in note should be capitalized.

anhu · 2023-10-11T20:07:05Z

wolfcrypt/src/ext_xmss.c

+    return 0;
+}
+
+/* Sets the Xmss key parameters, given an oid.


In comments I think all instances of "oid" should be "OID" as its an acronym.

anhu · 2023-10-11T20:08:11Z

wolfcrypt/src/ext_xmss.c

+    return 0;
+}
+
+/* Set the Xmss key parameter string.


Xmss should be XMSS in comments. (all caps) here and in other places.

anhu · 2023-10-12T20:29:24Z

This looks good so far but I noticed it depends on samples repo. Can you assign me as review on that as well?

anhu · 2023-10-12T20:33:36Z

Took it for a spin and got

*** Warning: Linking the shared library src/libwolfssl.la against the
*** static library /home/anthony/code/xmss/xmss-reference/xmss_lib.a is not portable!

Not sure if this is okay or not.

philljj · 2023-10-13T18:55:11Z

Took it for a spin and got

*** Warning: Linking the shared library src/libwolfssl.la against the
*** static library /home/anthony/code/xmss/xmss-reference/xmss_lib.a is not portable!

Not sure if this is okay or not.

Correct, this and the LMS --with-liblms= will give this warning. The reason is we are giving a specific path to link the static lib, rather than a generic $(PATH) that would be portable across systems. We could require users install the patched xmss-reference to /usr/local, but these projects do not have install rules and I haven't patched them to be added yet. IMO this is fine for the purposes of demonstrating a test integration.

The wolfBoot LMS and XMSS builds do not have this warning, as they do not build libwolfssl, but rather directly link against the wolfcrypt objects needed.

wolfcrypt/src/ext_xmss.c

…view.

Add XMSS/XMSSMT wolfCrypt hooks.

33d4b33

philljj self-assigned this Oct 5, 2023

philljj mentioned this pull request Oct 5, 2023

Example for xmss hooks support. wolfSSL/wolfssl-examples#403

Merged

XMSS/XMSSMT hooks support: fix build.

f32c6a0

philljj requested a review from anhu October 9, 2023 16:46

philljj assigned anhu Oct 9, 2023

anhu requested changes Oct 11, 2023

View reviewed changes

XMSS/XMSSMT hooks support: small update for requested changes.

60fea5e

philljj requested a review from anhu October 12, 2023 06:33

anhu previously approved these changes Oct 13, 2023

View reviewed changes

philljj assigned wolfSSL-Bot and unassigned anhu and philljj Oct 13, 2023

JacobBarthelmeh requested changes Oct 13, 2023

View reviewed changes

wolfcrypt/src/ext_xmss.c Outdated Show resolved Hide resolved

wolfcrypt/src/ext_xmss.c Show resolved Hide resolved

JacobBarthelmeh assigned philljj and unassigned wolfSSL-Bot Oct 13, 2023

XMSS/XMSSMT hooks support: fix g++ warnings, and small cleanup for re…

46b1a03

…view.

philljj dismissed anhu’s stale review via 46b1a03 October 13, 2023 23:06

philljj mentioned this pull request Oct 13, 2023

xmss_example: tiny cleanup to match g++ warning fix. wolfSSL/wolfssl-examples#407

Merged

philljj requested a review from JacobBarthelmeh October 14, 2023 00:03

philljj assigned wolfSSL-Bot and unassigned philljj Oct 14, 2023

JacobBarthelmeh approved these changes Oct 16, 2023

View reviewed changes

JacobBarthelmeh merged commit d351120 into wolfSSL:master Oct 16, 2023
103 checks passed

philljj deleted the xmss_hooks_support branch October 26, 2023 20:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add XMSS/XMSSMT wolfCrypt hooks. #6840

Add XMSS/XMSSMT wolfCrypt hooks. #6840

philljj commented Oct 5, 2023 •

edited

philljj commented Oct 6, 2023

anhu Oct 11, 2023

anhu Oct 11, 2023

anhu Oct 11, 2023

philljj Oct 11, 2023

anhu Oct 11, 2023

anhu Oct 11, 2023

anhu Oct 11, 2023

anhu commented Oct 12, 2023

anhu commented Oct 12, 2023

philljj commented Oct 13, 2023

Add XMSS/XMSSMT wolfCrypt hooks. #6840

Add XMSS/XMSSMT wolfCrypt hooks. #6840

Conversation

philljj commented Oct 5, 2023 • edited

Description

Supported xmss parameters

Building

Documentation

Benchmark

philljj commented Oct 6, 2023

anhu Oct 11, 2023

Choose a reason for hiding this comment

anhu Oct 11, 2023

Choose a reason for hiding this comment

anhu Oct 11, 2023

Choose a reason for hiding this comment

philljj Oct 11, 2023

Choose a reason for hiding this comment

anhu Oct 11, 2023

Choose a reason for hiding this comment

anhu Oct 11, 2023

Choose a reason for hiding this comment

anhu Oct 11, 2023

Choose a reason for hiding this comment

anhu commented Oct 12, 2023

anhu commented Oct 12, 2023

philljj commented Oct 13, 2023

philljj commented Oct 5, 2023 •

edited