Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

XMSS RFC 8391 Update #1858

Merged
merged 8 commits into from May 24, 2019
Merged

XMSS RFC 8391 Update #1858

merged 8 commits into from May 24, 2019

Conversation

@mgierlings
Copy link
Collaborator

@mgierlings mgierlings commented Mar 15, 2019

Updates XMSS (draft status) to conform to RFC 8391

XMSS_SHAKE256_W16_H16 = 0x0b00000b,
XMSS_SHAKE256_W16_H20 = 0x0c00000c
XMSS_SHA2_10_256 = 0x00000001,
XMSS_SHA2_16_256 = 0x00000002,

This comment has been minimized.

@randombit

randombit Mar 15, 2019
Owner

So the format changed between draft-06 and RFC? :/

Is there any reasonable way we can continue supporting older keys/sigs? It seems this change would invalidate all existing uses.

At the very least we'll also want to use a different OID for keys so it is not possible to mix them.

This comment has been minimized.

@mgierlings

mgierlings Mar 15, 2019
Author Collaborator

Yes, unfortunately the OID format did change. There exists another, new draft status document of interest regarding XMSS Algorithm Identifiers for HSS and XMSS for Use in the Internet X.509 Public Key Infrastructure. Also somewhere down the road XMSS will likely be assigned an "official" OID. So this may not be the last incompatible change to XMSS we see. There is also a disclaimer in RFC 8391:

This document is not an Internet Standards Track specification; it is
published for informational purposes.

This document is a product of the Internet Research Task Force
(IRTF). The IRTF publishes the results of Internet-related research
and development activities. These results might not be suitable for
deployment. This RFC represents the consensus of the Crypto Forum
Research Group of the Internet Research Task Force (IRTF). Documents
approved for publication by the IRSG are not candidates for any level
of Internet Standard; see Section 2 of RFC 7841.

Information about the current status of this document, any errata,
and how to provide feedback on it may be obtained at
https://www.rfc-editor.org/info/rfc8391.

A possibility to support older formats would be to integrate a key conversion utility into the botan cli.

randombit added a commit that referenced this pull request Mar 15, 2019
@randombit
Copy link
Owner

@randombit randombit commented Mar 15, 2019

I sent out a warning letting anyone who is actively using current draft-06 support to let us know https://lists.randombit.net/pipermail/botan-devel/2019-March/002278.html and added a warning to the 2.10 release notes. If we don't hear from anyone by say May 1st, I say go ahead and merge this.

In the mean time we should still work to minimize disturbance:

  • Increase XMSS datestamp in info.txt
  • Change to a new OID arc (update also doc/oids.txt)
  • Maybe even change BOTAN_HAS_XMSS to BOTAN_HAS_XMSS_RFC8391?
@randombit
Copy link
Owner

@randombit randombit commented Apr 7, 2019

Haven't heard from anyone about this so likely this is good to merge, but lets wait until end of the month just to be sure.

@randombit randombit mentioned this pull request Apr 7, 2019
13 of 18 tasks
@randombit
Copy link
Owner

@randombit randombit commented Apr 19, 2019

@mgierlings Can you rebase to address the merge conflict?

@mgierlings
Copy link
Collaborator Author

@mgierlings mgierlings commented Apr 19, 2019

@randombit Will do. I also have another update to this PR in the pipeline, which I'll push soon.

@randombit
Copy link
Owner

@randombit randombit commented Apr 22, 2019

It looks like you merged master into your branch rather than rebasing, this makes history very hard to read. Please rebase instead.

@mgierlings mgierlings force-pushed the mgierlings:xmss_rfc8391 branch from 91983bc to 0c6a107 May 5, 2019
mgierlings added 6 commits Mar 3, 2019
Changes XMSS and XMSS WOTS algorithm names and OIDs to correspond
to RFC 8391.
- Replaces XMSS test vectors with new vectors that were
  generated using Bouncy Castle's XMSS implementation.
- Adjusts the XMSS test bench to recognize the new XMSS
  algorithm naming scheme.
Internally XMSS uses a 64 Bit type for the leaf index.
This patch removes the four leading zero bytes from the
XMSS leaf index and serializes it as a four byte value
as described in RFC 8391.

Test cases are adjusted accordingly.

The 64 Bit type is kept internally which potentially
allows for code reuse when implementing XMSS^MT on top
of the current XMSS code.
@mgierlings mgierlings force-pushed the mgierlings:xmss_rfc8391 branch from 0c6a107 to 6bbccff May 6, 2019
@mgierlings
Copy link
Collaborator Author

@mgierlings mgierlings commented May 9, 2019

@randombit The broken commit should be fixed. I'm not sure though what causes the softhsm error in some of the Travis CI builds.

@randombit
Copy link
Owner

@randombit randombit commented May 9, 2019

@mgierlings Thanks. I think the problem is bench_xmss in speed.cpp needs to be updated for the new param names.

@randombit
Copy link
Owner

@randombit randombit commented May 10, 2019

I don't think any change in performance was expected either way, but ftr I did check master vs this branch, they had only insignificant differences in sign and verify time.

@codecov-io
Copy link

@codecov-io codecov-io commented May 10, 2019

Codecov Report

Merging #1858 into master will decrease coverage by <.01%.
The diff coverage is 69.23%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1858      +/-   ##
==========================================
- Coverage   91.38%   91.37%   -0.01%     
==========================================
  Files         531      532       +1     
  Lines       56462    56556      +94     
  Branches     5877     5882       +5     
==========================================
+ Hits        51596    51677      +81     
- Misses       4866     4879      +13
Impacted Files Coverage Δ
src/lib/pubkey/xmss/xmss_publickey.cpp 94.11% <ø> (ø) ⬆️
src/lib/pubkey/xmss/xmss_signature_operation.cpp 100% <ø> (ø) ⬆️
src/lib/pubkey/xmss/xmss_wots_publickey.cpp 100% <ø> (ø) ⬆️
src/cli/speed.cpp 90.75% <ø> (ø) ⬆️
src/lib/pubkey/xmss/xmss_signature.cpp 100% <100%> (ø) ⬆️
src/lib/pubkey/pk_algs.cpp 94.81% <100%> (ø) ⬆️
src/lib/pubkey/xmss/xmss_privatekey.cpp 95.74% <100%> (ø) ⬆️
src/lib/pubkey/xmss/xmss_wots_parameters.cpp 86.11% <50%> (ø) ⬆️
src/tests/test_xmss.cpp 93.02% <50%> (ø) ⬆️
src/lib/pubkey/xmss/xmss_parameters.cpp 90% <69.44%> (ø) ⬆️
... and 16 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2c9caac...e390adc. Read the comment docs.

@randombit
Copy link
Owner

@randombit randombit commented May 10, 2019

@mgierlings I fixed the test issue in 10630cc. Is this PR ready to merge?

@mgierlings
Copy link
Collaborator Author

@mgierlings mgierlings commented May 10, 2019

@randombit Looks good to me! One remark though on the XMSS parameter name changes introduced with 10630cc: The RFC actually uses "-" between XMSS and the hash function name, see RFC 8391 Section 5.3 XMSS Parameters. By replacing the dash with an underscore we technically use non standard compliant algorithm names.

@securitykernel
Copy link
Collaborator

@securitykernel securitykernel commented May 12, 2019

@mgierlings BouncyCastle uses the same naming scheme, maybe this is a thing? Anyway, whatever we decide here, we should probably make it consistent in both libraries.

@mgierlings
Copy link
Collaborator Author

@mgierlings mgierlings commented May 12, 2019

@securitykernel Agreed, we should be consistent. If others start implementing XMSS, they will likely use names from the standard document. The cleanest solution would probably be to adhere to the RFC's naming scheme (in both BouncyCastle and Botan). If we come up with our own now we might need to revisit this and change naming again in the future.

@randombit
Copy link
Owner

@randombit randombit commented May 12, 2019

I renamed the params because that's what the patched handbook uses for the param names. I find combining hyphen and underscore really awkward. But if that is the standard naming scheme, probably best to follow it.

@mgierlings
Copy link
Collaborator Author

@mgierlings mgierlings commented May 12, 2019

@randombit The algorithm names in the handbook are supposed to correspond to the xmss_algorithm_t enum items, since this is what is passed as argument in the code snippet below. However I did omit the bulky prefix Botan::XMSS_Parameters::xmss_algorithm_t, as a result the algorithm list in the handbook is somewhat ambiguous. To be more clear, we could change the handbook to list the available algorithms as follows:

Botan::XMSS_Parameters::xmss_algorithm_t::XMSS_SHA2_10_256 ("XMSS-SHA2_10_256")
Botan::XMSS_Parameters::xmss_algorithm_t::XMSS_SHA2_16_256 ("XMSS-SHA2_16_256")
Botan::XMSS_Parameters::xmss_algorithm_t::XMSS_SHA2_20_256 ("XMSS-SHA2_20_256")
...

or

"XMSS-SHA2_10_256" (Botan::XMSS_Parameters::xmss_algorithm_t::XMSS_SHA2_10_256)
"XMSS-SHA2_16_256" (Botan::XMSS_Parameters::xmss_algorithm_t::XMSS_SHA2_16_256)
"XMSS-SHA2_20_256" (Botan::XMSS_Parameters::xmss_algorithm_t::XMSS_SHA2_20_256)
...
@randombit
Copy link
Owner

@randombit randombit commented May 24, 2019

I switched param names back. So I think this is now good to merge? It's been almost two months since the notification went out warning about the change and it looks like nobody sees an issue with it.

@mgierlings
Copy link
Collaborator Author

@mgierlings mgierlings commented May 24, 2019

@randombit Yes, I'd say we're good to merge.

@randombit randombit merged commit e390adc into randombit:master May 24, 2019
3 of 4 checks passed
3 of 4 checks passed
codecov/patch 69.23% of diff hit (target 91.38%)
Details
codecov/project 91.37% (-0.01%) compared to 2c9caac
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
randombit added a commit that referenced this pull request May 24, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

4 participants