Schnorr (Incremental) Half Aggregation #261

b-wagn · 2023-07-30T17:57:07Z

Revisited PR #130 by @jonasnick.
I am happy to hear your thoughts.

Summary of changes compared to #130:

Address comments from @rustyrussell
Use tagged hash
Compute hashes with common prefix by copying midstate
Allow Incremental Aggregation and make code consistent with the draft spec

jonasnick

Cool, thanks @b-wagn.

Can you remove the added vscode stuff from the commits? This would allow to getting rid of the two "remove vscode settings" commits.

In order to verify that this matches the spec, it would help to add the preliminary test vectors (they don't seem to cover incremental aggregation yet).

real-or-random

Here are some comments but I haven't had a closer look yet.

This also needs trivial rebase on master as we made quite a lot of changes there (synced with upstream secp256k1).

include/secp256k1_schnorrsig.h

src/modules/schnorrsig/main_impl.h

b-wagn · 2023-08-02T16:07:40Z

Thanks for your feedback.
I will provide an updated version asap.

real-or-random

This is very clean already, I honestly appreciate the quality of the code.

A somewhat annoying point is that this should probably moved to a separate module. If you want, you can give it a try, but this will involve modifying the build system, too. (Just the autotools build system with configure.ac and Makefile.am. You can ignore the CMake, we don't support it currently in this repo, it's just there because it's in upstream secp256k1 and we would like to keep the diff small.) Happy to help if you want to give it a try; otherwise Jonas or I could just take of it.

.gitignore

real-or-random · 2023-08-18T12:29:00Z

src/modules/schnorrsig/main_impl.h

+    ARG_CHECK(new_sigs64 != NULL);
+
+    /* Check that aggsig_size is large enough, i.e. aggsig_size >= 32*(n+1) */
+    n = n_before + n_new;


I think this should return 0 also when the addition here overflows. (I mean, you could say then anyway, something is wrong somewhere else, because the user would think they pass an array of size larger than size_t...) I just think it's more obviously correct (and thus easier to review with the overflow check.)

On a second thought, I wonder whether this should be an ARG_CHECK instead of return 0. Not sure, we'll need to think about it.

src/modules/schnorrsig/main_impl.h

src/modules/schnorrsig/tests_impl.h

include/secp256k1_schnorrsig.h

jonasnick

Here's a (maybe somewhat outdated) template for creating an independent module: 048f9f8

src/modules/schnorrsig/main_impl.h

include/secp256k1_schnorrsig.h

jonasnick · 2023-10-12T20:53:42Z

We've just merged #270 which means that this PR can be rebased to fix CI.

jonasnick · 2023-10-25T20:27:49Z

configure.ac

+if test x"$enable_module_schnorrsig_halfagg" = x"yes"; then
+  SECP_CONFIG_DEFINES="$SECP_CONFIG_DEFINES -DENABLE_MODULE_SCHNORRSIG_HALFAGG=1"
+  enable_module_schnorrsig=yes
+fi


You must do this before testing for schnorrsig because the setting enable_module_schnorrsig=yes happens after checking for enable_module_schnorrsig=yes. There's a comment above mentioning this (sorry, hard to find):

# Besides testing whether modules are enabled, the following code also enables # module dependencies. The order of the tests matters: the dependency must be # tested first.

real-or-random

@b-wagn I think the main thing to do here from your side is to organize this PR in proper commits. Normally, we would probably prefer small atomic commits (one logical change at a time) but, as this adds an entirely new module instead of changing existing code, all the changes could just be squashed into a single commit. Or you could have one commit for the code, one commit for the tests, and one commit for the build system + CI or something like this. Please don't hesitate to force-push to your branch.

I think then we can do a final review, and I don't except further significant changes then. If you ask me), it's ready for merge then as an experimental module.

We should perhaps add a version number (or some revision identifier) to the BIP draft, as long as the BIP is unstable. We could then say here which version the current code implements. Since it's an experimental module, we don't guarantee API stability, so it's not a big deal if we change the BIP in an incompatible way and need to update the code accordingly.

real-or-random · 2023-11-24T15:11:46Z

src/modules/schnorrsig_halfagg/Makefile.am.include

@@ -0,0 +1,3 @@
+include_HEADERS += include/secp256k1_schnorrsig_halfagg.h
+noinst_HEADERS += src/modules/schnorrsig_halfagg/main_impl.h
+noinst_HEADERS += src/modules/schnorrsig_halfagg/tests_impl.h


nit: file should end with a line break

real-or-random · 2023-11-24T15:27:35Z

We should perhaps add a version number (or some revision identifier) to the BIP draft, as long as the BIP is unstable. We could then say here which version the current code implements. Since it's an experimental module, we don't guarantee API stability, so it's not a big deal if we change the BIP in an incompatible way and need to update the code accordingly.

Okay, while breaking changes are possible, two things we should think about are the top two items in BlockstreamResearch/cross-input-aggregation#11... Ideally, we would reach a decision on these before we merge this PR, but on the other hand, I don't think these considerations should really hold off this PR. It's already large enough, and changes can also be done in further PRs.

b-wagn · 2023-11-27T15:05:59Z

We should perhaps add a version number (or some revision identifier) to the BIP draft, as long as the BIP is unstable. We could then say here which version the current code implements. Since it's an experimental module, we don't guarantee API stability, so it's not a big deal if we change the BIP in an incompatible way and need to update the code accordingly.

Okay, while breaking changes are possible, two things we should think about are the top two items in BlockstreamResearch/cross-input-aggregation#11... Ideally, we would reach a decision on these before we merge this PR, but on the other hand, I don't think these considerations should really hold off this PR. It's already large enough, and changes can also be done in further PRs.

Either option is fine. I would probably prefer first merging this and then opening a new PR with the z0 = 1 option and properly adjusted tests and so on.

real-or-random

I still need to look at the tests and the build system/CI stuff again.

include/secp256k1_schnorrsig_halfagg.h

src/modules/schnorrsig_halfagg/main_impl.h

include/secp256k1_schnorrsig_halfagg.h

b-wagn · 2024-01-05T07:57:14Z

Thank you for your comments @real-or-random.
I addressed them, and I will organize the PR into one commit as soon as your review is completed.

real-or-random · 2024-01-09T17:56:35Z

When reviewing the tests, I had a few comments in mind, and then I just started addressing them on my own because I thought that's more efficient in the end. But then this ended up in a larger rewrite. :D (Also one reason why the diff seems so large is that I made the comments a bit more consistent...)

I pushed my suggestions to https://github.com/real-or-random/secp256k1-zkp/tree/schnorr-half-agg. Please don't hesitate to cherry-pick the commit, but please also double-check that I didn't screw up anything...

b-wagn · 2024-01-11T08:21:27Z

Thank you @real-or-random. I went through your changes and they make sense to me, so I included them.

src/modules/schnorrsig_halfagg/tests_impl.h

real-or-random · 2024-02-05T15:37:51Z

Changes look good!

This needs rebase again due to changes in configure.ac (hopefully the last time!), but I just noticed that we'd like to do another change to configure.ac, so let us do this first to avoid that you'll need to rebase here twice...

real-or-random · 2024-02-16T15:08:33Z

Changes look good!

This needs rebase again due to changes in configure.ac (hopefully the last time!), but I just noticed that we'd like to do another change to configure.ac, so let us do this first to avoid that you'll need to rebase here twice...

#290 has been merged, so this is ready for rebase. (#290 adds a comment that explains in which order modules should appear in configure.ac and elsewhere.)

real-or-random · 2024-02-23T16:45:20Z

Thanks, the NULL changes look good

ACK This is read to merge from my side, mod the following two nits!

I think it will be consistent to do the same then in inc_aggregate (where the first two lines must be moved below the line where we compute n, of course ^^):

    ARG_CHECK(all_pubkeys != NULL || n == 0);
    ARG_CHECK(all_msgs32 != NULL || n== 0);
    ARG_CHECK(new_sigs64 != NULL || n_new == 0);

#290 adds a comment that explains in which order modules should appear in configure.ac and elsewhere.

Have you seen these comments? They're here: https://github.com/BlockstreamResearch/secp256k1-zkp/blob/master/configure.ac#L612-L640

This would mean that the schnorrsig_halfagg module should appear immediately after (or before, if the order is reversed) the bppp whenever we list modules. AFAICT this affects configure.ac (multiple listings), Makefile.am, tests.c, ci.yml, and ci.sh in this PR. It may seem a bit pedantic, but I hope that sticking to that convention will make diffs to upstream libsecp256k1 more manageable in the future.

real-or-random

ACK e1d15fd

Can you squash this into a single commit? Then we can merge it.

…natures

jonasnick

Concept ACK

I did a few sanity checks and read the include file again. Looks all good. I have a few nits that can be cleaned up in a separate PR. Your detailed and helpful comments in the main code and tests is very much appreciated.

Nits:

we may want to add a header to the include file that links to the BIP
there's still a mention of aggsig_size
we may want to move aggregate before inc_aggregate
we should mention expected size of input aggsig array in_aggregate
"Should be aggsig_len = 32*(n+1)" -> "Must be"

EDIT: to be clear, I think with @real-or-random's ACK this is ready for merge.

real-or-random

ACK 3a9b1d4

jonasnick reviewed Jul 31, 2023

View reviewed changes

b-wagn force-pushed the schnorr-half-agg branch from c2cab8e to 0495479 Compare July 31, 2023 10:22

real-or-random reviewed Aug 1, 2023

View reviewed changes

b-wagn force-pushed the schnorr-half-agg branch 3 times, most recently from 6045c3d to e49b9e3 Compare August 6, 2023 11:27

real-or-random reviewed Aug 18, 2023

View reviewed changes

jonasnick mentioned this pull request Sep 10, 2023

Add Schnorrsig half aggregation #130

Closed

jonasnick reviewed Sep 10, 2023

View reviewed changes

src/modules/schnorrsig/main_impl.h Outdated Show resolved Hide resolved

src/modules/schnorrsig/main_impl.h Outdated Show resolved Hide resolved

include/secp256k1_schnorrsig.h Outdated Show resolved Hide resolved

b-wagn force-pushed the schnorr-half-agg branch from 6c03afa to b989f93 Compare October 13, 2023 06:52

jonasnick reviewed Oct 25, 2023

View reviewed changes

real-or-random reviewed Nov 24, 2023

View reviewed changes

b-wagn force-pushed the schnorr-half-agg branch 2 times, most recently from ee54b1f to 959b070 Compare November 26, 2023 15:47

b-wagn marked this pull request as ready for review November 27, 2023 15:06

real-or-random reviewed Jan 4, 2024

View reviewed changes

real-or-random reviewed Jan 12, 2024

View reviewed changes

src/modules/schnorrsig_halfagg/tests_impl.h Outdated Show resolved Hide resolved

jonasnick mentioned this pull request Jan 31, 2024

BIP Halfagg: Fix two inconsistencies BlockstreamResearch/cross-input-aggregation#15

Merged

b-wagn force-pushed the schnorr-half-agg branch from 96a6e9d to efd98c1 Compare February 18, 2024 18:20

real-or-random approved these changes Feb 27, 2024

View reviewed changes

New Experimental Module: Incremental Half-Aggregation for Schnorr Sig…

3a9b1d4

…natures

b-wagn force-pushed the schnorr-half-agg branch from e1d15fd to 3a9b1d4 Compare February 27, 2024 13:08

jonasnick reviewed Feb 28, 2024

View reviewed changes

real-or-random approved these changes Mar 5, 2024

View reviewed changes

real-or-random merged commit a7907b1 into BlockstreamResearch:master Mar 5, 2024
107 checks passed

real-or-random mentioned this pull request Mar 5, 2024

Follow-ups to #261 #291

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Schnorr (Incremental) Half Aggregation #261

Schnorr (Incremental) Half Aggregation #261

b-wagn commented Jul 30, 2023 •

edited by real-or-random

jonasnick left a comment

real-or-random left a comment

b-wagn commented Aug 2, 2023

real-or-random left a comment •

edited

real-or-random Aug 18, 2023

jonasnick left a comment

jonasnick commented Oct 12, 2023

jonasnick Oct 25, 2023

real-or-random left a comment

real-or-random Nov 24, 2023

real-or-random commented Nov 24, 2023

b-wagn commented Nov 27, 2023

real-or-random left a comment

b-wagn commented Jan 5, 2024

real-or-random commented Jan 9, 2024

b-wagn commented Jan 11, 2024

real-or-random commented Feb 5, 2024

real-or-random commented Feb 16, 2024

real-or-random commented Feb 23, 2024

real-or-random left a comment •

edited

jonasnick left a comment •

edited

real-or-random left a comment

Schnorr (Incremental) Half Aggregation #261

Schnorr (Incremental) Half Aggregation #261

Conversation

b-wagn commented Jul 30, 2023 • edited by real-or-random

jonasnick left a comment

Choose a reason for hiding this comment

real-or-random left a comment

Choose a reason for hiding this comment

b-wagn commented Aug 2, 2023

real-or-random left a comment • edited

Choose a reason for hiding this comment

real-or-random Aug 18, 2023

Choose a reason for hiding this comment

jonasnick left a comment

Choose a reason for hiding this comment

jonasnick commented Oct 12, 2023

jonasnick Oct 25, 2023

Choose a reason for hiding this comment

real-or-random left a comment

Choose a reason for hiding this comment

real-or-random Nov 24, 2023

Choose a reason for hiding this comment

real-or-random commented Nov 24, 2023

b-wagn commented Nov 27, 2023

real-or-random left a comment

Choose a reason for hiding this comment

b-wagn commented Jan 5, 2024

real-or-random commented Jan 9, 2024

b-wagn commented Jan 11, 2024

real-or-random commented Feb 5, 2024

real-or-random commented Feb 16, 2024

real-or-random commented Feb 23, 2024

real-or-random left a comment • edited

Choose a reason for hiding this comment

jonasnick left a comment • edited

Choose a reason for hiding this comment

real-or-random left a comment

Choose a reason for hiding this comment

b-wagn commented Jul 30, 2023 •

edited by real-or-random

real-or-random left a comment •

edited

real-or-random left a comment •

edited

jonasnick left a comment •

edited