Squashed 'src/secp256k1/' changes from b19c000..df8c69879

df8c69879 f rename private to secret key to be compatible with PR 701
59f2fd7c1 f add test that xonly_add zeroes the output on failure
af3f0e52a f do xonly_pubkey_tweak_add in place to be consistent with ec_pubkey_tweak_add
6e0290679 f is_negated and added test
1901f3bf9 Add taproot test case to schnorrsig module
6a7ee8a86 Add schnorrsig module which implements BIP-schnorr [0] compatible signing, verification and batch verification
c6473591a Add initialize_tagged to sha256 which initializes and writes the 64 byte string SHA256(tag)||SHA256(tag) into it.
fd9609d7c Add chacha20 function
1abbe1bb6 Add tweak functions for xonly_pubkeys that allow to add a tweak to a secret key, xonly_public key and verify a tweak.
9eb303a0f Add xonly_pubkeys which are serialized as 32 byte and whose Y coordinate is a quadratic residue
387d723 Merge bitcoin#679: Add SECURITY.md
0db61d2 Merge bitcoin#685: Fix issue where travis does not show the ./tests seed…
a0771d1 Explicitly disable buffering for stderr in tests
fb424fb Make travis show the ./tests seed by removing stdout buffering and always cat tests.log after a travis run.
22a6031 Merge bitcoin#690: Add valgrind check to travis
544002c Merge bitcoin#678: Preventing compiler optimizations in benchmarks without a memory fence
dd98cc9 travis: Added a valgrind test without endro and enabled recovery+ecdh
b4c1382 Add valgrind check to travis
0c774d8 Merge bitcoin#688: Fix ASM setting in travis
5c5f71e Fix ASM setting in travis
e2625f8 Merge bitcoin#684: Make no-float policy explicit
bae1bea Make no-float policy explicit
78c3836 Add SECURITY.md
362bb25 Modified bench_scalar_split so it won't get optimized out
73a30c6 Added accumulators and checks on benchmarks so they won't get optimized out
770b3dc Merge bitcoin#677: Remove note about heap allocation in secp256k1_ecmult_odd_multiples_table_storage_var
b76142f Remove note about heap allocation in secp256k1_ecmult_odd_multiples_table_storage_var which was removed in 4704527
137d304 Merge bitcoin#647: Increase robustness against UB in secp256k1_scalar_cadd_bit
0d9540b Merge bitcoin#664: Remove mention of ec_privkey_export because it doesn't exist
59782c6 Remove mention of ec_privkey_export because it doesn't exist
96cd94e Merge bitcoin#337: variable sized precomputed table for signing
dcb2e3b variable signing precompute table
b4bff99 Merge bitcoin#661: Make ./configure string consistent
a467047 Make ./configure string consistent
e729cc7 Merge bitcoin#657: Fix a nit in the recovery tests
b64a2e2 Fix a nit in the recovery tests
e028aa3 Merge bitcoin#650: secp256k1/src/tests.c:  Properly handle sscanf return value
f1e11d3 Merge bitcoin#654: Fix typo (∞)
ef83281 Merge pull request bitcoin#656 from real-or-random/patch-1
556caad Fix typo in docs for _context_set_illegal_callback
0d82732 Improve VERIFY_CHECK of overflow in secp256k1_scalar_cadd_bit. This added check ensures that any curve order overflow doesn't go undetected due a uint32_t overflow.
786dfb4 Merge bitcoin#583: JNI: fix use sig array
e95f8ab Merge bitcoin#644: Avoid optimizing out a verify_check
384f556 Merge bitcoin#652: README.md: update instruction to run tests
ee56acc Merge bitcoin#651: Fix typo in secp256k1_preallocated.h
7b9b117 Merge bitcoin#640: scalar_impl.h: fix includes
d99bec2 Merge bitcoin#655: jni: Use only Guava for hex encoding and decoding
2abcf95 jni: Use only Guava for hex encoding and decoding
271582b Fix typo
ce6d438 README.md: update instruction to run tests
b1e68cb Fix typo in secp256k1_preallocated.h
a11c76c secp256k1/src/tests.c:  Properly handle sscanf return value
8fe63e5 Increase robustness against UB. Thanks to elichai2 who noted that the literal '1' is a signed integer, and that shifting a signed 32-bit integer by 31 bits causes an overflow and yields undefined behaviour. While 'scalar_low_impl''s 'secp256k1_scalar_cadd_bit' is only used for testing purposes and currently the 'bit' parameter is only 0 or 1, it is better to avoid undefined behaviour in case the used domain of 'secp256k1_scalar_cadd_bit' expands.
94ae7cb Moved a dereference so the null check will be before the dereferencing
2cb73b1 scalar_impl.h: fix includes
fa33017 Merge bitcoin#634: Add a descriptive comment for secp256k1_ecmult_const.
ee9e68c Add a descriptive comment for secp256k1_ecmult_const.
d0d738d Merge bitcoin#631: typo in comment for secp256k1_ec_pubkey_tweak_mul ()
6914c25 typo in comment for secp256k1_ec_pubkey_tweak_mul ()
e541a90 Merge bitcoin#629: Avoid calling _is_zero when _set_b32 fails.
f34b0c3 Merge bitcoin#630: Note intention of timing sidechannel freeness.
8d1563b Note intention of timing sidechannel freeness.
1669bb2 Merge bitcoin#628: Fix ability to compile tests without -DVERIFY.
ecc94ab Merge bitcoin#627: Guard memcmp in tests against mixed size inputs.
544435f Merge bitcoin#578: Avoid implementation-defined and undefined behavior when dealing with sizes
143dc6e Merge bitcoin#595: Allow to use external default callbacks
e49f799 Add missing #(un)defines to base-config.h
77defd2 Add secp256k1_ prefix to default callback functions
908bdce Include stdio.h and stdlib.h explicitly in secp256k1.c
5db782e Allow usage of external default callbacks
6095a86 Replace CHECKs for no_precomp ctx by ARG_CHECKs without a return
cd473e0 Avoid calling secp256k1_*_is_zero when secp256k1_*_set_b32 fails.
6c36de7 Merge bitcoin#600: scratch space: use single allocation
98836b1 scratch: replace frames with "checkpoint" system
7623cf2 scratch: save a couple bytes of unnecessarily-allocated memory
a7a164f scratch: rename `max_size` to `size`, document that extra will actually be allocated
5a4bc0b scratch: unify allocations
c2b028a scratch space: thread `error_callback` into all scratch space functions
0be1a4a scratch: add magic bytes to beginning of structure
92a48a7 scratch space: use single allocation
40839e2 Merge bitcoin#592: Use trivial algorithm in ecmult_multi if scratch space is small
dcf3920 Fix ability to compile tests without -DVERIFY.
a484e00 Merge bitcoin#566: Enable context creation in preallocated memory
0522caa Explain caller's obligations for preallocated memory
238305f Move _preallocated functions to separate header
695feb6 Export _preallocated functions
814cc78 Add tests for contexts in preallocated memory
ba12dd0 Check arguments of _preallocated functions
5feadde Support cloning a context into preallocated memory
c4fd5da Switch to a single malloc call
ef020de Add size constants for preallocated memory
1bf7c05 Prepare for manual memory management in preallocated memory
248bffb Guard memcmp in tests against mixed size inputs.
36698dc Merge bitcoin#596: Make WINDOW_G configurable
a61a93f Clean up ./configure help strings
2842dc5 Make WINDOW_G configurable
1a02d6c Merge bitcoin#626: Revert "Merge bitcoin#620: Install headers automatically"
662918c Revert "Merge bitcoin#620: Install headers automatically"
14c7dbd Simplify control flow in DER parsing
ec8f20b Avoid out-of-bound pointers and integer overflows in size comparisons
01ee1b3 Parse DER-enconded length into a size_t instead of an int
912680e Merge bitcoin#561: Respect LDFLAGS and #undef STATIC_PRECOMPUTATION if using basic config
91fae3a Merge bitcoin#620: Install headers automatically
5df77a0 Merge bitcoin#533: Make sure we're not using an uninitialized variable in secp256k1_wnaf_const(...)
975e51e Merge bitcoin#617: Pass scalar by reference in secp256k1_wnaf_const()
735fbde Merge bitcoin#619: Clear a copied secret key after negation
16e8615 Install headers automatically
069870d Clear a copied secret key after negation
8979ec0 Pass scalar by reference in secp256k1_wnaf_const()
84a8085 Merge bitcoin#612: Allow field_10x26_arm.s to compile for ARMv7 architecture
d4d270a Allow field_10x26_arm.s to compile for ARMv7 architecture
248f046 Make sure we're not using an uninitialized variable in secp256k1_wnaf_const(...)
9ab96f7 Use trivial algorithm in ecmult_multi if scratch space is small
dbed75d Undefine `STATIC_PRECOMPUTATION` if using the basic config
310111e Keep LDFLAGS if `--coverage`
74e2dbd JNI: fix use sig array
3cb057f Fix possible integer overflow in DER parsing

git-subtree-dir: src/secp256k1
git-subtree-split: df8c698793a51639b6fdb5b908b1ddfaedbe1fd5

.gitignore

@@ -1,9 +1,9 @@
 bench_inv
 bench_ecdh
 bench_ecmult
+bench_schnorrsig
 bench_sign
 bench_verify
-bench_schnorr_verify
 bench_recover
 bench_internal
 tests

.travis.yml

@@ -11,25 +11,27 @@ cache:
   - src/java/guava/
 env:
   global:
-    - FIELD=auto  BIGNUM=auto  SCALAR=auto  ENDOMORPHISM=no  STATICPRECOMPUTATION=yes  ASM=no  BUILD=check  EXTRAFLAGS=  HOST=  ECDH=no  RECOVERY=no  EXPERIMENTAL=no  JNI=no
+    - FIELD=auto  BIGNUM=auto  SCALAR=auto  ENDOMORPHISM=no  STATICPRECOMPUTATION=yes  ECMULTGENPRECISION=auto  ASM=no  BUILD=check  EXTRAFLAGS=  HOST=  ECDH=no  RECOVERY=no  EXPERIMENTAL=no  JNI=no JNI=no SCHNORRSIG=no
     - GUAVA_URL=https://search.maven.org/remotecontent?filepath=com/google/guava/guava/18.0/guava-18.0.jar GUAVA_JAR=src/java/guava/guava-18.0.jar
   matrix:
     - SCALAR=32bit    RECOVERY=yes
-    - SCALAR=32bit    FIELD=32bit       ECDH=yes  EXPERIMENTAL=yes
+    - SCALAR=32bit    FIELD=32bit       ECDH=yes  EXPERIMENTAL=yes   SCHNORRSIG=yes
     - SCALAR=64bit
-    - FIELD=64bit     RECOVERY=yes
+    - FIELD=64bit     RECOVERY=yes      EXPERIMENTAL=yes             SCHNORRSIG=yes
     - FIELD=64bit     ENDOMORPHISM=yes
-    - FIELD=64bit     ENDOMORPHISM=yes  ECDH=yes EXPERIMENTAL=yes
+    - FIELD=64bit     ENDOMORPHISM=yes  ECDH=yes EXPERIMENTAL=yes    SCHNORRSIG=yes
     - FIELD=64bit                       ASM=x86_64
     - FIELD=64bit     ENDOMORPHISM=yes  ASM=x86_64
     - FIELD=32bit     ENDOMORPHISM=yes
     - BIGNUM=no
-    - BIGNUM=no       ENDOMORPHISM=yes RECOVERY=yes EXPERIMENTAL=yes
+    - BIGNUM=no       ENDOMORPHISM=yes RECOVERY=yes EXPERIMENTAL=yes SCHNORRSIG=yes
     - BIGNUM=no       STATICPRECOMPUTATION=no
     - BUILD=distcheck
     - EXTRAFLAGS=CPPFLAGS=-DDETERMINISTIC
     - EXTRAFLAGS=CFLAGS=-O0
     - BUILD=check-java JNI=yes ECDH=yes EXPERIMENTAL=yes
+    - ECMULTGENPRECISION=2
+    - ECMULTGENPRECISION=8
 matrix:
   fast_finish: true
   include:
@@ -59,10 +61,40 @@ matrix:
           packages:
             - gcc-multilib
             - libgmp-dev:i386
+    - compiler: gcc
+      env:
+        - BIGNUM=no  ENDOMORPHISM=yes  ASM=x86_64 EXPERIMENTAL=yes ECDH=yes  RECOVERY=yes
+        - VALGRIND=yes EXTRAFLAGS="--disable-openssl-tests CPPFLAGS=-DVALGRIND" BUILD=
+      addons:
+        apt:
+          packages:
+            - valgrind
+    - compiler: gcc
+      env: # The same as above but without endomorphism.
+        - BIGNUM=no  ENDOMORPHISM=no  ASM=x86_64 EXPERIMENTAL=yes ECDH=yes  RECOVERY=yes
+        - VALGRIND=yes EXTRAFLAGS="--disable-openssl-tests CPPFLAGS=-DVALGRIND" BUILD=
+      addons:
+        apt:
+          packages:
+            - valgrind
+
 before_install: mkdir -p `dirname $GUAVA_JAR`
 install: if [ ! -f $GUAVA_JAR ]; then wget $GUAVA_URL -O $GUAVA_JAR; fi
 before_script: ./autogen.sh
+
 script:
  - if [ -n "$HOST" ]; then export USE_HOST="--host=$HOST"; fi
  - if [ "x$HOST" = "xi686-linux-gnu" ]; then export CC="$CC -m32"; fi
- - ./configure --enable-experimental=$EXPERIMENTAL --enable-endomorphism=$ENDOMORPHISM --with-field=$FIELD --with-bignum=$BIGNUM --with-scalar=$SCALAR --enable-ecmult-static-precomputation=$STATICPRECOMPUTATION --enable-module-ecdh=$ECDH --enable-module-recovery=$RECOVERY --enable-jni=$JNI $EXTRAFLAGS $USE_HOST && make -j2 $BUILD
+ - ./configure --enable-experimental=$EXPERIMENTAL --enable-endomorphism=$ENDOMORPHISM --with-field=$FIELD --with-bignum=$BIGNUM --with-asm=$ASM --with-scalar=$SCALAR --enable-ecmult-static-precomputation=$STATICPRECOMPUTATION --with-ecmult-gen-precision=$ECMULTGENPRECISION --enable-module-ecdh=$ECDH --enable-module-recovery=$RECOVERY --enable-module-schnorrsig=$SCHNORRSIG --enable-jni=$JNI $EXTRAFLAGS $USE_HOST
+ - if [ -n "$BUILD" ]; then make -j2 $BUILD; fi
+ - # travis_wait extends the 10 minutes without output allowed (https://docs.travis-ci.com/user/common-build-problems/#build-times-out-because-no-output-was-received)
+ - # the `--error-exitcode` is required to make the test fail if valgrind found errors, otherwise it'll return 0 (http://valgrind.org/docs/manual/manual-core.html)
+ - if [ -n "$VALGRIND" ]; then
+   make -j2 &&
+   travis_wait 30 valgrind --error-exitcode=42 ./tests 16 &&
+   travis_wait 30 valgrind --error-exitcode=42 ./exhaustive_tests;
+   fi
+
+after_script:
+    - cat ./tests.log
+    - cat ./exhaustive_tests.log

Makefile.am

@@ -8,6 +8,7 @@ else
 JNI_LIB =
 endif
 include_HEADERS = include/secp256k1.h
+include_HEADERS += include/secp256k1_preallocated.h
 noinst_HEADERS =
 noinst_HEADERS += src/scalar.h
 noinst_HEADERS += src/scalar_4x64.h
@@ -150,11 +151,11 @@ endif
 endif
 
 if USE_ECMULT_STATIC_PRECOMPUTATION
-CPPFLAGS_FOR_BUILD +=-I$(top_srcdir)
+CPPFLAGS_FOR_BUILD +=-I$(top_srcdir) -I$(builddir)/src
 
 gen_context_OBJECTS = gen_context.o
 gen_context_BIN = gen_context$(BUILD_EXEEXT)
-gen_%.o: src/gen_%.c
+gen_%.o: src/gen_%.c src/libsecp256k1-config.h
 	$(CC_FOR_BUILD) $(CPPFLAGS_FOR_BUILD) $(CFLAGS_FOR_BUILD) -c $< -o $@
 
 $(gen_context_BIN): $(gen_context_OBJECTS)
@@ -177,6 +178,10 @@ if ENABLE_MODULE_ECDH
 include src/modules/ecdh/Makefile.am.include
 endif
 
+if ENABLE_MODULE_SCHNORRSIG
+include src/modules/schnorrsig/Makefile.am.include
+endif
+
 if ENABLE_MODULE_RECOVERY
 include src/modules/recovery/Makefile.am.include
 endif

README.md

@@ -23,6 +23,7 @@ Implementation details
   * Extensive testing infrastructure.
   * Structured to facilitate review and analysis.
   * Intended to be portable to any system with a C89 compiler and uint64_t support.
+  * No use of floating types, except in benchmarks.
   * Expose only higher level interfaces to minimize the API surface and improve application security. ("Be difficult to use insecurely.")
 * Field operations
   * Optimized implementation of arithmetic modulo the curve's field size (2^256 - 0x1000003D1).
@@ -45,8 +46,10 @@ Implementation details
   * Optionally (off by default) use secp256k1's efficiently-computable endomorphism to split the P multiplicand into 2 half-sized ones.
 * Point multiplication for signing
   * Use a precomputed table of multiples of powers of 16 multiplied with the generator, so general multiplication becomes a series of additions.
-  * Access the table with branch-free conditional moves so memory access is uniform.
-  * No data-dependent branches
+  * Intended to be completely free of timing sidechannels for secret-key operations (on reasonable hardware/toolchains)
+    * Access the table with branch-free conditional moves so memory access is uniform.
+    * No data-dependent branches
+  * Optional runtime blinding which attempts to frustrate differential power analysis.
   * The precomputed tables add and eventually subtract points for which no known scalar (private key) is known, preventing even an attacker with control over the private key used to control the data internally.
 
 Build steps
@@ -57,5 +60,19 @@ libsecp256k1 is built using autotools:
     $ ./autogen.sh
     $ ./configure
     $ make
-    $ ./tests
+    $ make check
     $ sudo make install  # optional
+
+Exhaustive tests
+-----------
+
+    $ ./exhaustive_tests
+
+With valgrind, you might need to increase the max stack size:
+
+    $ valgrind --max-stackframe=2500000 ./exhaustive_tests
+
+Reporting a vulnerability
+------------
+
+See [SECURITY.md](SECURITY.md)

SECURITY.md

@@ -0,0 +1,15 @@
+# Security Policy
+
+## Reporting a Vulnerability
+
+To report security issues send an email to secp256k1-security@bitcoincore.org (not for support).
+
+The following keys may be used to communicate sensitive information to developers:
+
+| Name | Fingerprint |
+|------|-------------|
+| Pieter Wuille | 133E AC17 9436 F14A 5CF1  B794 860F EB80 4E66 9320 |
+| Andrew Poelstra | 699A 63EF C17A D3A9 A34C  FFC0 7AD0 A91C 40BD 0091 |
+| Tim Ruffing | 09E0 3F87 1092 E40E 106E  902B 33BC 86AB 80FF 5516 |
+
+You can import a key by running the following command with that individual’s fingerprint: `gpg --recv-keys "<fingerprint>"` Ensure that you put quotes around fingerprints containing spaces.