Efficient comparisons #1523

mitschabaude · 2024-03-26T21:14:55Z

closes #1434

Implement foreign field comparison assertions on two variables
Use those to implement full-field comparisons which support 255-bit field elements
- constraints reduced by up to 20x
Unify small field comparisons by defining generic versions that work for any range check (which is an input).
- These are now used for all uints and more efficient than the previous boolean comparisons
Reimplement isEven() and isOdd() using full field comparison, which makes them much cheaper and also handle 255-bit field elements
- very nice effect on signature methods, which use isEven() for converting between compressed and uncompressed public keys
To be able to use foreign field gadgets inside the Field class, and prevent against import cycle problems in the future, I got rid of the dependency on Field in many of the gadgets and a few other places

… field

mitschabaude · 2024-03-28T10:43:23Z

src/lib/provable/crypto/signature.ts

here we use the new efficient isOdd(), plus some boyscouting

mitschabaude · 2024-03-28T10:43:59Z

src/lib/provable/field.ts

+      return [x & 1n, x >> 1n];
+    });
+    let isOdd = b.assertBool();
+    z.assertLessThan((Field.ORDER + 1n) / 2n);


using the new full field comparison!

mitschabaude · 2024-03-28T10:45:22Z

src/lib/provable/field.ts

-   * Field(2).lessThan(3).assertEquals(Bool(true));
+   * let isTrue = Field(2).lessThan(3);


some comments boyscouting in this file. I didn't like that the example for lessThan() was one that should actually have used assertLessThan()

mitschabaude · 2024-03-28T10:46:41Z

src/lib/provable/field.ts

-   * Calling this function is equivalent to `Field(...).lessThan(...).assertEquals(Bool(true))`.
+   *
+   * Note: This uses fewer constraints than `x.lessThan(y).assertTrue()`.


didn't like that this said it "was equivalent", people should use the assertion version if possible because they are cheaper

mitschabaude · 2024-03-28T10:47:59Z

src/lib/provable/gadgets/basic.ts

removing the import cycles was mainly about importing Field only as a type, and using what we need from field-constructor.ts instead

mitschabaude · 2024-03-28T10:49:26Z

src/lib/provable/gadgets/foreign-field.ts

+/**
+ * Version of `multiRangeCheck` which does the check on a truncated version of the input,
+ * so that it always succeeds, and then checks equality of the truncated and full input.
+ *
+ * This is a hack to get an error when the constraint fails, around the fact that multiRangeCheck
+ * is not checked by snarky.
+ */
+function indirectMultiRangeChange(


added this to not make the (negative) unit tests break. it's also better DX

mitschabaude · 2024-03-28T10:50:24Z

src/lib/provable/int.ts

-      let xMinusYFits = RangeCheck.isDefinitelyInRangeN(
-        UInt64.NUM_BITS,
-        xMinusY
-      );
-
-      let yMinusXFits = RangeCheck.isDefinitelyInRangeN(
-        UInt64.NUM_BITS,
-        yMinusX
-      );
-
-      xMinusYFits.or(yMinusXFits).assertEquals(true);
-      // x <= y if y - x fits in 64 bits
-      return yMinusXFits;
    }
+    return lessThanOrEqualGeneric(this.value, y.value, 1n << 64n, (v) =>
+      RangeCheck.rangeCheckN(UInt64.NUM_BITS, v)
+    );


this used to be less efficient because it called rangeCheckHelper() twice. the new version only calls it once

mitschabaude · 2024-03-28T10:50:59Z

src/lib/provable/provable.ts

  asProver,
  constraintSystem,
  generateWitness,
 } from './core/provable-context.js';
-import { exists, existsAsync } from './core/exists.js';
+import { witness, witnessAsync } from './types/witness.js';


moved witness() to its own file to make it not depend on Field

mitschabaude · 2024-03-28T10:52:17Z

src/lib/provable/field.ts

      assertBoolean(this);
+      return Bool.Unsafe.fromField(this);


I made assertBool() return the Bool because this is a common use case: assert that some witnessed field element is boolean and then use the bool

mitschabaude · 2024-03-28T10:53:05Z

src/lib/provable/test/foreign-field-gadgets.unit-test.ts

-  equivalentProvable({ from: [big264, big264], to: big264 })(
+  // addition can fail on two unreduced inputs because we can get x + y - f > 2^264
+  equivalentProvable({ from: [big264, f], to: big264 })(


we didn't catch this before because of the missing errors on multi range check failures 😬

Trivo25 · 2024-03-28T11:14:42Z

CHANGELOG.md

@@ -30,6 +30,12 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
  - Remove `CircuitValue`, `prop`, `arrayProp` and `matrixProp` https://github.com/o1-labs/o1js/pull/1507
  - Remove `Mina.accountCreationFee()`, `Mina.BerkeleyQANet`, all APIs which accept private keys for feepayers, `Token`, `AccountUpdate.tokenSymbol`, `SmartContract.{token, setValue, setPermissions}`, "assert" methods for preconditions, `MerkleTee.calculateRootSlow()`, `Scalar.fromBigInt()`, `UInt64.lt()` and friends, deprecated static methods on `Group`, utility methods on `Circuit` like `Circuit.if()`, `Field.isZero()`, `isReady` and `shutdown()` https://github.com/o1-labs/o1js/pull/1515
 - Remove `privateKey` from the accepted arguments of `SmartContract.deploy()` https://github.com/o1-labs/o1js/pull/1515
+- **Efficient comparisons**. Support arbitrary bit lengths for `Field` comparisons and massively reduce their constraints https://github.com/o1-labs/o1js/pull/1523
+  - `Field.assertLessThan()` goes from 510 to 24 constraints, `Field.lessThan()` from 509 to 38


Trivo25 · 2024-03-28T11:14:56Z

CHANGELOG.md

@@ -30,6 +30,12 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
  - Remove `CircuitValue`, `prop`, `arrayProp` and `matrixProp` https://github.com/o1-labs/o1js/pull/1507
  - Remove `Mina.accountCreationFee()`, `Mina.BerkeleyQANet`, all APIs which accept private keys for feepayers, `Token`, `AccountUpdate.tokenSymbol`, `SmartContract.{token, setValue, setPermissions}`, "assert" methods for preconditions, `MerkleTee.calculateRootSlow()`, `Scalar.fromBigInt()`, `UInt64.lt()` and friends, deprecated static methods on `Group`, utility methods on `Circuit` like `Circuit.if()`, `Field.isZero()`, `isReady` and `shutdown()` https://github.com/o1-labs/o1js/pull/1515
 - Remove `privateKey` from the accepted arguments of `SmartContract.deploy()` https://github.com/o1-labs/o1js/pull/1515
+- **Efficient comparisons**. Support arbitrary bit lengths for `Field` comparisons and massively reduce their constraints https://github.com/o1-labs/o1js/pull/1523
+  - `Field.assertLessThan()` goes from 510 to 24 constraints, `Field.lessThan()` from 509 to 38
+  - Moderately improve other comparisons: `UInt64.assertLessThan()` from 27 to 14,, `UInt64.lessThan()` from 27 to 15, `UInt32` similar.


double ,,

mitschabaude added 3 commits March 27, 2024 11:12

bigint assert less than

9c6fa34

bigint assert less than or equal

e274fcd

unit tests for comparison assertions

ffae661

mitschabaude force-pushed the feature/efficient-comparisons branch from cc39156 to ffae661 Compare March 27, 2024 10:13

mitschabaude changed the base branch from feature/basic-gadgets-cs to feature/compatible-comparisons March 27, 2024 10:16

mitschabaude mentioned this pull request Mar 27, 2024

Compatible comparisons #1524

Merged

mitschabaude added 24 commits March 27, 2024 13:29

helper in field stub module

0af69c1

more efficient comparison assertions which are generic over the range

41f75d1

refine assumptions of comparison gadget

ea60f08

use generic comparison assertion for uint, improve efficiency

9b5c337

generic boolean comparisons

79e610d

use generic boolean comparison for field

65e21bd

use generic boolean comparisons for uint, improve efficiency

71a1b0d

full field comparison

8726900

fixup uint lessthan

34bbd56

get rid of circular dependencies from using foreign field gadgets for…

17fbdb6

… field

constant spec

49f04f1

make failing foreign field additions throw, so we can trust unit tests

53e5b11

full less than or equal

9e2357c

use full comparison assertions for field, massive efficiency gain

e703bdd

full boolean comparisons

87be871

use full boolean comparisons

1b80114

remove 253-bits limitation from tests and docs

9531b51

fix edge case in bigint assert less than

9606b23

explicit unsafe bool creation and optimization for constants

76f261c

more efficient isEven that always works; assertBool returns a Bool

10138fa

it was a lovely trick, but this is more efficient

f04c396

fixup diverse zkprogram types

af78122

use fast isOdd gadget in signature gadgets

9a362d2

cleanup and changelog

0b56545

mitschabaude added 2 commits March 28, 2024 10:54

final changelog tweaks

a7808e2

bindings

b973423

mitschabaude marked this pull request as ready for review March 28, 2024 10:27

force recompile changes the vk

d2dc857

mitschabaude commented Mar 28, 2024

View reviewed changes

Base automatically changed from feature/compatible-comparisons to main March 28, 2024 11:13

Trivo25 approved these changes Mar 28, 2024

View reviewed changes

mitschabaude added 2 commits March 28, 2024 13:26

unrelated: fix ./run-in-browser on scripts that import o1js

5348010

typos

e15f77d

mitschabaude merged commit 8dde2c3 into main Mar 28, 2024
16 checks passed

mitschabaude deleted the feature/efficient-comparisons branch March 28, 2024 13:11

mitschabaude mentioned this pull request Mar 28, 2024

Issue with LessThan Function: No Support for Field Elements Greater Than 253 Bits #1061

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Efficient comparisons #1523

Efficient comparisons #1523

mitschabaude commented Mar 26, 2024 •

edited

mitschabaude Mar 28, 2024

mitschabaude Mar 28, 2024

mitschabaude Mar 28, 2024

mitschabaude Mar 28, 2024

mitschabaude Mar 28, 2024

mitschabaude Mar 28, 2024

mitschabaude Mar 28, 2024

mitschabaude Mar 28, 2024

mitschabaude Mar 28, 2024

mitschabaude Mar 28, 2024

Trivo25 Mar 28, 2024

Trivo25 Mar 28, 2024

		* Field(2).lessThan(3).assertEquals(Bool(true));
		* let isTrue = Field(2).lessThan(3);

Efficient comparisons #1523

Efficient comparisons #1523

Conversation

mitschabaude commented Mar 26, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mitschabaude commented Mar 26, 2024 •

edited