improve: add some macros to generate big testing suite of curves #129

duguorong009 · 2024-01-18T03:40:51Z

Description

Create some macros to generate the big testing suites for all of curves in the lib

Related issues

Upgrade testing across the lib #76

Changes

Add the curve_testing_suite macro for tests of curves
Create new macros for testing
Replace the functions used for testing, with new macros

CPerezz

This should also be testing the PrimeFiedBits remember! Also we need to see if for BN is necessary to add further testing for lookup-based ops or small scalars.
Also, each curve & Field should have tests for their associated constants to check they're correct.
Make sure all of the constants are indeed checked! :)

Aside from that, the rest looks really nice! I like the direction this is going towards!
Just left some comments :)

src/tests/curve.rs

CPerezz · 2024-01-19T10:56:44Z

Changes look much better @duguorong009 !

I like them a lot! Let's try to extend it to the Field tests and see how the solution looks like!

duguorong009 · 2024-01-19T15:51:14Z

@CPerezz
Please check the current status of the PR when you are free.
I've implemented all of macros for curve testing.
I'd like to first complete the curve testing part before moving to field testing.

CPerezz

Instead of having a #[cfg(test)] on each test fn,
can't you have a module which is under the test flag and have all the macros without it inside.

I talk about thinks like this:

halo2curves/src/bn256/curve.rs

Lines 201 to 273 in 762a329

    
           #[cfg(test)] 
        
           crate::curve_testing_suite!(G1, G2); 
        
           #[cfg(test)] 
        
           crate::curve_testing_suite!(G1, "hash_to_curve"); 
        
           #[cfg(test)] 
        
           crate::curve_testing_suite!(G1, "endo_consistency"); 
        
           #[cfg(test)] 
        
           crate::curve_testing_suite!( 
        
               G1, 
        
               "endo", 
        
               [ 
        
                   0x8b17ea66b99c90dd, 
        
                   0x5bfc41088d8daaa7, 
        
                   0xb3c4d79d41a91758, 
        
                   0x00, 
        
               ] 
        
           ); 
        
           #[cfg(test)] 
        
           crate::curve_testing_suite!( 
        
               G1, 
        
               "svdw_map_to_curve", 
        
               ( 
        
                   // Precomputed constants taken from https://github.com/ConsenSys/gnark-crypto/blob/441dc0ffe639294b8d09e394f24ba7575577229c/internal/generator/config/bn254.go#L26-L32. 
        
                   [ 
        
                       "4", 
        
                       "10944121435919637611123202872628637544348155578648911831344518947322613104291", 
        
                       "8815841940592487685674414971303048083897117035520822607866", 
        
                       "7296080957279758407415468581752425029565437052432607887563012631548408736189", 
        
                   ], 
        
                   // List of (u, (Q.x, Q.y)) taken from https://github.com/ConsenSys/gnark-crypto/blob/441dc0ffe639294b8d09e394f24ba7575577229c/ecc/bn254/hash_vectors_test.go#L4-L28 
        
                   [ 
        
                       ( 
        
                           "0xcb81538a98a2e3580076eed495256611813f6dae9e16d3d4f8de7af0e9833e1", 
        
                           ( 
        
                               "0x1bb8810e2ceaf04786d4efd216fc2820ddd9363712efc736ada11049d8af5925", 
        
                               "0x1efbf8d54c60d865cce08437668ea30f5bf90d287dbd9b5af31da852915e8f11", 
        
                           ), 
        
                       ), 
        
                       ( 
        
                           "0xba35e127276e9000b33011860904ddee28f1d48ddd3577e2a797ef4a5e62319", 
        
                           ( 
        
                               "0xda4a96147df1f35b0f820bd35c6fac3b80e8e320de7c536b1e054667b22c332", 
        
                               "0x189bd3fbffe4c8740d6543754d95c790e44cd2d162858e3b733d2b8387983bb7", 
        
                           ), 
        
                       ), 
        
                       ( 
        
                           "0x11852286660cd970e9d7f46f99c7cca2b75554245e91b9b19d537aa6147c28fc", 
        
                           ( 
        
                               "0x2ff727cfaaadb3acab713fa22d91f5fddab3ed77948f3ef6233d7ea9b03f4da1", 
        
                               "0x304080768fd2f87a852155b727f97db84b191e41970506f0326ed4046d1141aa", 
        
                           ), 
        
                       ), 
        
                       ( 
        
                           "0x174d1c85d8a690a876cc1deba0166d30569fafdb49cb3ed28405bd1c5357a1cc", 
        
                           ( 
        
                               "0x11a2eaa8e3e89de056d1b3a288a7f733c8a1282efa41d28e71af065ab245df9b", 
        
                               "0x60f37c447ac29fd97b9bb83be98ddccf15e34831a9cdf5493b7fede0777ae06", 
        
                           ), 
        
                       ), 
        
                       ( 
        
                           "0x73b81432b4cf3a8a9076201500d1b94159539f052a6e0928db7f2df74bff672", 
        
                           ( 
        
                               "0x27409dccc6ee4ce90e24744fda8d72c0bc64e79766f778da0c1c0ef1c186ea84", 
        
                               "0x1ac201a542feca15e77f30370da183514dc99d8a0b2c136d64ede35cd0b51dc0", 
        
                           ), 
        
                       ), 
        
                   ] 
        
               ) 
        
           );

Which hold a lot of unnecessary #[cfg(test)].

CPerezz · 2024-01-22T08:49:16Z

src/bn256/curve.rs

+    [
+        0x8b17ea66b99c90dd,
+        0x5bfc41088d8daaa7,
+        0xb3c4d79d41a91758,
+        0x00,
+    ]


Can we import this from the EndoParams directly?
I see it risky to duplicate constants arround.
Also, please add some comments if there's no alternative to duplication in order to know what that is.

Um, @CPerezz
Can you help me here?
I wanted to add the comments related to z_other.
But, I failed to find the parameter or algorithm related to ZETA and z_other. 😢
(Just updated the "endo" testing macro to receive the z_other param as optional here)

Can't we compute it by ourseleves here or something similar? I think it makes it complex if we indeed need to pass more parameters to the macro.

Maybe if it gets too complex we can leave it as is and adding a comment saying how the number was obtained.

I find the z_other in this test quite absurd.
It is checking that a variable that is declared in the test is the other 3-root of unity and nothing else.
It doesn't even check that is not the value being used as ZETA.
I would remove this variable altogether and remove it from the macro as well.

Oh, I missed the outdated tag, just saw that you did exactly this. Great! :)

@CPerezz @davidnevadoc
Here, I tried to add the comments. 9ee6301

CPerezz · 2024-01-22T08:49:54Z

src/grumpkin/curve.rs

+    G1,
+    "endo",
+    [
+        0xe4bd44e5607cfd48,


Removed z_other since it is ZETA, in essence. f7f5aa8

CPerezz · 2024-01-22T08:51:05Z

src/tests/curve.rs

-        assert_eq!(affine_point, affine_point_rec);
-    }
-}
+        macro_rules! projective_to_affine_affine_to_projective {


Suggested change

macro_rules! projective_to_affine_affine_to_projective {

macro_rules! projective_affine_roundtrip {

Is shorter and means the same

Done 71069a7

duguorong009 · 2024-01-22T09:08:52Z

Thanks for the comments! @CPerezz
Will take care of them.

BTW, I think it is better to add only changes for curve testing, in this PR.
And we add the field testing macros in separate PR.
I'm afraid the PR would be unnecessarily HUGE at last.
What do you think?

CPerezz · 2024-01-22T09:24:07Z

Thanks for the comments! @CPerezz Will take care of them.

BTW, I think it is better to add only changes for curve testing, in this PR. And we add the field testing macros in separate PR. I'm afraid the PR would be unnecessarily HUGE at last. What do you think?

Sounds good to me!!!

Address all the stuff and set this to ready and I'll review ASAP! On the meantime, you can branch out from here and start the Field testing work if you want!

duguorong009 · 2024-01-22T14:37:41Z

Instead of having a #[cfg(test)] on each test fn, can't you have a module which is under the test flag and have all the macros without it inside.

I talk about thinks like this:

halo2curves/src/bn256/curve.rs

Lines 201 to 273 in 762a329

#[cfg(test)]

crate::curve_testing_suite!(G1, G2);

#[cfg(test)]

crate::curve_testing_suite!(G1, "hash_to_curve");

#[cfg(test)]

crate::curve_testing_suite!(G1, "endo_consistency");

#[cfg(test)]

crate::curve_testing_suite!(

G1,

"endo",

[

0x8b17ea66b99c90dd,

0x5bfc41088d8daaa7,

0xb3c4d79d41a91758,

0x00,

]

);

#[cfg(test)]

crate::curve_testing_suite!(

G1,

"svdw_map_to_curve",

(

// Precomputed constants taken from https://github.com/ConsenSys/gnark-crypto/blob/441dc0ffe639294b8d09e394f24ba7575577229c/internal/generator/config/bn254.go#L26-L32.

[

"4",

"10944121435919637611123202872628637544348155578648911831344518947322613104291",

"8815841940592487685674414971303048083897117035520822607866",

"7296080957279758407415468581752425029565437052432607887563012631548408736189",

],

// List of (u, (Q.x, Q.y)) taken from https://github.com/ConsenSys/gnark-crypto/blob/441dc0ffe639294b8d09e394f24ba7575577229c/ecc/bn254/hash_vectors_test.go#L4-L28

[

(

"0xcb81538a98a2e3580076eed495256611813f6dae9e16d3d4f8de7af0e9833e1",

(

"0x1bb8810e2ceaf04786d4efd216fc2820ddd9363712efc736ada11049d8af5925",

"0x1efbf8d54c60d865cce08437668ea30f5bf90d287dbd9b5af31da852915e8f11",

),

),

(

"0xba35e127276e9000b33011860904ddee28f1d48ddd3577e2a797ef4a5e62319",

(

"0xda4a96147df1f35b0f820bd35c6fac3b80e8e320de7c536b1e054667b22c332",

"0x189bd3fbffe4c8740d6543754d95c790e44cd2d162858e3b733d2b8387983bb7",

),

),

(

"0x11852286660cd970e9d7f46f99c7cca2b75554245e91b9b19d537aa6147c28fc",

(

"0x2ff727cfaaadb3acab713fa22d91f5fddab3ed77948f3ef6233d7ea9b03f4da1",

"0x304080768fd2f87a852155b727f97db84b191e41970506f0326ed4046d1141aa",

),

),

(

"0x174d1c85d8a690a876cc1deba0166d30569fafdb49cb3ed28405bd1c5357a1cc",

(

"0x11a2eaa8e3e89de056d1b3a288a7f733c8a1282efa41d28e71af065ab245df9b",

"0x60f37c447ac29fd97b9bb83be98ddccf15e34831a9cdf5493b7fede0777ae06",

),

),

(

"0x73b81432b4cf3a8a9076201500d1b94159539f052a6e0928db7f2df74bff672",

(

"0x27409dccc6ee4ce90e24744fda8d72c0bc64e79766f778da0c1c0ef1c186ea84",

"0x1ac201a542feca15e77f30370da183514dc99d8a0b2c136d64ede35cd0b51dc0",

),

),

]

)

);

Which hold a lot of unnecessary #[cfg(test)].

Try to remove the unnecessary attrs in d205675

davidnevadoc · 2024-01-22T18:55:56Z

src/tests/curve.rs

+            ($c: ident) => {
+                assert!(bool::from($c::identity().is_on_curve()));
+                assert!(bool::from($c::generator().is_on_curve()));
+                assert!(bool::from($c::identity().is_on_curve()));


Why are these checks duplicated?

Thanks for indication! @davidnevadoc
Here, I removed the duplicates. 9ee6301

davidnevadoc · 2024-01-22T19:08:50Z

src/bn256/curve.rs

+    [
+        0x8b17ea66b99c90dd,
+        0x5bfc41088d8daaa7,
+        0xb3c4d79d41a91758,
+        0x00,
+    ]


I find the z_other in this test quite absurd.
It is checking that a variable that is declared in the test is the other 3-root of unity and nothing else.
It doesn't even check that is not the value being used as ZETA.
I would remove this variable altogether and remove it from the macro as well.

duguorong009 · 2024-01-23T04:12:19Z

@CPerezz @davidnevadoc
Please review this commit e1c92b5.
This is for checking the curve constants/params.

CPerezz

LGTM!!

Great work!! 🥳

CPerezz · 2024-01-23T08:17:57Z

src/tests/curve.rs

+                    println!(
+                        "{:?} \n{:?}",
+                        projective_repr.as_ref(),
+                        affine_repr.as_ref()
+                    );


Oops, removing now. ( 😅 )
d13e9dd

duguorong009 and others added 9 commits January 17, 2024 14:37

refactor: move tests into "tests" module

89848d4

improve: add edge cases(ONE and ZERO tests) to field testing suite

80d136a

feat: add some macros for curve tests (0)

61cdc56

feat: add the "curve_testing_suite" macro

f965325

feat: add "test_endo_consistency" to "curve_testing_suite" macro

6b5406b

refactor: small refactoring in "secp256k1" curve tests

655f9de

chore: roll back useless refactorings

928da66

feat: add basic "field_testing_suite" macro

455f0f4

fix: use the field testing macro for "secp256k1" & "secp256r1"

f58f42b

CPerezz reviewed Jan 18, 2024

View reviewed changes

src/tests/curve.rs Outdated Show resolved Hide resolved

src/tests/curve.rs Outdated Show resolved Hide resolved

src/tests/curve.rs Outdated Show resolved Hide resolved

duguorong009 added 4 commits January 19, 2024 15:38

fix: replace curve test functions with macros

7c8501d

fix: use the exported macro for curve testing

42fb4f1

fix: add missing attr to "random_serde_test" macro

2f42753

fix: fix the small error in "field_testing_suite" macro

51b49ae

duguorong009 added 6 commits January 19, 2024 22:25

feat: add "endo_consistency" macro branch

c64921d

feat: add the "endo" macro branch

7fcbbc3

feat: add the "ecdsa_example" macro branch

64ab44c

fix: add type casting in "endo" macro branch

3238f4d

feat: add the "map_to_curve" macro branch

c5c1d41

feat: add the "hash_to_curve" macro branch

5c7e860

chore: roll back the field testing refactorings

762a329

CPerezz reviewed Jan 22, 2024

View reviewed changes

Merge branch 'main' into gr@refactor-tests

7db46ab

duguorong009 added 3 commits January 22, 2024 17:32

chore: use better naming for macro

71069a7

fix: update the "endo" testing macro

f7f5aa8

chore: remove the unnecessary "#[cfg(test)]" attrs

d205675

duguorong009 marked this pull request as ready for review January 22, 2024 14:37

duguorong009 changed the title ~~improve: add some macros to generate big testing suite of curves and fields~~ improve: add some macros to generate big testing suite of curves Jan 22, 2024

duguorong009 requested a review from CPerezz January 22, 2024 14:40

davidnevadoc self-requested a review January 22, 2024 15:01

davidnevadoc reviewed Jan 22, 2024

View reviewed changes

duguorong009 added 2 commits January 23, 2024 11:06

chore: remove duplicates & add comments

9ee6301

feat: add the macro to test the curve constants/params

e1c92b5

duguorong009 requested a review from davidnevadoc January 23, 2024 05:19

CPerezz approved these changes Jan 23, 2024

View reviewed changes

chore: remove the leftover(println)

d13e9dd

duguorong009 added this pull request to the merge queue Jan 23, 2024

Merged via the queue into privacy-scaling-explorations:main with commit 53dd906 Jan 23, 2024
11 checks passed

CPerezz mentioned this pull request Feb 8, 2024

bump: v0.6.1 #138

Merged

jonathanpwang mentioned this pull request Apr 24, 2024

[feat] v0.6.1 from upstream (cyclone MSM) axiom-crypto/halo2curves#27

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

improve: add some macros to generate big testing suite of curves #129

improve: add some macros to generate big testing suite of curves #129

duguorong009 commented Jan 18, 2024 •

edited

Loading

CPerezz left a comment •

edited

Loading

CPerezz commented Jan 19, 2024

duguorong009 commented Jan 19, 2024

CPerezz left a comment

CPerezz Jan 22, 2024

duguorong009 Jan 22, 2024

CPerezz Jan 22, 2024

davidnevadoc Jan 22, 2024

davidnevadoc Jan 22, 2024

duguorong009 Jan 23, 2024

CPerezz Jan 22, 2024

duguorong009 Jan 22, 2024

CPerezz Jan 22, 2024

duguorong009 Jan 22, 2024

duguorong009 commented Jan 22, 2024 •

edited

Loading

CPerezz commented Jan 22, 2024

duguorong009 commented Jan 22, 2024

davidnevadoc Jan 22, 2024

duguorong009 Jan 23, 2024

davidnevadoc Jan 22, 2024

duguorong009 commented Jan 23, 2024

CPerezz left a comment

CPerezz Jan 23, 2024

duguorong009 Jan 23, 2024

	#[cfg(test)]
	crate::curve_testing_suite!(G1, G2);

	#[cfg(test)]
	crate::curve_testing_suite!(G1, "hash_to_curve");

	#[cfg(test)]
	crate::curve_testing_suite!(G1, "endo_consistency");

	#[cfg(test)]
	crate::curve_testing_suite!(
	G1,
	"endo",
	[
	0x8b17ea66b99c90dd,
	0x5bfc41088d8daaa7,
	0xb3c4d79d41a91758,
	0x00,
	]
	);

	#[cfg(test)]
	crate::curve_testing_suite!(
	G1,
	"svdw_map_to_curve",
	(
	// Precomputed constants taken from https://github.com/ConsenSys/gnark-crypto/blob/441dc0ffe639294b8d09e394f24ba7575577229c/internal/generator/config/bn254.go#L26-L32.
	[
	"4",
	"10944121435919637611123202872628637544348155578648911831344518947322613104291",
	"8815841940592487685674414971303048083897117035520822607866",
	"7296080957279758407415468581752425029565437052432607887563012631548408736189",
	],
	// List of (u, (Q.x, Q.y)) taken from https://github.com/ConsenSys/gnark-crypto/blob/441dc0ffe639294b8d09e394f24ba7575577229c/ecc/bn254/hash_vectors_test.go#L4-L28
	[
	(
	"0xcb81538a98a2e3580076eed495256611813f6dae9e16d3d4f8de7af0e9833e1",
	(
	"0x1bb8810e2ceaf04786d4efd216fc2820ddd9363712efc736ada11049d8af5925",
	"0x1efbf8d54c60d865cce08437668ea30f5bf90d287dbd9b5af31da852915e8f11",
	),
	),
	(
	"0xba35e127276e9000b33011860904ddee28f1d48ddd3577e2a797ef4a5e62319",
	(
	"0xda4a96147df1f35b0f820bd35c6fac3b80e8e320de7c536b1e054667b22c332",
	"0x189bd3fbffe4c8740d6543754d95c790e44cd2d162858e3b733d2b8387983bb7",
	),
	),
	(
	"0x11852286660cd970e9d7f46f99c7cca2b75554245e91b9b19d537aa6147c28fc",
	(
	"0x2ff727cfaaadb3acab713fa22d91f5fddab3ed77948f3ef6233d7ea9b03f4da1",
	"0x304080768fd2f87a852155b727f97db84b191e41970506f0326ed4046d1141aa",
	),
	),
	(
	"0x174d1c85d8a690a876cc1deba0166d30569fafdb49cb3ed28405bd1c5357a1cc",
	(
	"0x11a2eaa8e3e89de056d1b3a288a7f733c8a1282efa41d28e71af065ab245df9b",
	"0x60f37c447ac29fd97b9bb83be98ddccf15e34831a9cdf5493b7fede0777ae06",
	),
	),
	(
	"0x73b81432b4cf3a8a9076201500d1b94159539f052a6e0928db7f2df74bff672",
	(
	"0x27409dccc6ee4ce90e24744fda8d72c0bc64e79766f778da0c1c0ef1c186ea84",
	"0x1ac201a542feca15e77f30370da183514dc99d8a0b2c136d64ede35cd0b51dc0",
	),
	),
	]
	)
	);

	macro_rules! projective_to_affine_affine_to_projective {
	macro_rules! projective_affine_roundtrip {

improve: add some macros to generate big testing suite of curves #129

improve: add some macros to generate big testing suite of curves #129

Conversation

duguorong009 commented Jan 18, 2024 • edited Loading

Description

Related issues

Changes

CPerezz left a comment • edited Loading

Choose a reason for hiding this comment

CPerezz commented Jan 19, 2024

duguorong009 commented Jan 19, 2024

CPerezz left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

duguorong009 commented Jan 22, 2024 • edited Loading

CPerezz commented Jan 22, 2024

duguorong009 commented Jan 22, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

duguorong009 commented Jan 23, 2024

CPerezz left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

duguorong009 commented Jan 18, 2024 •

edited

Loading

CPerezz left a comment •

edited

Loading

duguorong009 commented Jan 22, 2024 •

edited

Loading