Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: ignore code block for doctest #52

Merged
merged 1 commit into from
May 23, 2023
Merged

Conversation

jonathanpwang
Copy link
Contributor

No description provided.

@jonathanpwang jonathanpwang merged commit 3fce68c into release-0.3.0 May 23, 2023
@jonathanpwang jonathanpwang deleted the fix/doctest branch May 23, 2023 02:39
jonathanpwang added a commit that referenced this pull request May 23, 2023
jonathanpwang added a commit that referenced this pull request May 23, 2023
commit 6256db1
Author: Jonathan Wang <jonathanpwang@users.noreply.github.com>
Date:   Mon May 22 22:31:13 2023 -0700

    chore: sync with release-0.3.0 and update CI

commit 096e370
Author: Jonathan Wang <31040440+jonathanpwang@users.noreply.github.com>
Date:   Mon May 22 21:48:28 2023 -0500

    feat: add docs and assert with non-empty array checks (#53)

commit fb7352a
Author: Jonathan Wang <31040440+jonathanpwang@users.noreply.github.com>
Date:   Mon May 22 21:39:19 2023 -0500

    fix: ignore code block for doctest (#52)

commit dcfee63
Author: Jonathan Wang <31040440+jonathanpwang@users.noreply.github.com>
Date:   Mon May 22 20:37:20 2023 -0500

    feat: add Github CI running tests (#51)

commit ae152e0
Author: Jonathan Wang <31040440+jonathanpwang@users.noreply.github.com>
Date:   Mon May 22 20:20:31 2023 -0500

    chore: add assert for query_cell_at_pos (#50)

commit 7f429a4
Author: Jonathan Wang <31040440+jonathanpwang@users.noreply.github.com>
Date:   Mon May 22 18:51:56 2023 -0500

    feat: `fixed_base::msm_par` handles identity point (#48)

    We still require fixed base points to be non-identity, but now handle
    the case when scalars may be zero or the final MSM value is identity
    point.

commit 2d2c7ff
Author: Jonathan Wang <31040440+jonathanpwang@users.noreply.github.com>
Date:   Mon May 22 17:41:17 2023 -0500

    fix: minor code quality fixes (#47)

commit 0fff063
Author: Jonathan Wang <31040440+jonathanpwang@users.noreply.github.com>
Date:   Mon May 22 17:16:07 2023 -0500

    feat: add `parallelize_in` helper function (#46)

    Multi-threading of witness generation is tricky because one has to
    ensure the circuit column assignment order stays deterministic. To
    ensure good developer experience / avoiding pitfalls, we provide a new
    helper function for this.

    Co-authored-by: Jonathan Wang <jonathanpwang@users.noreply.github.com>

commit 805a21c
Author: Jonathan Wang <31040440+jonathanpwang@users.noreply.github.com>
Date:   Mon May 22 13:45:15 2023 -0500

    feat: use strict ec ops more often (#45)

    * `msm` implementations now always use `ec_{add,sub}_unequal` in strict
    mode for safety
    * Add docs to `scalar_multiply` and a flag to specify when it's safe to
      turn off some strict assumptions

commit 2c276b4
Author: Jonathan Wang <31040440+jonathanpwang@users.noreply.github.com>
Date:   Mon May 22 13:25:50 2023 -0500

    Better handling of EC point at infinity (#44)

    * feat: allow `msm_par` to return identity point

    * feat: handle point at infinity

    `multi_scalar_multiply` and `multi_exp_par` now handle point at infinity
    completely

    Add docs for `ec_add_unequal, ec_sub_unequal, ec_double_and_add_unequal`
    to specify point at infinity leads to undefined behavior

commit 8e9032c
Author: Jonathan Wang <31040440+jonathanpwang@users.noreply.github.com>
Date:   Mon May 22 03:52:52 2023 -0500

    Use new types to validate input assumptions (#43)

    * feat: add new types `ProperUint` and `ProperCrtUint`

    To guard around assumptions about big integer representations

    * fix: remove unused `FixedAssignedCRTInteger`

    * feat: use new types for bigint and field chips

    New types now guard for different assumptions on non-native bigint
    arithmetic. Distinguish between:
    - Overflow CRT integers
    - Proper BigUint with native part derived from limbs
    - Field elements where inequality < modulus is checked

    Also add type to help guard for inequality check in
    ec_add_unequal_strict

    Rust traits did not play so nicely with references, so I had to switch
    many functions to move inputs instead of borrow by reference. However to
    avoid writing `clone` everywhere, we allow conversion `From` reference
    to the new type via cloning.

    * feat: use `ProperUint` for `big_less_than`

    * feat(ecc): add fns for assign private witness points

    that constrain point to lie on curve

    * fix: unnecessary lifetimes

    * chore: remove clones

commit 01a8ac9
Author: Jonathan Wang <31040440+jonathanpwang@users.noreply.github.com>
Date:   Fri May 19 13:33:14 2023 -0500

    fix: `FieldChip::divide` renamed `divide_unsafe` (#41)

    Add `divide` that checks denomintor is nonzero.
    Add documentation in cases where `divide_unsafe` is used.

commit 4fdafab
Author: Jonathan Wang <31040440+jonathanpwang@users.noreply.github.com>
Date:   Fri May 19 01:23:53 2023 -0500

    Add documentation for all debug_asserts (#40)

    feat: add documentation for all debug_asserts

commit 98f8b1d
Author: Jonathan Wang <31040440+jonathanpwang@users.noreply.github.com>
Date:   Fri May 19 00:19:30 2023 -0500

    fix: get_last_bit two errors (#39)

    2 embarassing errors:
    * Witness gen for last bit was wrong (used xor instead of &)
    * `ctx.get` was called after `range_check` so it was getting the wrong
      cell

commit 9298a78
Author: Jonathan Wang <31040440+jonathanpwang@users.noreply.github.com>
Date:   Thu May 18 22:09:54 2023 -0500

    Guard `ScalarField` byte representations to always be little-endian (#38)

    fix: guard `ScalarField` to be little-endian

commit b33af5b
Author: Jonathan Wang <31040440+jonathanpwang@users.noreply.github.com>
Date:   Thu May 18 19:52:51 2023 -0500

    fix: `log2_ceil(0)` should return `0` (#37)

commit c685862
Author: Jonathan Wang <31040440+jonathanpwang@users.noreply.github.com>
Date:   Thu May 18 19:46:07 2023 -0500

    fix(ecdsa): allow u1*G == u2*PK case (#36)

    NOTE: current ecdsa requires `r, s` to be given as proper CRT integers

    TODO: newtypes to guard this assumption

commit 16a3e9d
Author: PatStiles <33334338+PatStiles@users.noreply.github.com>
Date:   Thu May 18 16:35:11 2023 -0500

    Feat: extend halo2 base test coverage (#35)

    * feat: add flex_gate_test.rs and pos add() test

    * feat: add pos sub() test

    * feat: add pos neg() test

    * feat: add pos mul() test

    * feat: add pos mul_add() test

    * feat: add pos mul_not() test

    * feat: add pos assert_bit

    * feat: add pos div_unsafe() test

    * feat: add pos assert_is_const test

    * feat: add pos inner_product() test

    * feat: add pos inner_product_left_last() test

    * feat: add pos inner_product_with_sums test

    * feat: add pos sum_products_with_coeff_and_var test

    * feat: add pos and() test

    * feat: add pos not() test

    * feat: add pos select() test

    * feat: add pos or_and() test

    * feat: add pos bits_to_indicator() test

    * feat: add pos idx_to_indicator() test

    * feat: add pos select_by_indicator() test

    * feat: add pos select_from_idx() test

    * feat: add pos is_zero() test

    * feat: add pos is_equal() test

    * feat: add pos num_to_bits() test

    * feat: add pos lagrange_eval() test

    * feat: add pos get_field_element() test

    * feat: add pos range_check() tests

    * feat: add pos check_less_than() test

    * feat: add pos check_less_than_safe() test

    * feat: add pos check_big_less_than_safe() test

    * feat: add pos is_less_than() test

    * feat: add pos is_less_than_safe() test

    * feat: add pos is_big_less_than_safe() test

    * feat: add pos div_mod() test

    * feat: add pos get_last_bit() test

    * feat: add pos div_mod_var() test

    * fix: pass slices into test functions not arrays

    * feat: Add pos property tests for flex_gate

    * feat: Add positive property tests for flex_gate

    * feat: add pos property tests for range_check.rs

    * feat: add neg pranking test for idx_to_indicator

    * fix: change div_mod_var test values

    * feat(refactor): refactor property tests

    * fix: fix neg test, assert_const, assert_bit

    * fix: failing prop tests

    * feat: expand negative testing is_less_than_failing

    * fix: Circuit overflow errors on neg tests

    * fix: prop_test_mul_not

    * fix: everything but get_last_bit & lagrange

    * fix: clippy

    * fix: set LOOKUP_BITS in range tests, make range check neg test more

    robust

    * fix: neg_prop_tests cannot prank inputs

    Inputs have many copy constraints; pranking initial input will cause all
    copy constraints to fail

    * fix: test_is_big_less_than_safe, 240 bits max

    * Didn't want to change current `is_less_than` implementation, which in
      order to optimize lookups for smaller bits, only works when inputs
    have at most `(F::CAPACITY // lookup_bits - 1) * lookup_bits` bits

    * fix: inline doc for lagrange_and_eval

    * Remove proptest for lagrange_and_eval and leave as todo

    * tests: add readme about serial execution

    ---------

    Co-authored-by: Jonathan Wang <jonathanpwang@users.noreply.github.com>

commit 44bc744
Author: yuliakot <93175658+yuliakot@users.noreply.github.com>
Date:   Sat May 20 00:24:18 2023 -0500

    Update ecdsa.rs

commit 33fcccc
Author: yuliakot <93175658+yuliakot@users.noreply.github.com>
Date:   Sat May 20 00:23:18 2023 -0500

    Update ecdsa.rs

commit c42a2ef
Author: yuliakot <93175658+yuliakot@users.noreply.github.com>
Date:   Sat May 20 00:22:51 2023 -0500

    Update ecdsa.rs

commit e9386b9
Author: yuliakot <93175658+yuliakot@users.noreply.github.com>
Date:   Sat May 20 00:20:23 2023 -0500

    Update tests.rs

commit 150b88f
Author: yuliakot <93175658+yuliakot@users.noreply.github.com>
Date:   Sat May 20 00:18:37 2023 -0500

    Update mod.rs

commit da7d5b1
Author: yulliakot <yulliakotel@gmail.com>
Date:   Fri May 19 23:57:48 2023 -0500

    More ecdsa tests
jonathanpwang added a commit that referenced this pull request May 23, 2023
* Feat: extend halo2 base test coverage (#35)

* feat: add flex_gate_test.rs and pos add() test

* feat: add pos sub() test

* feat: add pos neg() test

* feat: add pos mul() test

* feat: add pos mul_add() test

* feat: add pos mul_not() test

* feat: add pos assert_bit

* feat: add pos div_unsafe() test

* feat: add pos assert_is_const test

* feat: add pos inner_product() test

* feat: add pos inner_product_left_last() test

* feat: add pos inner_product_with_sums test

* feat: add pos sum_products_with_coeff_and_var test

* feat: add pos and() test

* feat: add pos not() test

* feat: add pos select() test

* feat: add pos or_and() test

* feat: add pos bits_to_indicator() test

* feat: add pos idx_to_indicator() test

* feat: add pos select_by_indicator() test

* feat: add pos select_from_idx() test

* feat: add pos is_zero() test

* feat: add pos is_equal() test

* feat: add pos num_to_bits() test

* feat: add pos lagrange_eval() test

* feat: add pos get_field_element() test

* feat: add pos range_check() tests

* feat: add pos check_less_than() test

* feat: add pos check_less_than_safe() test

* feat: add pos check_big_less_than_safe() test

* feat: add pos is_less_than() test

* feat: add pos is_less_than_safe() test

* feat: add pos is_big_less_than_safe() test

* feat: add pos div_mod() test

* feat: add pos get_last_bit() test

* feat: add pos div_mod_var() test

* fix: pass slices into test functions not arrays

* feat: Add pos property tests for flex_gate

* feat: Add positive property tests for flex_gate

* feat: add pos property tests for range_check.rs

* feat: add neg pranking test for idx_to_indicator

* fix: change div_mod_var test values

* feat(refactor): refactor property tests

* fix: fix neg test, assert_const, assert_bit

* fix: failing prop tests

* feat: expand negative testing is_less_than_failing

* fix: Circuit overflow errors on neg tests

* fix: prop_test_mul_not

* fix: everything but get_last_bit & lagrange

* fix: clippy

* fix: set LOOKUP_BITS in range tests, make range check neg test more

robust

* fix: neg_prop_tests cannot prank inputs

Inputs have many copy constraints; pranking initial input will cause all
copy constraints to fail

* fix: test_is_big_less_than_safe, 240 bits max

* Didn't want to change current `is_less_than` implementation, which in
  order to optimize lookups for smaller bits, only works when inputs
have at most `(F::CAPACITY // lookup_bits - 1) * lookup_bits` bits

* fix: inline doc for lagrange_and_eval

* Remove proptest for lagrange_and_eval and leave as todo

* tests: add readme about serial execution

---------

Co-authored-by: Jonathan Wang <jonathanpwang@users.noreply.github.com>

* fix(ecdsa): allow u1*G == u2*PK case (#36)

NOTE: current ecdsa requires `r, s` to be given as proper CRT integers

TODO: newtypes to guard this assumption

* fix: `log2_ceil(0)` should return `0` (#37)

* Guard `ScalarField` byte representations to always be little-endian (#38)

fix: guard `ScalarField` to be little-endian

* fix: get_last_bit two errors (#39)

2 embarassing errors:
* Witness gen for last bit was wrong (used xor instead of &)
* `ctx.get` was called after `range_check` so it was getting the wrong
  cell

* Add documentation for all debug_asserts (#40)

feat: add documentation for all debug_asserts

* fix: `FieldChip::divide` renamed `divide_unsafe` (#41)

Add `divide` that checks denomintor is nonzero.
Add documentation in cases where `divide_unsafe` is used.

* Use new types to validate input assumptions (#43)

* feat: add new types `ProperUint` and `ProperCrtUint`

To guard around assumptions about big integer representations

* fix: remove unused `FixedAssignedCRTInteger`

* feat: use new types for bigint and field chips

New types now guard for different assumptions on non-native bigint
arithmetic. Distinguish between:
- Overflow CRT integers
- Proper BigUint with native part derived from limbs
- Field elements where inequality < modulus is checked

Also add type to help guard for inequality check in
ec_add_unequal_strict

Rust traits did not play so nicely with references, so I had to switch
many functions to move inputs instead of borrow by reference. However to
avoid writing `clone` everywhere, we allow conversion `From` reference
to the new type via cloning.

* feat: use `ProperUint` for `big_less_than`

* feat(ecc): add fns for assign private witness points

that constrain point to lie on curve

* fix: unnecessary lifetimes

* chore: remove clones

* Better handling of EC point at infinity (#44)

* feat: allow `msm_par` to return identity point

* feat: handle point at infinity

`multi_scalar_multiply` and `multi_exp_par` now handle point at infinity
completely

Add docs for `ec_add_unequal, ec_sub_unequal, ec_double_and_add_unequal`
to specify point at infinity leads to undefined behavior

* feat: use strict ec ops more often (#45)

* `msm` implementations now always use `ec_{add,sub}_unequal` in strict
mode for safety
* Add docs to `scalar_multiply` and a flag to specify when it's safe to
  turn off some strict assumptions

* feat: add `parallelize_in` helper function (#46)

Multi-threading of witness generation is tricky because one has to
ensure the circuit column assignment order stays deterministic. To
ensure good developer experience / avoiding pitfalls, we provide a new
helper function for this.

Co-authored-by: Jonathan Wang <jonathanpwang@users.noreply.github.com>

* fix: minor code quality fixes (#47)

* feat: `fixed_base::msm_par` handles identity point (#48)

We still require fixed base points to be non-identity, but now handle
the case when scalars may be zero or the final MSM value is identity
point.

* chore: add assert for query_cell_at_pos (#50)

* feat: add Github CI running tests (#51)

* fix: ignore code block for doctest (#52)

* feat: add docs and assert with non-empty array checks (#53)

* Release 0.3.0 ecdsa tests (#54)

* More ecdsa tests

* Update mod.rs

* Update tests.rs

* Update ecdsa.rs

* Update ecdsa.rs

* Update ecdsa.rs

* chore: sync with release-0.3.0 and update CI

Co-authored-by: yulliakot <yulliakotel@gmail.com>
Co-authored-by: yuliakot <93175658+yuliakot@users.noreply.github.com>

* chore: fix CI

cannot multi-thread tests involving lookups due to environment variables

* fix: git CI for community-edition

smaller test cases for test 'bench_msm'

---------

Co-authored-by: PatStiles <33334338+PatStiles@users.noreply.github.com>
Co-authored-by: Jonathan Wang <jonathanpwang@users.noreply.github.com>
Co-authored-by: yulliakot <yulliakotel@gmail.com>
Co-authored-by: yuliakot <93175658+yuliakot@users.noreply.github.com>
jonathanpwang added a commit that referenced this pull request Jun 20, 2023
* feat: upgrade overall `halo2-base` API to support future multi-threaded
assignments using our basic gate

* WIP: currently `gates::flex_gate` is updated and passes basic test

* BUG: `GateInstructions::idx_to_indicator` was missing a constraint to
check that the indicator witness was equal to 1 when non-zero.
* Previously the constraint ensured that `ind[i] = 0` when `idx != i`
  however `ind[idx]` could be anything!!!

* update: working benches for `mul` and `inner_product`

* feat: add `test_multithread_gates`

* BUG: `get_last_bit` did not do an `assert_bit` check on the answer

* this function was not used anywhere

* fix: `builder::assign_*` was not handling cases where two gates overlap
and there is a break point in that overlap
* we need to copy a cell between columns to fix

* feat: update `gates::range` to working tests and new API

* In keygen mode, the `CircuitBuilder` will clone the `ThreadBuilder`
  instead of `take`ing it because the same circuit is used for both vk
gen and pk gen. This could lead to more memory usage for pk gen.

* fix: change `AssignedValue` type to `KeccakAssignedValue` for
compatibility after halo2-base update

* Initial version 0.3.0 of halo2-ecc (#12)

* add multi-thread witness assignment support for `variable_base_msm`
  and `fixed_base_msm`
* batch size 100 MSM witness generation went from 500ms -> 100ms

* Sync with updates in `halo2_proofs_axiom`
* `assign_advice` not longer returns `Result` so no more `unwrap`

* Fix: assert uses of size hint in release mode (#13)

* remove `size_hint` in `inner_product_simple`

* change other uses of `size_hint` to follow with `assert_eq!` instead
  of `debug_assert_eq!`

* Fix: bit decomposition edge cases (#14)

* fix: change `debug_assert` in `decompose_u64_digits_limbs` to restrict
`bit_len < 64` and `decompose_biguint` to `64 <= bit_len < 128`
* add more comprehensive tests for above two functions

* Initial checkpoint for halo2-ecc v0.3.0 (#15)

* chore: clippy --fix

* Feat/add readme (#4)

* feat: add README

* feat: re-enable `secp256k1` module with updated tests

* chore: fix result println

* chore: update Cargo halo2_proofs_axiom to axiom/dev branch

* compatibility update with `halo2_proofs_axiom`

Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de>

* Fix: make `GateThreadBuilder` compatible with external usage (#16)

* chore: expose gate_builder.unknown

* feat: `GateThreadBuilder::assign_all` takes assigned_{advices,constants}
    as input instead of new hashmap, in case we want to constrain equalities
    for cells not belonging to this builder

* chore: update halo2-pse tag

* fix: `GateThreadBuilder::assign_all` now returns `HashMap`s of
    assigned cells for external equality constraints (e.g., instance cells,
    `AssignedCells` from chips not using halo2-lib).

* fix: `assign_all` was not assigning constants as desired: it was
    assigning a new constant per context. This leads to confusion and
    possible undesired consequences down the line.

* Fix: under-constrained `idx_to_indicator` (#17)

*fix(BUG): `GateChip::idx_to_indicator` still had soundness bug where at index
`idx` the value could be 0 or 1 (instead of only 1)

* feat: add some function documentation

* test(idx_to_indicator): add comprehensive tests
* both positive and negative tests

* Fix: soundness error in `FpChip::assert_eq` due to typo (#18)

* chore: update halo2-ecc version to 0.3.0

* fix(BUG): `FpChip::assert_equal` had `a` instead of `b` typo

* feat: add tests for `FpChip::assert_eq`
* positive and negative tests

* Remove redundant code and prevent race conditions (#19)

* feat: move `GateCircuitBuilder::synthesize` to `sub_synthesize` function
which also returns the assigned advices.

* reduces code duplication between `GateCircuitBuilder::synthesize` and
  `RangeCircuitBuilder::synthesize` and also makes it easier to assign
public instances elsewhere (e.g., snark-verifier)

* feat: remove `Mutex` to prevent non-deterministism

* In variable and fixed base `msm_par` functions, remove use of
  `Mutex<GateThreadBuilder>` because even the `Mutex` is not thread-
  safe in the sense that: if you let `Mutex` decide order
  that `GateThreadBuilder` is unlocked, you may still add Contexts to
  the builder in a non-deterministic order.

* fix: `fixed_base::msm_par` loading new zeros

* In `msm_par` each parallelized context was loading a new zero via
  `ctx.load_zero()`
* This led to using more cells than the non-parallelized version
* In `fixed_base_msm_in`, the if statement depending on
  `rayon::current_number_threads` leads to inconsistent proving keys
between different machines. This has been removed and now uses a fixed
number `25`.

* chore: use `info!` instead of `println` for params

* Allow `assign_all` also if `witness_gen_only = true`

* Fix: `inner_product_left_last` size hint (#25)

* Add documentation for halo2-base (#27)

* adds draft documentation for range.rs

* draft docs for lib.rs, utiils.rs, builder.rs

* fix: add suggested doc edits for range.rs

* docs: add draft documentation for flex_gate.rs

* fix: range.rs doc capitalization error

* fix: suggested edits for utils.rs docs

* fix: resolve comments for range.rs docs

* fix: resolve comments on flex_gate.rs docs

* fix: resolve comments for lib.rs, util.rs docs

* fix: resolve comments for builder.rs docs

* chore: use `info!` instead of `println` for params

* Allow `assign_all` also if `witness_gen_only = true`

* Fix: `inner_product_left_last` size hint (#25)

* docs: minor fixes

---------

Co-authored-by: PatStiles <pdstiles78@gmail.com>

* Smart Range Builder (#29)

* feat: smart `RangeCircuitBuilder`

Allow `RangeCircuitBuilder` to not create lookup table if it detects
that there's nothing to look up.

* feat: add `RangeWithInstanceCircuitBuilder`

* Moved from `snark-verifier-sdk`
* Also made this circuit builder smart so it doesn't load lookup table
  if not necessary
    * In particular this can also be used as a
      `GateWithInstanceCircuitBuilder`

* chore: derive Eq for CircuitBuilderStage

* fix: RangeConfig should not unwrap LOOKUP_BITS

* fix: `div_mod_var` when `a_num_bits <= b_num_bits` (#31)

* Feat: extend halo2 base test coverage (#35)

* feat: add flex_gate_test.rs and pos add() test

* feat: add pos sub() test

* feat: add pos neg() test

* feat: add pos mul() test

* feat: add pos mul_add() test

* feat: add pos mul_not() test

* feat: add pos assert_bit

* feat: add pos div_unsafe() test

* feat: add pos assert_is_const test

* feat: add pos inner_product() test

* feat: add pos inner_product_left_last() test

* feat: add pos inner_product_with_sums test

* feat: add pos sum_products_with_coeff_and_var test

* feat: add pos and() test

* feat: add pos not() test

* feat: add pos select() test

* feat: add pos or_and() test

* feat: add pos bits_to_indicator() test

* feat: add pos idx_to_indicator() test

* feat: add pos select_by_indicator() test

* feat: add pos select_from_idx() test

* feat: add pos is_zero() test

* feat: add pos is_equal() test

* feat: add pos num_to_bits() test

* feat: add pos lagrange_eval() test

* feat: add pos get_field_element() test

* feat: add pos range_check() tests

* feat: add pos check_less_than() test

* feat: add pos check_less_than_safe() test

* feat: add pos check_big_less_than_safe() test

* feat: add pos is_less_than() test

* feat: add pos is_less_than_safe() test

* feat: add pos is_big_less_than_safe() test

* feat: add pos div_mod() test

* feat: add pos get_last_bit() test

* feat: add pos div_mod_var() test

* fix: pass slices into test functions not arrays

* feat: Add pos property tests for flex_gate

* feat: Add positive property tests for flex_gate

* feat: add pos property tests for range_check.rs

* feat: add neg pranking test for idx_to_indicator

* fix: change div_mod_var test values

* feat(refactor): refactor property tests

* fix: fix neg test, assert_const, assert_bit

* fix: failing prop tests

* feat: expand negative testing is_less_than_failing

* fix: Circuit overflow errors on neg tests

* fix: prop_test_mul_not

* fix: everything but get_last_bit & lagrange

* fix: clippy

* fix: set LOOKUP_BITS in range tests, make range check neg test more

robust

* fix: neg_prop_tests cannot prank inputs

Inputs have many copy constraints; pranking initial input will cause all
copy constraints to fail

* fix: test_is_big_less_than_safe, 240 bits max

* Didn't want to change current `is_less_than` implementation, which in
  order to optimize lookups for smaller bits, only works when inputs
have at most `(F::CAPACITY // lookup_bits - 1) * lookup_bits` bits

* fix: inline doc for lagrange_and_eval

* Remove proptest for lagrange_and_eval and leave as todo

* tests: add readme about serial execution

---------

Co-authored-by: Jonathan Wang <jonathanpwang@users.noreply.github.com>

* fix(ecdsa): allow u1*G == u2*PK case (#36)

NOTE: current ecdsa requires `r, s` to be given as proper CRT integers

TODO: newtypes to guard this assumption

* fix: `log2_ceil(0)` should return `0` (#37)

* Guard `ScalarField` byte representations to always be little-endian (#38)

fix: guard `ScalarField` to be little-endian

* fix: get_last_bit two errors (#39)

2 embarassing errors:
* Witness gen for last bit was wrong (used xor instead of &)
* `ctx.get` was called after `range_check` so it was getting the wrong
  cell

* Add documentation for all debug_asserts (#40)

feat: add documentation for all debug_asserts

* fix: `FieldChip::divide` renamed `divide_unsafe` (#41)

Add `divide` that checks denomintor is nonzero.
Add documentation in cases where `divide_unsafe` is used.

* Use new types to validate input assumptions (#43)

* feat: add new types `ProperUint` and `ProperCrtUint`

To guard around assumptions about big integer representations

* fix: remove unused `FixedAssignedCRTInteger`

* feat: use new types for bigint and field chips

New types now guard for different assumptions on non-native bigint
arithmetic. Distinguish between:
- Overflow CRT integers
- Proper BigUint with native part derived from limbs
- Field elements where inequality < modulus is checked

Also add type to help guard for inequality check in
ec_add_unequal_strict

Rust traits did not play so nicely with references, so I had to switch
many functions to move inputs instead of borrow by reference. However to
avoid writing `clone` everywhere, we allow conversion `From` reference
to the new type via cloning.

* feat: use `ProperUint` for `big_less_than`

* feat(ecc): add fns for assign private witness points

that constrain point to lie on curve

* fix: unnecessary lifetimes

* chore: remove clones

* Better handling of EC point at infinity (#44)

* feat: allow `msm_par` to return identity point

* feat: handle point at infinity

`multi_scalar_multiply` and `multi_exp_par` now handle point at infinity
completely

Add docs for `ec_add_unequal, ec_sub_unequal, ec_double_and_add_unequal`
to specify point at infinity leads to undefined behavior

* feat: use strict ec ops more often (#45)

* `msm` implementations now always use `ec_{add,sub}_unequal` in strict
mode for safety
* Add docs to `scalar_multiply` and a flag to specify when it's safe to
  turn off some strict assumptions

* feat: add `parallelize_in` helper function (#46)

Multi-threading of witness generation is tricky because one has to
ensure the circuit column assignment order stays deterministic. To
ensure good developer experience / avoiding pitfalls, we provide a new
helper function for this.

Co-authored-by: Jonathan Wang <jonathanpwang@users.noreply.github.com>

* fix: minor code quality fixes (#47)

* feat: `fixed_base::msm_par` handles identity point (#48)

We still require fixed base points to be non-identity, but now handle
the case when scalars may be zero or the final MSM value is identity
point.

* chore: add assert for query_cell_at_pos (#50)

* feat: add Github CI running tests (#51)

* fix: ignore code block for doctest (#52)

* feat: add docs and assert with non-empty array checks (#53)

* Release 0.3.0 ecdsa tests (#54)

* More ecdsa tests

* Update mod.rs

* Update tests.rs

* Update ecdsa.rs

* Update ecdsa.rs

* Update ecdsa.rs

* chore: sync with release-0.3.0 and update CI

Co-authored-by: yulliakot <yulliakotel@gmail.com>
Co-authored-by: yuliakot <93175658+yuliakot@users.noreply.github.com>

* chore: fix CI

cannot multi-thread tests involving lookups due to environment variables

* fix: `prop_test_is_less_than_safe` (#58)

This test doesn't run any prover so the input must satisfy range check
assumption. More serious coverage is provided by
`prop_test_neg_is_less_than_safe`

* Add halo2-base readme (#66)

* feat: add halo2-base readme

* fix: readme formatting

* fix: readme edits

* fix: grammer

* fix: use relative links and formatting

* fix: formatting

* feat: add RangeCircuitBuilder description

* feat: rewording and small edits

---------

Co-authored-by: PatStiles <pdstiles78@gmail.com>

* fix: change all `1` to `1u64` to prevent unexpected overflow (#72)

* [Fix] Panic when dealing with identity point (#71)

* More ecdsa tests

* Update mod.rs

* Update tests.rs

* Update ecdsa.rs

* Update ecdsa.rs

* Update ecdsa.rs

* msm tests

* Update mod.rs

* Update msm_sum_infinity.rs

* fix: ec_sub_strict was panicing when output is identity

* affects the MSM functions: right now if the answer is identity, there
  will be a panic due to divide by 0 instead of just returning 0
* there could be a more optimal solution, but due to the traits for
  EccChip, we just generate a random point solely to avoid divide by 0
in the case of identity point

* Fix/fb msm zero (#77)

* fix: fixed_base scalar multiply for [-1]P

* feat: use `multi_scalar_multiply` instead of `scalar_multiply`

* to reduce code maintanence / redundancy

* fix: add back scalar_multiply using any_point

* feat: remove flag from variable base `scalar_multiply`

* feat: add scalar multiply tests for secp256k1

* fix: variable scalar_multiply last select

* Fix/msm tests output identity (#75)

* fixed base msm tests for output infinity

* fixed base msm tests for output infinity

---------

Co-authored-by: yulliakot <yulliakotel@gmail.com>

* feat: add tests and update CI

---------

Co-authored-by: yuliakot <93175658+yuliakot@users.noreply.github.com>
Co-authored-by: yulliakot <yulliakotel@gmail.com>

---------

Co-authored-by: yulliakot <yulliakotel@gmail.com>
Co-authored-by: yuliakot <93175658+yuliakot@users.noreply.github.com>

* [Fix] scalar multiply completeness (#82)

* fix: replace `scalar_multiply` with passthrough to MSM for now

* feat(msm): use strict mode always

* Previously did not use strict because we make assumptions about the
  curve `C`. Since this was not documented and is easy to miss, we use
strict mode always.

* docs: add assumptions to ec_sub_strict (#84)

* fix: readme from previous merge

* chore: cleanup CI for merge into main

* chore: fix readme

---------

Co-authored-by: Jonathan Wang <jonathanpwang@users.noreply.github.com>
Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de>
Co-authored-by: PatStiles <pdstiles78@gmail.com>
Co-authored-by: PatStiles <33334338+PatStiles@users.noreply.github.com>
Co-authored-by: yulliakot <yulliakotel@gmail.com>
Co-authored-by: yuliakot <93175658+yuliakot@users.noreply.github.com>
jonathanpwang added a commit that referenced this pull request Sep 21, 2023
* fix: change all `1` to `1u64` to prevent unexpected overflow (#72)

* [Fix] Panic when dealing with identity point (#71)

* More ecdsa tests

* Update mod.rs

* Update tests.rs

* Update ecdsa.rs

* Update ecdsa.rs

* Update ecdsa.rs

* msm tests

* Update mod.rs

* Update msm_sum_infinity.rs

* fix: ec_sub_strict was panicing when output is identity

* affects the MSM functions: right now if the answer is identity, there
  will be a panic due to divide by 0 instead of just returning 0
* there could be a more optimal solution, but due to the traits for
  EccChip, we just generate a random point solely to avoid divide by 0
in the case of identity point

* Fix/fb msm zero (#77)

* fix: fixed_base scalar multiply for [-1]P

* feat: use `multi_scalar_multiply` instead of `scalar_multiply`

* to reduce code maintanence / redundancy

* fix: add back scalar_multiply using any_point

* feat: remove flag from variable base `scalar_multiply`

* feat: add scalar multiply tests for secp256k1

* fix: variable scalar_multiply last select

* Fix/msm tests output identity (#75)

* fixed base msm tests for output infinity

* fixed base msm tests for output infinity

---------

Co-authored-by: yulliakot <yulliakotel@gmail.com>

* feat: add tests and update CI

---------

Co-authored-by: yuliakot <93175658+yuliakot@users.noreply.github.com>
Co-authored-by: yulliakot <yulliakotel@gmail.com>

---------

Co-authored-by: yulliakot <yulliakotel@gmail.com>
Co-authored-by: yuliakot <93175658+yuliakot@users.noreply.github.com>

* fix: redundant check in `ec_sub_unequal`

* Add SafeType (#26)

* Add SafeType

* Refactor & add testing

* Add doc comment

* Refactor SafeChip

* Move gen_proof/check_proof to utils

* Fix merge issues

* feat(CI): switch to larger runner

* fix(builder): handle empty ctx with only equality constraints

* feat: add SafeAddress and SafeUint160 (#85)

* feat: add SafeAddress and SafeUint160

* fix incorrect byte size

* Release 0.3.0 (#86)

* feat: upgrade overall `halo2-base` API to support future multi-threaded
assignments using our basic gate

* WIP: currently `gates::flex_gate` is updated and passes basic test

* BUG: `GateInstructions::idx_to_indicator` was missing a constraint to
check that the indicator witness was equal to 1 when non-zero.
* Previously the constraint ensured that `ind[i] = 0` when `idx != i`
  however `ind[idx]` could be anything!!!

* update: working benches for `mul` and `inner_product`

* feat: add `test_multithread_gates`

* BUG: `get_last_bit` did not do an `assert_bit` check on the answer

* this function was not used anywhere

* fix: `builder::assign_*` was not handling cases where two gates overlap
and there is a break point in that overlap
* we need to copy a cell between columns to fix

* feat: update `gates::range` to working tests and new API

* In keygen mode, the `CircuitBuilder` will clone the `ThreadBuilder`
  instead of `take`ing it because the same circuit is used for both vk
gen and pk gen. This could lead to more memory usage for pk gen.

* fix: change `AssignedValue` type to `KeccakAssignedValue` for
compatibility after halo2-base update

* Initial version 0.3.0 of halo2-ecc (#12)

* add multi-thread witness assignment support for `variable_base_msm`
  and `fixed_base_msm`
* batch size 100 MSM witness generation went from 500ms -> 100ms

* Sync with updates in `halo2_proofs_axiom`
* `assign_advice` not longer returns `Result` so no more `unwrap`

* Fix: assert uses of size hint in release mode (#13)

* remove `size_hint` in `inner_product_simple`

* change other uses of `size_hint` to follow with `assert_eq!` instead
  of `debug_assert_eq!`

* Fix: bit decomposition edge cases (#14)

* fix: change `debug_assert` in `decompose_u64_digits_limbs` to restrict
`bit_len < 64` and `decompose_biguint` to `64 <= bit_len < 128`
* add more comprehensive tests for above two functions

* Initial checkpoint for halo2-ecc v0.3.0 (#15)

* chore: clippy --fix

* Feat/add readme (#4)

* feat: add README

* feat: re-enable `secp256k1` module with updated tests

* chore: fix result println

* chore: update Cargo halo2_proofs_axiom to axiom/dev branch

* compatibility update with `halo2_proofs_axiom`

Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de>

* Fix: make `GateThreadBuilder` compatible with external usage (#16)

* chore: expose gate_builder.unknown

* feat: `GateThreadBuilder::assign_all` takes assigned_{advices,constants}
    as input instead of new hashmap, in case we want to constrain equalities
    for cells not belonging to this builder

* chore: update halo2-pse tag

* fix: `GateThreadBuilder::assign_all` now returns `HashMap`s of
    assigned cells for external equality constraints (e.g., instance cells,
    `AssignedCells` from chips not using halo2-lib).

* fix: `assign_all` was not assigning constants as desired: it was
    assigning a new constant per context. This leads to confusion and
    possible undesired consequences down the line.

* Fix: under-constrained `idx_to_indicator` (#17)

*fix(BUG): `GateChip::idx_to_indicator` still had soundness bug where at index
`idx` the value could be 0 or 1 (instead of only 1)

* feat: add some function documentation

* test(idx_to_indicator): add comprehensive tests
* both positive and negative tests

* Fix: soundness error in `FpChip::assert_eq` due to typo (#18)

* chore: update halo2-ecc version to 0.3.0

* fix(BUG): `FpChip::assert_equal` had `a` instead of `b` typo

* feat: add tests for `FpChip::assert_eq`
* positive and negative tests

* Remove redundant code and prevent race conditions (#19)

* feat: move `GateCircuitBuilder::synthesize` to `sub_synthesize` function
which also returns the assigned advices.

* reduces code duplication between `GateCircuitBuilder::synthesize` and
  `RangeCircuitBuilder::synthesize` and also makes it easier to assign
public instances elsewhere (e.g., snark-verifier)

* feat: remove `Mutex` to prevent non-deterministism

* In variable and fixed base `msm_par` functions, remove use of
  `Mutex<GateThreadBuilder>` because even the `Mutex` is not thread-
  safe in the sense that: if you let `Mutex` decide order
  that `GateThreadBuilder` is unlocked, you may still add Contexts to
  the builder in a non-deterministic order.

* fix: `fixed_base::msm_par` loading new zeros

* In `msm_par` each parallelized context was loading a new zero via
  `ctx.load_zero()`
* This led to using more cells than the non-parallelized version
* In `fixed_base_msm_in`, the if statement depending on
  `rayon::current_number_threads` leads to inconsistent proving keys
between different machines. This has been removed and now uses a fixed
number `25`.

* chore: use `info!` instead of `println` for params

* Allow `assign_all` also if `witness_gen_only = true`

* Fix: `inner_product_left_last` size hint (#25)

* Add documentation for halo2-base (#27)

* adds draft documentation for range.rs

* draft docs for lib.rs, utiils.rs, builder.rs

* fix: add suggested doc edits for range.rs

* docs: add draft documentation for flex_gate.rs

* fix: range.rs doc capitalization error

* fix: suggested edits for utils.rs docs

* fix: resolve comments for range.rs docs

* fix: resolve comments on flex_gate.rs docs

* fix: resolve comments for lib.rs, util.rs docs

* fix: resolve comments for builder.rs docs

* chore: use `info!` instead of `println` for params

* Allow `assign_all` also if `witness_gen_only = true`

* Fix: `inner_product_left_last` size hint (#25)

* docs: minor fixes

---------

Co-authored-by: PatStiles <pdstiles78@gmail.com>

* Smart Range Builder (#29)

* feat: smart `RangeCircuitBuilder`

Allow `RangeCircuitBuilder` to not create lookup table if it detects
that there's nothing to look up.

* feat: add `RangeWithInstanceCircuitBuilder`

* Moved from `snark-verifier-sdk`
* Also made this circuit builder smart so it doesn't load lookup table
  if not necessary
    * In particular this can also be used as a
      `GateWithInstanceCircuitBuilder`

* chore: derive Eq for CircuitBuilderStage

* fix: RangeConfig should not unwrap LOOKUP_BITS

* fix: `div_mod_var` when `a_num_bits <= b_num_bits` (#31)

* Feat: extend halo2 base test coverage (#35)

* feat: add flex_gate_test.rs and pos add() test

* feat: add pos sub() test

* feat: add pos neg() test

* feat: add pos mul() test

* feat: add pos mul_add() test

* feat: add pos mul_not() test

* feat: add pos assert_bit

* feat: add pos div_unsafe() test

* feat: add pos assert_is_const test

* feat: add pos inner_product() test

* feat: add pos inner_product_left_last() test

* feat: add pos inner_product_with_sums test

* feat: add pos sum_products_with_coeff_and_var test

* feat: add pos and() test

* feat: add pos not() test

* feat: add pos select() test

* feat: add pos or_and() test

* feat: add pos bits_to_indicator() test

* feat: add pos idx_to_indicator() test

* feat: add pos select_by_indicator() test

* feat: add pos select_from_idx() test

* feat: add pos is_zero() test

* feat: add pos is_equal() test

* feat: add pos num_to_bits() test

* feat: add pos lagrange_eval() test

* feat: add pos get_field_element() test

* feat: add pos range_check() tests

* feat: add pos check_less_than() test

* feat: add pos check_less_than_safe() test

* feat: add pos check_big_less_than_safe() test

* feat: add pos is_less_than() test

* feat: add pos is_less_than_safe() test

* feat: add pos is_big_less_than_safe() test

* feat: add pos div_mod() test

* feat: add pos get_last_bit() test

* feat: add pos div_mod_var() test

* fix: pass slices into test functions not arrays

* feat: Add pos property tests for flex_gate

* feat: Add positive property tests for flex_gate

* feat: add pos property tests for range_check.rs

* feat: add neg pranking test for idx_to_indicator

* fix: change div_mod_var test values

* feat(refactor): refactor property tests

* fix: fix neg test, assert_const, assert_bit

* fix: failing prop tests

* feat: expand negative testing is_less_than_failing

* fix: Circuit overflow errors on neg tests

* fix: prop_test_mul_not

* fix: everything but get_last_bit & lagrange

* fix: clippy

* fix: set LOOKUP_BITS in range tests, make range check neg test more

robust

* fix: neg_prop_tests cannot prank inputs

Inputs have many copy constraints; pranking initial input will cause all
copy constraints to fail

* fix: test_is_big_less_than_safe, 240 bits max

* Didn't want to change current `is_less_than` implementation, which in
  order to optimize lookups for smaller bits, only works when inputs
have at most `(F::CAPACITY // lookup_bits - 1) * lookup_bits` bits

* fix: inline doc for lagrange_and_eval

* Remove proptest for lagrange_and_eval and leave as todo

* tests: add readme about serial execution

---------

Co-authored-by: Jonathan Wang <jonathanpwang@users.noreply.github.com>

* fix(ecdsa): allow u1*G == u2*PK case (#36)

NOTE: current ecdsa requires `r, s` to be given as proper CRT integers

TODO: newtypes to guard this assumption

* fix: `log2_ceil(0)` should return `0` (#37)

* Guard `ScalarField` byte representations to always be little-endian (#38)

fix: guard `ScalarField` to be little-endian

* fix: get_last_bit two errors (#39)

2 embarassing errors:
* Witness gen for last bit was wrong (used xor instead of &)
* `ctx.get` was called after `range_check` so it was getting the wrong
  cell

* Add documentation for all debug_asserts (#40)

feat: add documentation for all debug_asserts

* fix: `FieldChip::divide` renamed `divide_unsafe` (#41)

Add `divide` that checks denomintor is nonzero.
Add documentation in cases where `divide_unsafe` is used.

* Use new types to validate input assumptions (#43)

* feat: add new types `ProperUint` and `ProperCrtUint`

To guard around assumptions about big integer representations

* fix: remove unused `FixedAssignedCRTInteger`

* feat: use new types for bigint and field chips

New types now guard for different assumptions on non-native bigint
arithmetic. Distinguish between:
- Overflow CRT integers
- Proper BigUint with native part derived from limbs
- Field elements where inequality < modulus is checked

Also add type to help guard for inequality check in
ec_add_unequal_strict

Rust traits did not play so nicely with references, so I had to switch
many functions to move inputs instead of borrow by reference. However to
avoid writing `clone` everywhere, we allow conversion `From` reference
to the new type via cloning.

* feat: use `ProperUint` for `big_less_than`

* feat(ecc): add fns for assign private witness points

that constrain point to lie on curve

* fix: unnecessary lifetimes

* chore: remove clones

* Better handling of EC point at infinity (#44)

* feat: allow `msm_par` to return identity point

* feat: handle point at infinity

`multi_scalar_multiply` and `multi_exp_par` now handle point at infinity
completely

Add docs for `ec_add_unequal, ec_sub_unequal, ec_double_and_add_unequal`
to specify point at infinity leads to undefined behavior

* feat: use strict ec ops more often (#45)

* `msm` implementations now always use `ec_{add,sub}_unequal` in strict
mode for safety
* Add docs to `scalar_multiply` and a flag to specify when it's safe to
  turn off some strict assumptions

* feat: add `parallelize_in` helper function (#46)

Multi-threading of witness generation is tricky because one has to
ensure the circuit column assignment order stays deterministic. To
ensure good developer experience / avoiding pitfalls, we provide a new
helper function for this.

Co-authored-by: Jonathan Wang <jonathanpwang@users.noreply.github.com>

* fix: minor code quality fixes (#47)

* feat: `fixed_base::msm_par` handles identity point (#48)

We still require fixed base points to be non-identity, but now handle
the case when scalars may be zero or the final MSM value is identity
point.

* chore: add assert for query_cell_at_pos (#50)

* feat: add Github CI running tests (#51)

* fix: ignore code block for doctest (#52)

* feat: add docs and assert with non-empty array checks (#53)

* Release 0.3.0 ecdsa tests (#54)

* More ecdsa tests

* Update mod.rs

* Update tests.rs

* Update ecdsa.rs

* Update ecdsa.rs

* Update ecdsa.rs

* chore: sync with release-0.3.0 and update CI

Co-authored-by: yulliakot <yulliakotel@gmail.com>
Co-authored-by: yuliakot <93175658+yuliakot@users.noreply.github.com>

* chore: fix CI

cannot multi-thread tests involving lookups due to environment variables

* fix: `prop_test_is_less_than_safe` (#58)

This test doesn't run any prover so the input must satisfy range check
assumption. More serious coverage is provided by
`prop_test_neg_is_less_than_safe`

* Add halo2-base readme (#66)

* feat: add halo2-base readme

* fix: readme formatting

* fix: readme edits

* fix: grammer

* fix: use relative links and formatting

* fix: formatting

* feat: add RangeCircuitBuilder description

* feat: rewording and small edits

---------

Co-authored-by: PatStiles <pdstiles78@gmail.com>

* fix: change all `1` to `1u64` to prevent unexpected overflow (#72)

* [Fix] Panic when dealing with identity point (#71)

* More ecdsa tests

* Update mod.rs

* Update tests.rs

* Update ecdsa.rs

* Update ecdsa.rs

* Update ecdsa.rs

* msm tests

* Update mod.rs

* Update msm_sum_infinity.rs

* fix: ec_sub_strict was panicing when output is identity

* affects the MSM functions: right now if the answer is identity, there
  will be a panic due to divide by 0 instead of just returning 0
* there could be a more optimal solution, but due to the traits for
  EccChip, we just generate a random point solely to avoid divide by 0
in the case of identity point

* Fix/fb msm zero (#77)

* fix: fixed_base scalar multiply for [-1]P

* feat: use `multi_scalar_multiply` instead of `scalar_multiply`

* to reduce code maintanence / redundancy

* fix: add back scalar_multiply using any_point

* feat: remove flag from variable base `scalar_multiply`

* feat: add scalar multiply tests for secp256k1

* fix: variable scalar_multiply last select

* Fix/msm tests output identity (#75)

* fixed base msm tests for output infinity

* fixed base msm tests for output infinity

---------

Co-authored-by: yulliakot <yulliakotel@gmail.com>

* feat: add tests and update CI

---------

Co-authored-by: yuliakot <93175658+yuliakot@users.noreply.github.com>
Co-authored-by: yulliakot <yulliakotel@gmail.com>

---------

Co-authored-by: yulliakot <yulliakotel@gmail.com>
Co-authored-by: yuliakot <93175658+yuliakot@users.noreply.github.com>

* [Fix] scalar multiply completeness (#82)

* fix: replace `scalar_multiply` with passthrough to MSM for now

* feat(msm): use strict mode always

* Previously did not use strict because we make assumptions about the
  curve `C`. Since this was not documented and is easy to miss, we use
strict mode always.

* docs: add assumptions to ec_sub_strict (#84)

* fix: readme from previous merge

* chore: cleanup CI for merge into main

* chore: fix readme

---------

Co-authored-by: Jonathan Wang <jonathanpwang@users.noreply.github.com>
Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de>
Co-authored-by: PatStiles <pdstiles78@gmail.com>
Co-authored-by: PatStiles <33334338+PatStiles@users.noreply.github.com>
Co-authored-by: yulliakot <yulliakotel@gmail.com>
Co-authored-by: yuliakot <93175658+yuliakot@users.noreply.github.com>

* chore: fix halo2_proofs_axiom SHA commit

* Merge release v0.3.0 into develop (#90)

* feat: `FpChip::range_check` now works with `max_bits < n * (k-1)` (#91)

* feat(base): range_check 0 bits by asserting is zero

* chore: add range_check 0 bits test

* feat(ecc): `FpChip::range_check` now works with `max_bits < n * (k-1)`

* fix(test): zkevm-keccak test should have `first_pass = SKIP_FIRST_PASS` (#96)

Currently with `first_pass = true`, it skips the first pass, but when
feature "halo2-axiom" is used, there is only one pass of `synthesize` so
the whole thing gets skipped. Mea culpa!

* Feat: test suite (#92)

* feat: stop using env var to pass around FLEX_GATE_CONFIG_PARAMS and

LOOKUP_BITS. Bad for testing (multi-threaded issues). Now we use
thread_local to have a global static for these config params that can be
passed around.

* chore: make utils folder and move some files

* Fix halo2 base tests naming (#76)

* feat: `BaseConfig` to switch between `FlexGateConfig` and `RangeConfig`

- `RangeCircuitBuilder` now uses `BaseConfig` to auto-decide whether to
  create lookup table or not.
    - In the future this should be renamed `BaseCircuitBuilder` or just
      `CircuitBuilder`, but for backwards compatibility we leave the name for now.
- `GateCircuitBuilder` no longer implements `Circuit` because we want to
  switch to having everyone just use `RangeCircuitBuilder`.
- Tests won't compile now because we still need to refactor

* feat: refactored halo2-base tests to use new test suite

* feat: remove use of env var in halo2-ecc

CI now can just run `cargo test`

* feat: remove use of env var from zkevm-keccak

* Add zkevm-keccak test to CI

* chore: fix CI

* chore: add lint to CI

* chore: make Baseconfig fns public

* fix(test): zkevm-keccak test should have `first_pass = SKIP_FIRST_PASS`

Currently with `first_pass = true`, it skips the first pass, but when
feature "halo2-axiom" is used, there is only one pass of `synthesize` so
the whole thing gets skipped. Mea culpa!

---------

Co-authored-by: Xinding Wei <weixinding@gmail.com>

* chore: make `bit_length` const function

* feat: add debugging functions (#99)

* feat: add debugging functions

Functions only available for testing:
* `ctx.debug_assert_false` for debug break point to search for other
  constrain failures in mock prover
* `assigned_value.debug_prank(prank_value)` to prank witness values for
  negative tests

* chore: code pretty

* chore: remove cfg(test) for debug functions

* feat(halo2-base): add `GateChip::pow_var` (#103)

* Use halo2curves v0.4.0 and ff v0.13 (#107)

* wip: change import to ff v0.13

* feat: remove `GateInstructions::get_field_element`

halo2curves now has `bn256-table` which creates table of small field
elements at compile time, so we should just use `F::from` always. This
also improves readability.

* chore: fix syntax and imports after update

* chore: add asm feature

* chore: workspace.resolver = 2

* chore: update ethers-core

* chore: add jemallocator feature to zkevm-keccak crate

* test: add bigger test case to keccak prover

* feat: use `configure_with_params`

remove `thread_local!` usage

* chore: bump zkevm-keccak version to 0.1.1

* feat: add `GateThreadBuilder::from_stage` for convenience

* chore: fixes

* fix: removed `lookup_bits` from `GateThreadBuilder::config`

* fix: debug_assert_false should load witness for debugging

* chore: use unreachable to document that Circuit::configure is never used

* chore: fix comment

* feat(keccak): use configure_with_params

* chore: fix halo2-pse errors

* chore: change halo2_proofs to main

* Add `sub_mul` to GateInstructions (#102)

* Add `sub_mul` to GateInstructions

* Add `sub_mul` prop test

* fix(test): `select_from_idx` wasn't calling the right function (#105)

* chore: add back RangeCircuitBuilder::config (#111)

* fix: `RangeCircuitBuilder::config` remember `lookup_bits`

* [Feat] Add Poseidon Hasher Chip (#110)

* Add Poseidon chip

* chore: minor fixes

* test(poseidon): add compatbility tests

Cherry-picked from #98

Co-authored-by: Antonio Mejías Gil <anmegi.95@gmail.com>

* chore: minor refactor to more closely match snark-verifier

https://github.com/axiom-crypto/snark-verifier/blob/main/snark-verifier/src/util/hash/poseidon.rs

---------

Co-authored-by: Xinding Wei <xinding@intrinsictech.xyz>
Co-authored-by: Jonathan Wang <31040440+jonathanpwang@users.noreply.github.com>
Co-authored-by: Antonio Mejías Gil <anmegi.95@gmail.com>

* feat: add VariableByteArray (#88)

* feat: add VariableByteArray

* fix: correct type in panic msg

* feat: make MAX_VAR_LEN const generic

* feat: add `SafeBool` and `SafeByte` types

These are very common so we have separate wrapper to avoid the extra length 1
vector heap allocation.

* wip: add VarLenBytes

* Refactor VarLenBytes
Add VarLenBytesVec and FixLenBytes
Fix tests

* Add unsafe methods for bytes
Address NITs

---------

Co-authored-by: Jonathan Wang <31040440+jonathanpwang@users.noreply.github.com>
Co-authored-by: Xinding Wei <xinding@intrinsictech.xyz>

* chore: reduce CI real prover load

* Rename PoseidonHasherChip to PoseidonHasher (#113)

Co-authored-by: Xinding Wei <xinding@intrinsictech.xyz>

* chore(safe_types): add conversion `SafeType` to/from `FixLenBytes`

* chore(safe_type): add `unsafe_to_safe_type` unsafe conversion fn

* feat: add `select_array_by_indicator` to `GateInstructions` (#115)

feat(base): add `select_array_by_indicator` to `GateInstructions`

* cleanup: use test-utils for benching (#112)

* cleanup: use test-utils for benching

* feat: add `{gen,check}_proof_with_instances`

* feat: add initial `bench_builder` cmd to `BaseTester`

* fix: cargo fmt

* fix(safe_types): `VarLenBytes` should allow `len == MAX_LEN` (#117)

* [feat] Add Poseidon Chip (#114)

* Add Poseidon hasher

* Fix test/lint

* Fix nits

* Fix lint

* Fix nits & add comments

* Add prover test

* Fix CI

* [chore] Reorg Folder Structure of hashes/zkevm (#118)

* chore: rename crate zkevm-keccak to zkevm-hashes

* fix: add `input_len` back to `KeccakTable`

* chore: move keccak specific constants to `keccak_packed_multi/util`

* Fix test

---------

Co-authored-by: Jonathan Wang <31040440+jonathanpwang@users.noreply.github.com>

* [fix] CI for zkevm hashes (#119)

Fix CI for zkevm hashes

* [chore] Split keccak implementation into multiple files (#120)

Split keccak implementation into multiple files

* feat: keccak constant visibility changes (#121)

feat: constant visibility changes

* [feat] Keccak Raw Output (#122)

* Replace raw keccak output instead of RLCOC

* Fix lint

* Add comments & improve expression performance

* Virtual region managers and dynamic lookup support (#123)

* feat: add `VirtualRegionManager` trait

Create `CopyConstraintManager` to manage global copy constraints.

* wip: separate `SinglePhaseGateManager` and `CopyConstraintManager`

`GateThreadBuilder` was very messy before

Todo:
- Split out lookup functionality

* wip: add `LookupAnyManager`

* wip: `RangeChip` with `LookupAnyManager`

reorg:
- previous `builder/threads` moved to `flex_gate/threads` because
it is all part of `FlexGateConfig` advice assignment logic
- `builder` moved to `range/circuit/builder.rs` as it is part of the
  assignment logic of `RangeCircuitBuilder`

* feat: working `BaseCircuitBuilder` backwards compatible

- `GateThreadBuilder` has become `MultiPhaseCoreManager`
- Some of the functionality has been moved into `BaseCircuitBuilder`,
  which is a generalization of `RangeCircuitBuilder`
- Some fixes on virtual managers because keygen calls `synthesize` twice
  (once for vk, once for pk) so can't drop

* fix: update halo2-ecc and sort `constant_equalities`

Sort `constant_equalities` to ensure deterministism.
Update `halo2-ecc` (mostly the tests) with new circuit builder format.

* fix: `LookupAnyManager` drop check `Arc` strong_count

* feat: add back single column lookup with selector

Special case: if only single advice column that you need to lookup, you
can create a selector and enable lookup on that column.
This means you add 1 selector column, instead of 1 advice column.

Only using this for `RangeConfig` and not generalizing it for now.

* feat: add example of dynamic lookup memory table

* Bump versions to 0.4.0

* chore: re-enable poseidon and safe_types

`Drop` for managers no longer panics because rust `should_panic` test
cannot handle non-unwinding panics.

* chore: remove `row_offset` from `assigned_advices`

This PR was merged: privacy-scaling-explorations/halo2#192

* chore: move `range::circuit` to `gates::circuit`

* nits: address review comments

* feat: add `num_instance_columns` to `BaseCircuitParams`

No longer a const generic

* chore(CI): use larger runner

* Use `raw_assign_{advice,fixed}` in keccak (#125)

* chore: switch `halo2_proofs` branch to `main`

* chore: use `raw_assign_{advice,fixed}` in keccak

* [feat] PoseidonHasher supports multiple inputs in compact format (#127)

* PoseidonHasher supports multiple inputs in compact format

* Add comments

* Remove unnecessary uses

* [feat] Expose Keccack Raw Inputs in Bytes instead of Input RLCs (#124)

* Expose Keccack raw inputs in bytes instead of input RLCs

* Fix column name in comments

* Add comments

* Compress 8 bytes of inputs into a single witness

* chore: add some comments

* Rewrite gates

* Fix comments & typos

* Fix naming

* Add comments

* Selector improvement

* Remove unused

---------

Co-authored-by: Jonathan Wang <31040440+jonathanpwang@users.noreply.github.com>

* Bump `zkevm-hashes` to v0.1.4

* chore: clippy fix

* Generic vertical gate assignment (#129)

* feat: make `single_phase::assign_with_constraints` generic

Use const generic for max rotations accessed by the vertical gate.
This way we can re-use the code for RLC gate.

* chore: make single_phase pub

* feat: add safety check for overlapping gates

* Add `deep_clone` to `BaseCircuitBuilder` (#131)

* chore: add convenience function to `BaseConfig`

* feat: add `deep_clone` to `BaseCircuitBuilder`

We sometimes want to clone `BaseCircuitBuilder` completely (for example
to re-run witness generation). The derived clone only clones the shared
references, instead of the underlying objects.

* fix: `SingleCorePhaseManager` should not create thread in constructor

Because the thread will default to phase 0.

* chore: make `new_context` public

* Convenience functions and fixes for multi-phase (#133)

* feat: add `clear` function to circuit builder and managers

* feat: add `BaseConfig::initialize`

* fix: break points for multiphase

* fix: clear should not change phase

* chore: remove dbg

* chore: add `BaseCircuitBuilder::set_k` fn

* fix: `CopyConstraintManager::clear` was dropping

* feat: impl `From<SafeByte>` for `AssignedValue`

* chore(poseidon): add `derive` statements

* fix(copy_constraints): backend permutation argument depends on order

Backend implementation of `constrain_equal` depends on the order in
which you add equality constraints, so it is not thread-safe...

* feat: add `left_pad` functions for var length arrays (#137)

* chore: use `PrimeField` for `OptimizedPoseidonSpec` (#139)

* chore: add getter functions to Poseidon spec (#140)

* feat: use `(TypeId, usize)` instead of `usize` for lookup tag (#142)

* chore: add `ContextTag` type alias

* feat(base): add `GateInstructions::inner_product_left` function (#143)

* chore: fix comments

* feat(base): add `GateInstructions::inner_product_left` function

* [feat] Keccak Coprocessor Leaf Circuit (#130)

* WIP

* chore: make `KeccakAssignedRow` fields public

* Refactor Keccak coprocessor circuit

* Optimize Keccak circuit MAX_DEGREE

* Fix comments

* Fix bug & typos

* Add testing & refactor folder structure

* Debugging

* Fix bugs

* Fix typo & bug

* feat(test): real prover tests use dummy input for keygen

* chore: make `LoadedKeccakF` public

* Also made `encoded_inputs_from_keccak_fs` public
* Both are useful for external use to make lookup tables in app circuits

* fix(keccak_leaf): review comments and optimization

* chore: use `gate` when `range` not necessary

* Move calculate base ciruit params out & Fix naming/comments

* Make Field-related parameter functions const

* feat: change `is_final` to `SafeBool`

* nit

* Fix typo

---------

Co-authored-by: Jonathan Wang <31040440+jonathanpwang@users.noreply.github.com>

* [feat] App Circuit Utils for Keccak Coprocessor (#141)

* Add keccak coprocessor encoding for VarLenBytesVec/FixLenBytesVec

* Fix naming/nits

* Fix nit

* [chore] Fix fmt (#144)

Fix fmt

* chore: add misc utility functions (#146)

* chore(keccak_leaf): make `generate_circuit_final_outputs` public

* chore: add misc utility functions

* feat(keccak): add `ingestion` module for Rust native input formatting (#147)

* chore(keccak): use `snark-verifier` native Poseidon for encoding (#148)

Currently only used for testing

* feat: optimize leaf poseidon with `hash_compact_chunk_inputs` (#149)

* [chore] cleanup code (#150)

chore: cleanup code

* chore: get halo2-pse working again

* chore: fix fmt

* chore: fix fmt

* [Doc] Keccak Doc (#145)

* Keccak docs

* Fix typos

* Add examples

* Fix comments/docs

* chore: pin snark-verifier branch

* Revert "chore: pin snark-verifier branch"

This reverts commit 1bf36b1.

* [fix] max_rows in BaseCircuitBuilder in Keccak Leaf Circuit (#152)

Fix max_rows in BaseCircuitBuilder inside Keccak Vanilla

* fix: readme after merge

* Fix bls test (#157)

* fix bls test

* rm

* [fix] max_rows in BaseCircuitBuilder in Keccak Leaf Circuit (#152)

Fix max_rows in BaseCircuitBuilder inside Keccak Vanilla

* [chore] Remove Unnecessary Lookup in Keccak Coprocessor Leaf Circuit (#153)

* chore: fix fmt

* [fix] max_rows in BaseCircuitBuilder in Keccak Leaf Circuit (#152)

Fix max_rows in BaseCircuitBuilder inside Keccak Vanilla

* Remove lookup in Keccak Leaf circuit

* Revert "chore: pin snark-verifier branch"

This reverts commit 1bf36b1.

* [rename] (coprocessor, leaf) -> (component, shard) (#161)

* Fix schnorr test (#158)

* fix schnorr test

* fix infinite loop due to rnd not change, fmt

* fix rng clone err

* [fix] Multiple Phase Lookup (#162)

Fix multiple phase lookup

* chore: clippy fix

---------

Co-authored-by: yulliakot <yulliakotel@gmail.com>
Co-authored-by: yuliakot <93175658+yuliakot@users.noreply.github.com>
Co-authored-by: Xinding Wei <weixinding@gmail.com>
Co-authored-by: PatStiles <33334338+PatStiles@users.noreply.github.com>
Co-authored-by: Jonathan Wang <jonathanpwang@users.noreply.github.com>
Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de>
Co-authored-by: PatStiles <pdstiles78@gmail.com>
Co-authored-by: mmagician <marcin.gorny.94@protonmail.com>
Co-authored-by: Xinding Wei <xinding@intrinsictech.xyz>
Co-authored-by: Antonio Mejías Gil <anmegi.95@gmail.com>
Co-authored-by: MonkeyKing-1 <67293785+MonkeyKing-1@users.noreply.github.com>
Co-authored-by: doxxed account - may discard soon, i love u since there's some nice work i've done with u but unfortunately i may have to switch again <104515688+justcode740@users.noreply.github.com>
jonathanpwang added a commit that referenced this pull request Nov 22, 2023
* fix: change all `1` to `1u64` to prevent unexpected overflow (#72)

* [Fix] Panic when dealing with identity point (#71)

* More ecdsa tests

* Update mod.rs

* Update tests.rs

* Update ecdsa.rs

* Update ecdsa.rs

* Update ecdsa.rs

* msm tests

* Update mod.rs

* Update msm_sum_infinity.rs

* fix: ec_sub_strict was panicing when output is identity

* affects the MSM functions: right now if the answer is identity, there
  will be a panic due to divide by 0 instead of just returning 0
* there could be a more optimal solution, but due to the traits for
  EccChip, we just generate a random point solely to avoid divide by 0
in the case of identity point

* Fix/fb msm zero (#77)

* fix: fixed_base scalar multiply for [-1]P

* feat: use `multi_scalar_multiply` instead of `scalar_multiply`

* to reduce code maintanence / redundancy

* fix: add back scalar_multiply using any_point

* feat: remove flag from variable base `scalar_multiply`

* feat: add scalar multiply tests for secp256k1

* fix: variable scalar_multiply last select

* Fix/msm tests output identity (#75)

* fixed base msm tests for output infinity

* fixed base msm tests for output infinity

---------

Co-authored-by: yulliakot <yulliakotel@gmail.com>

* feat: add tests and update CI

---------

Co-authored-by: yuliakot <93175658+yuliakot@users.noreply.github.com>
Co-authored-by: yulliakot <yulliakotel@gmail.com>

---------

Co-authored-by: yulliakot <yulliakotel@gmail.com>
Co-authored-by: yuliakot <93175658+yuliakot@users.noreply.github.com>

* fix: redundant check in `ec_sub_unequal`

* Add SafeType (#26)

* Add SafeType

* Refactor & add testing

* Add doc comment

* Refactor SafeChip

* Move gen_proof/check_proof to utils

* Fix merge issues

* feat(CI): switch to larger runner

* fix(builder): handle empty ctx with only equality constraints

* feat: add SafeAddress and SafeUint160 (#85)

* feat: add SafeAddress and SafeUint160

* fix incorrect byte size

* Release 0.3.0 (#86)

* feat: upgrade overall `halo2-base` API to support future multi-threaded
assignments using our basic gate

* WIP: currently `gates::flex_gate` is updated and passes basic test

* BUG: `GateInstructions::idx_to_indicator` was missing a constraint to
check that the indicator witness was equal to 1 when non-zero.
* Previously the constraint ensured that `ind[i] = 0` when `idx != i`
  however `ind[idx]` could be anything!!!

* update: working benches for `mul` and `inner_product`

* feat: add `test_multithread_gates`

* BUG: `get_last_bit` did not do an `assert_bit` check on the answer

* this function was not used anywhere

* fix: `builder::assign_*` was not handling cases where two gates overlap
and there is a break point in that overlap
* we need to copy a cell between columns to fix

* feat: update `gates::range` to working tests and new API

* In keygen mode, the `CircuitBuilder` will clone the `ThreadBuilder`
  instead of `take`ing it because the same circuit is used for both vk
gen and pk gen. This could lead to more memory usage for pk gen.

* fix: change `AssignedValue` type to `KeccakAssignedValue` for
compatibility after halo2-base update

* Initial version 0.3.0 of halo2-ecc (#12)

* add multi-thread witness assignment support for `variable_base_msm`
  and `fixed_base_msm`
* batch size 100 MSM witness generation went from 500ms -> 100ms

* Sync with updates in `halo2_proofs_axiom`
* `assign_advice` not longer returns `Result` so no more `unwrap`

* Fix: assert uses of size hint in release mode (#13)

* remove `size_hint` in `inner_product_simple`

* change other uses of `size_hint` to follow with `assert_eq!` instead
  of `debug_assert_eq!`

* Fix: bit decomposition edge cases (#14)

* fix: change `debug_assert` in `decompose_u64_digits_limbs` to restrict
`bit_len < 64` and `decompose_biguint` to `64 <= bit_len < 128`
* add more comprehensive tests for above two functions

* Initial checkpoint for halo2-ecc v0.3.0 (#15)

* chore: clippy --fix

* Feat/add readme (#4)

* feat: add README

* feat: re-enable `secp256k1` module with updated tests

* chore: fix result println

* chore: update Cargo halo2_proofs_axiom to axiom/dev branch

* compatibility update with `halo2_proofs_axiom`

Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de>

* Fix: make `GateThreadBuilder` compatible with external usage (#16)

* chore: expose gate_builder.unknown

* feat: `GateThreadBuilder::assign_all` takes assigned_{advices,constants}
    as input instead of new hashmap, in case we want to constrain equalities
    for cells not belonging to this builder

* chore: update halo2-pse tag

* fix: `GateThreadBuilder::assign_all` now returns `HashMap`s of
    assigned cells for external equality constraints (e.g., instance cells,
    `AssignedCells` from chips not using halo2-lib).

* fix: `assign_all` was not assigning constants as desired: it was
    assigning a new constant per context. This leads to confusion and
    possible undesired consequences down the line.

* Fix: under-constrained `idx_to_indicator` (#17)

*fix(BUG): `GateChip::idx_to_indicator` still had soundness bug where at index
`idx` the value could be 0 or 1 (instead of only 1)

* feat: add some function documentation

* test(idx_to_indicator): add comprehensive tests
* both positive and negative tests

* Fix: soundness error in `FpChip::assert_eq` due to typo (#18)

* chore: update halo2-ecc version to 0.3.0

* fix(BUG): `FpChip::assert_equal` had `a` instead of `b` typo

* feat: add tests for `FpChip::assert_eq`
* positive and negative tests

* Remove redundant code and prevent race conditions (#19)

* feat: move `GateCircuitBuilder::synthesize` to `sub_synthesize` function
which also returns the assigned advices.

* reduces code duplication between `GateCircuitBuilder::synthesize` and
  `RangeCircuitBuilder::synthesize` and also makes it easier to assign
public instances elsewhere (e.g., snark-verifier)

* feat: remove `Mutex` to prevent non-deterministism

* In variable and fixed base `msm_par` functions, remove use of
  `Mutex<GateThreadBuilder>` because even the `Mutex` is not thread-
  safe in the sense that: if you let `Mutex` decide order
  that `GateThreadBuilder` is unlocked, you may still add Contexts to
  the builder in a non-deterministic order.

* fix: `fixed_base::msm_par` loading new zeros

* In `msm_par` each parallelized context was loading a new zero via
  `ctx.load_zero()`
* This led to using more cells than the non-parallelized version
* In `fixed_base_msm_in`, the if statement depending on
  `rayon::current_number_threads` leads to inconsistent proving keys
between different machines. This has been removed and now uses a fixed
number `25`.

* chore: use `info!` instead of `println` for params

* Allow `assign_all` also if `witness_gen_only = true`

* Fix: `inner_product_left_last` size hint (#25)

* Add documentation for halo2-base (#27)

* adds draft documentation for range.rs

* draft docs for lib.rs, utiils.rs, builder.rs

* fix: add suggested doc edits for range.rs

* docs: add draft documentation for flex_gate.rs

* fix: range.rs doc capitalization error

* fix: suggested edits for utils.rs docs

* fix: resolve comments for range.rs docs

* fix: resolve comments on flex_gate.rs docs

* fix: resolve comments for lib.rs, util.rs docs

* fix: resolve comments for builder.rs docs

* chore: use `info!` instead of `println` for params

* Allow `assign_all` also if `witness_gen_only = true`

* Fix: `inner_product_left_last` size hint (#25)

* docs: minor fixes

---------

Co-authored-by: PatStiles <pdstiles78@gmail.com>

* Smart Range Builder (#29)

* feat: smart `RangeCircuitBuilder`

Allow `RangeCircuitBuilder` to not create lookup table if it detects
that there's nothing to look up.

* feat: add `RangeWithInstanceCircuitBuilder`

* Moved from `snark-verifier-sdk`
* Also made this circuit builder smart so it doesn't load lookup table
  if not necessary
    * In particular this can also be used as a
      `GateWithInstanceCircuitBuilder`

* chore: derive Eq for CircuitBuilderStage

* fix: RangeConfig should not unwrap LOOKUP_BITS

* fix: `div_mod_var` when `a_num_bits <= b_num_bits` (#31)

* Feat: extend halo2 base test coverage (#35)

* feat: add flex_gate_test.rs and pos add() test

* feat: add pos sub() test

* feat: add pos neg() test

* feat: add pos mul() test

* feat: add pos mul_add() test

* feat: add pos mul_not() test

* feat: add pos assert_bit

* feat: add pos div_unsafe() test

* feat: add pos assert_is_const test

* feat: add pos inner_product() test

* feat: add pos inner_product_left_last() test

* feat: add pos inner_product_with_sums test

* feat: add pos sum_products_with_coeff_and_var test

* feat: add pos and() test

* feat: add pos not() test

* feat: add pos select() test

* feat: add pos or_and() test

* feat: add pos bits_to_indicator() test

* feat: add pos idx_to_indicator() test

* feat: add pos select_by_indicator() test

* feat: add pos select_from_idx() test

* feat: add pos is_zero() test

* feat: add pos is_equal() test

* feat: add pos num_to_bits() test

* feat: add pos lagrange_eval() test

* feat: add pos get_field_element() test

* feat: add pos range_check() tests

* feat: add pos check_less_than() test

* feat: add pos check_less_than_safe() test

* feat: add pos check_big_less_than_safe() test

* feat: add pos is_less_than() test

* feat: add pos is_less_than_safe() test

* feat: add pos is_big_less_than_safe() test

* feat: add pos div_mod() test

* feat: add pos get_last_bit() test

* feat: add pos div_mod_var() test

* fix: pass slices into test functions not arrays

* feat: Add pos property tests for flex_gate

* feat: Add positive property tests for flex_gate

* feat: add pos property tests for range_check.rs

* feat: add neg pranking test for idx_to_indicator

* fix: change div_mod_var test values

* feat(refactor): refactor property tests

* fix: fix neg test, assert_const, assert_bit

* fix: failing prop tests

* feat: expand negative testing is_less_than_failing

* fix: Circuit overflow errors on neg tests

* fix: prop_test_mul_not

* fix: everything but get_last_bit & lagrange

* fix: clippy

* fix: set LOOKUP_BITS in range tests, make range check neg test more

robust

* fix: neg_prop_tests cannot prank inputs

Inputs have many copy constraints; pranking initial input will cause all
copy constraints to fail

* fix: test_is_big_less_than_safe, 240 bits max

* Didn't want to change current `is_less_than` implementation, which in
  order to optimize lookups for smaller bits, only works when inputs
have at most `(F::CAPACITY // lookup_bits - 1) * lookup_bits` bits

* fix: inline doc for lagrange_and_eval

* Remove proptest for lagrange_and_eval and leave as todo

* tests: add readme about serial execution

---------

Co-authored-by: Jonathan Wang <jonathanpwang@users.noreply.github.com>

* fix(ecdsa): allow u1*G == u2*PK case (#36)

NOTE: current ecdsa requires `r, s` to be given as proper CRT integers

TODO: newtypes to guard this assumption

* fix: `log2_ceil(0)` should return `0` (#37)

* Guard `ScalarField` byte representations to always be little-endian (#38)

fix: guard `ScalarField` to be little-endian

* fix: get_last_bit two errors (#39)

2 embarassing errors:
* Witness gen for last bit was wrong (used xor instead of &)
* `ctx.get` was called after `range_check` so it was getting the wrong
  cell

* Add documentation for all debug_asserts (#40)

feat: add documentation for all debug_asserts

* fix: `FieldChip::divide` renamed `divide_unsafe` (#41)

Add `divide` that checks denomintor is nonzero.
Add documentation in cases where `divide_unsafe` is used.

* Use new types to validate input assumptions (#43)

* feat: add new types `ProperUint` and `ProperCrtUint`

To guard around assumptions about big integer representations

* fix: remove unused `FixedAssignedCRTInteger`

* feat: use new types for bigint and field chips

New types now guard for different assumptions on non-native bigint
arithmetic. Distinguish between:
- Overflow CRT integers
- Proper BigUint with native part derived from limbs
- Field elements where inequality < modulus is checked

Also add type to help guard for inequality check in
ec_add_unequal_strict

Rust traits did not play so nicely with references, so I had to switch
many functions to move inputs instead of borrow by reference. However to
avoid writing `clone` everywhere, we allow conversion `From` reference
to the new type via cloning.

* feat: use `ProperUint` for `big_less_than`

* feat(ecc): add fns for assign private witness points

that constrain point to lie on curve

* fix: unnecessary lifetimes

* chore: remove clones

* Better handling of EC point at infinity (#44)

* feat: allow `msm_par` to return identity point

* feat: handle point at infinity

`multi_scalar_multiply` and `multi_exp_par` now handle point at infinity
completely

Add docs for `ec_add_unequal, ec_sub_unequal, ec_double_and_add_unequal`
to specify point at infinity leads to undefined behavior

* feat: use strict ec ops more often (#45)

* `msm` implementations now always use `ec_{add,sub}_unequal` in strict
mode for safety
* Add docs to `scalar_multiply` and a flag to specify when it's safe to
  turn off some strict assumptions

* feat: add `parallelize_in` helper function (#46)

Multi-threading of witness generation is tricky because one has to
ensure the circuit column assignment order stays deterministic. To
ensure good developer experience / avoiding pitfalls, we provide a new
helper function for this.

Co-authored-by: Jonathan Wang <jonathanpwang@users.noreply.github.com>

* fix: minor code quality fixes (#47)

* feat: `fixed_base::msm_par` handles identity point (#48)

We still require fixed base points to be non-identity, but now handle
the case when scalars may be zero or the final MSM value is identity
point.

* chore: add assert for query_cell_at_pos (#50)

* feat: add Github CI running tests (#51)

* fix: ignore code block for doctest (#52)

* feat: add docs and assert with non-empty array checks (#53)

* Release 0.3.0 ecdsa tests (#54)

* More ecdsa tests

* Update mod.rs

* Update tests.rs

* Update ecdsa.rs

* Update ecdsa.rs

* Update ecdsa.rs

* chore: sync with release-0.3.0 and update CI

Co-authored-by: yulliakot <yulliakotel@gmail.com>
Co-authored-by: yuliakot <93175658+yuliakot@users.noreply.github.com>

* chore: fix CI

cannot multi-thread tests involving lookups due to environment variables

* fix: `prop_test_is_less_than_safe` (#58)

This test doesn't run any prover so the input must satisfy range check
assumption. More serious coverage is provided by
`prop_test_neg_is_less_than_safe`

* Add halo2-base readme (#66)

* feat: add halo2-base readme

* fix: readme formatting

* fix: readme edits

* fix: grammer

* fix: use relative links and formatting

* fix: formatting

* feat: add RangeCircuitBuilder description

* feat: rewording and small edits

---------

Co-authored-by: PatStiles <pdstiles78@gmail.com>

* fix: change all `1` to `1u64` to prevent unexpected overflow (#72)

* [Fix] Panic when dealing with identity point (#71)

* More ecdsa tests

* Update mod.rs

* Update tests.rs

* Update ecdsa.rs

* Update ecdsa.rs

* Update ecdsa.rs

* msm tests

* Update mod.rs

* Update msm_sum_infinity.rs

* fix: ec_sub_strict was panicing when output is identity

* affects the MSM functions: right now if the answer is identity, there
  will be a panic due to divide by 0 instead of just returning 0
* there could be a more optimal solution, but due to the traits for
  EccChip, we just generate a random point solely to avoid divide by 0
in the case of identity point

* Fix/fb msm zero (#77)

* fix: fixed_base scalar multiply for [-1]P

* feat: use `multi_scalar_multiply` instead of `scalar_multiply`

* to reduce code maintanence / redundancy

* fix: add back scalar_multiply using any_point

* feat: remove flag from variable base `scalar_multiply`

* feat: add scalar multiply tests for secp256k1

* fix: variable scalar_multiply last select

* Fix/msm tests output identity (#75)

* fixed base msm tests for output infinity

* fixed base msm tests for output infinity

---------

Co-authored-by: yulliakot <yulliakotel@gmail.com>

* feat: add tests and update CI

---------

Co-authored-by: yuliakot <93175658+yuliakot@users.noreply.github.com>
Co-authored-by: yulliakot <yulliakotel@gmail.com>

---------

Co-authored-by: yulliakot <yulliakotel@gmail.com>
Co-authored-by: yuliakot <93175658+yuliakot@users.noreply.github.com>

* [Fix] scalar multiply completeness (#82)

* fix: replace `scalar_multiply` with passthrough to MSM for now

* feat(msm): use strict mode always

* Previously did not use strict because we make assumptions about the
  curve `C`. Since this was not documented and is easy to miss, we use
strict mode always.

* docs: add assumptions to ec_sub_strict (#84)

* fix: readme from previous merge

* chore: cleanup CI for merge into main

* chore: fix readme

---------

Co-authored-by: Jonathan Wang <jonathanpwang@users.noreply.github.com>
Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de>
Co-authored-by: PatStiles <pdstiles78@gmail.com>
Co-authored-by: PatStiles <33334338+PatStiles@users.noreply.github.com>
Co-authored-by: yulliakot <yulliakotel@gmail.com>
Co-authored-by: yuliakot <93175658+yuliakot@users.noreply.github.com>

* chore: fix halo2_proofs_axiom SHA commit

* Merge release v0.3.0 into develop (#90)

* feat: `FpChip::range_check` now works with `max_bits < n * (k-1)` (#91)

* feat(base): range_check 0 bits by asserting is zero

* chore: add range_check 0 bits test

* feat(ecc): `FpChip::range_check` now works with `max_bits < n * (k-1)`

* fix(test): zkevm-keccak test should have `first_pass = SKIP_FIRST_PASS` (#96)

Currently with `first_pass = true`, it skips the first pass, but when
feature "halo2-axiom" is used, there is only one pass of `synthesize` so
the whole thing gets skipped. Mea culpa!

* Feat: test suite (#92)

* feat: stop using env var to pass around FLEX_GATE_CONFIG_PARAMS and

LOOKUP_BITS. Bad for testing (multi-threaded issues). Now we use
thread_local to have a global static for these config params that can be
passed around.

* chore: make utils folder and move some files

* Fix halo2 base tests naming (#76)

* feat: `BaseConfig` to switch between `FlexGateConfig` and `RangeConfig`

- `RangeCircuitBuilder` now uses `BaseConfig` to auto-decide whether to
  create lookup table or not.
    - In the future this should be renamed `BaseCircuitBuilder` or just
      `CircuitBuilder`, but for backwards compatibility we leave the name for now.
- `GateCircuitBuilder` no longer implements `Circuit` because we want to
  switch to having everyone just use `RangeCircuitBuilder`.
- Tests won't compile now because we still need to refactor

* feat: refactored halo2-base tests to use new test suite

* feat: remove use of env var in halo2-ecc

CI now can just run `cargo test`

* feat: remove use of env var from zkevm-keccak

* Add zkevm-keccak test to CI

* chore: fix CI

* chore: add lint to CI

* chore: make Baseconfig fns public

* fix(test): zkevm-keccak test should have `first_pass = SKIP_FIRST_PASS`

Currently with `first_pass = true`, it skips the first pass, but when
feature "halo2-axiom" is used, there is only one pass of `synthesize` so
the whole thing gets skipped. Mea culpa!

---------

Co-authored-by: Xinding Wei <weixinding@gmail.com>

* chore: make `bit_length` const function

* feat: add debugging functions (#99)

* feat: add debugging functions

Functions only available for testing:
* `ctx.debug_assert_false` for debug break point to search for other
  constrain failures in mock prover
* `assigned_value.debug_prank(prank_value)` to prank witness values for
  negative tests

* chore: code pretty

* chore: remove cfg(test) for debug functions

* feat(halo2-base): add `GateChip::pow_var` (#103)

* Use halo2curves v0.4.0 and ff v0.13 (#107)

* wip: change import to ff v0.13

* feat: remove `GateInstructions::get_field_element`

halo2curves now has `bn256-table` which creates table of small field
elements at compile time, so we should just use `F::from` always. This
also improves readability.

* chore: fix syntax and imports after update

* chore: add asm feature

* chore: workspace.resolver = 2

* chore: update ethers-core

* chore: add jemallocator feature to zkevm-keccak crate

* test: add bigger test case to keccak prover

* feat: use `configure_with_params`

remove `thread_local!` usage

* chore: bump zkevm-keccak version to 0.1.1

* feat: add `GateThreadBuilder::from_stage` for convenience

* chore: fixes

* fix: removed `lookup_bits` from `GateThreadBuilder::config`

* fix: debug_assert_false should load witness for debugging

* chore: use unreachable to document that Circuit::configure is never used

* chore: fix comment

* feat(keccak): use configure_with_params

* chore: fix halo2-pse errors

* chore: change halo2_proofs to main

* Add `sub_mul` to GateInstructions (#102)

* Add `sub_mul` to GateInstructions

* Add `sub_mul` prop test

* fix(test): `select_from_idx` wasn't calling the right function (#105)

* chore: add back RangeCircuitBuilder::config (#111)

* fix: `RangeCircuitBuilder::config` remember `lookup_bits`

* [Feat] Add Poseidon Hasher Chip (#110)

* Add Poseidon chip

* chore: minor fixes

* test(poseidon): add compatbility tests

Cherry-picked from #98

Co-authored-by: Antonio Mejías Gil <anmegi.95@gmail.com>

* chore: minor refactor to more closely match snark-verifier

https://github.com/axiom-crypto/snark-verifier/blob/main/snark-verifier/src/util/hash/poseidon.rs

---------

Co-authored-by: Xinding Wei <xinding@intrinsictech.xyz>
Co-authored-by: Jonathan Wang <31040440+jonathanpwang@users.noreply.github.com>
Co-authored-by: Antonio Mejías Gil <anmegi.95@gmail.com>

* feat: add VariableByteArray (#88)

* feat: add VariableByteArray

* fix: correct type in panic msg

* feat: make MAX_VAR_LEN const generic

* feat: add `SafeBool` and `SafeByte` types

These are very common so we have separate wrapper to avoid the extra length 1
vector heap allocation.

* wip: add VarLenBytes

* Refactor VarLenBytes
Add VarLenBytesVec and FixLenBytes
Fix tests

* Add unsafe methods for bytes
Address NITs

---------

Co-authored-by: Jonathan Wang <31040440+jonathanpwang@users.noreply.github.com>
Co-authored-by: Xinding Wei <xinding@intrinsictech.xyz>

* chore: reduce CI real prover load

* Rename PoseidonHasherChip to PoseidonHasher (#113)

Co-authored-by: Xinding Wei <xinding@intrinsictech.xyz>

* chore(safe_types): add conversion `SafeType` to/from `FixLenBytes`

* chore(safe_type): add `unsafe_to_safe_type` unsafe conversion fn

* feat: add `select_array_by_indicator` to `GateInstructions` (#115)

feat(base): add `select_array_by_indicator` to `GateInstructions`

* cleanup: use test-utils for benching (#112)

* cleanup: use test-utils for benching

* feat: add `{gen,check}_proof_with_instances`

* feat: add initial `bench_builder` cmd to `BaseTester`

* fix: cargo fmt

* fix(safe_types): `VarLenBytes` should allow `len == MAX_LEN` (#117)

* [feat] Add Poseidon Chip (#114)

* Add Poseidon hasher

* Fix test/lint

* Fix nits

* Fix lint

* Fix nits & add comments

* Add prover test

* Fix CI

* [chore] Reorg Folder Structure of hashes/zkevm (#118)

* chore: rename crate zkevm-keccak to zkevm-hashes

* fix: add `input_len` back to `KeccakTable`

* chore: move keccak specific constants to `keccak_packed_multi/util`

* Fix test

---------

Co-authored-by: Jonathan Wang <31040440+jonathanpwang@users.noreply.github.com>

* [fix] CI for zkevm hashes (#119)

Fix CI for zkevm hashes

* [chore] Split keccak implementation into multiple files (#120)

Split keccak implementation into multiple files

* feat: keccak constant visibility changes (#121)

feat: constant visibility changes

* [feat] Keccak Raw Output (#122)

* Replace raw keccak output instead of RLCOC

* Fix lint

* Add comments & improve expression performance

* Virtual region managers and dynamic lookup support (#123)

* feat: add `VirtualRegionManager` trait

Create `CopyConstraintManager` to manage global copy constraints.

* wip: separate `SinglePhaseGateManager` and `CopyConstraintManager`

`GateThreadBuilder` was very messy before

Todo:
- Split out lookup functionality

* wip: add `LookupAnyManager`

* wip: `RangeChip` with `LookupAnyManager`

reorg:
- previous `builder/threads` moved to `flex_gate/threads` because
it is all part of `FlexGateConfig` advice assignment logic
- `builder` moved to `range/circuit/builder.rs` as it is part of the
  assignment logic of `RangeCircuitBuilder`

* feat: working `BaseCircuitBuilder` backwards compatible

- `GateThreadBuilder` has become `MultiPhaseCoreManager`
- Some of the functionality has been moved into `BaseCircuitBuilder`,
  which is a generalization of `RangeCircuitBuilder`
- Some fixes on virtual managers because keygen calls `synthesize` twice
  (once for vk, once for pk) so can't drop

* fix: update halo2-ecc and sort `constant_equalities`

Sort `constant_equalities` to ensure deterministism.
Update `halo2-ecc` (mostly the tests) with new circuit builder format.

* fix: `LookupAnyManager` drop check `Arc` strong_count

* feat: add back single column lookup with selector

Special case: if only single advice column that you need to lookup, you
can create a selector and enable lookup on that column.
This means you add 1 selector column, instead of 1 advice column.

Only using this for `RangeConfig` and not generalizing it for now.

* feat: add example of dynamic lookup memory table

* Bump versions to 0.4.0

* chore: re-enable poseidon and safe_types

`Drop` for managers no longer panics because rust `should_panic` test
cannot handle non-unwinding panics.

* chore: remove `row_offset` from `assigned_advices`

This PR was merged: privacy-scaling-explorations/halo2#192

* chore: move `range::circuit` to `gates::circuit`

* nits: address review comments

* feat: add `num_instance_columns` to `BaseCircuitParams`

No longer a const generic

* chore(CI): use larger runner

* Use `raw_assign_{advice,fixed}` in keccak (#125)

* chore: switch `halo2_proofs` branch to `main`

* chore: use `raw_assign_{advice,fixed}` in keccak

* [feat] PoseidonHasher supports multiple inputs in compact format (#127)

* PoseidonHasher supports multiple inputs in compact format

* Add comments

* Remove unnecessary uses

* [feat] Expose Keccack Raw Inputs in Bytes instead of Input RLCs (#124)

* Expose Keccack raw inputs in bytes instead of input RLCs

* Fix column name in comments

* Add comments

* Compress 8 bytes of inputs into a single witness

* chore: add some comments

* Rewrite gates

* Fix comments & typos

* Fix naming

* Add comments

* Selector improvement

* Remove unused

---------

Co-authored-by: Jonathan Wang <31040440+jonathanpwang@users.noreply.github.com>

* Bump `zkevm-hashes` to v0.1.4

* chore: clippy fix

* Generic vertical gate assignment (#129)

* feat: make `single_phase::assign_with_constraints` generic

Use const generic for max rotations accessed by the vertical gate.
This way we can re-use the code for RLC gate.

* chore: make single_phase pub

* feat: add safety check for overlapping gates

* Add `deep_clone` to `BaseCircuitBuilder` (#131)

* chore: add convenience function to `BaseConfig`

* feat: add `deep_clone` to `BaseCircuitBuilder`

We sometimes want to clone `BaseCircuitBuilder` completely (for example
to re-run witness generation). The derived clone only clones the shared
references, instead of the underlying objects.

* fix: `SingleCorePhaseManager` should not create thread in constructor

Because the thread will default to phase 0.

* chore: make `new_context` public

* Convenience functions and fixes for multi-phase (#133)

* feat: add `clear` function to circuit builder and managers

* feat: add `BaseConfig::initialize`

* fix: break points for multiphase

* fix: clear should not change phase

* chore: remove dbg

* chore: add `BaseCircuitBuilder::set_k` fn

* fix: `CopyConstraintManager::clear` was dropping

* feat: impl `From<SafeByte>` for `AssignedValue`

* chore(poseidon): add `derive` statements

* fix(copy_constraints): backend permutation argument depends on order

Backend implementation of `constrain_equal` depends on the order in
which you add equality constraints, so it is not thread-safe...

* feat: add `left_pad` functions for var length arrays (#137)

* chore: use `PrimeField` for `OptimizedPoseidonSpec` (#139)

* chore: add getter functions to Poseidon spec (#140)

* feat: use `(TypeId, usize)` instead of `usize` for lookup tag (#142)

* chore: add `ContextTag` type alias

* feat(base): add `GateInstructions::inner_product_left` function (#143)

* chore: fix comments

* feat(base): add `GateInstructions::inner_product_left` function

* [feat] Keccak Coprocessor Leaf Circuit (#130)

* WIP

* chore: make `KeccakAssignedRow` fields public

* Refactor Keccak coprocessor circuit

* Optimize Keccak circuit MAX_DEGREE

* Fix comments

* Fix bug & typos

* Add testing & refactor folder structure

* Debugging

* Fix bugs

* Fix typo & bug

* feat(test): real prover tests use dummy input for keygen

* chore: make `LoadedKeccakF` public

* Also made `encoded_inputs_from_keccak_fs` public
* Both are useful for external use to make lookup tables in app circuits

* fix(keccak_leaf): review comments and optimization

* chore: use `gate` when `range` not necessary

* Move calculate base ciruit params out & Fix naming/comments

* Make Field-related parameter functions const

* feat: change `is_final` to `SafeBool`

* nit

* Fix typo

---------

Co-authored-by: Jonathan Wang <31040440+jonathanpwang@users.noreply.github.com>

* [feat] App Circuit Utils for Keccak Coprocessor (#141)

* Add keccak coprocessor encoding for VarLenBytesVec/FixLenBytesVec

* Fix naming/nits

* Fix nit

* [chore] Fix fmt (#144)

Fix fmt

* chore: add misc utility functions (#146)

* chore(keccak_leaf): make `generate_circuit_final_outputs` public

* chore: add misc utility functions

* feat(keccak): add `ingestion` module for Rust native input formatting (#147)

* chore(keccak): use `snark-verifier` native Poseidon for encoding (#148)

Currently only used for testing

* feat: optimize leaf poseidon with `hash_compact_chunk_inputs` (#149)

* [chore] cleanup code (#150)

chore: cleanup code

* chore: get halo2-pse working again

* chore: fix fmt

* [Doc] Keccak Doc (#145)

* Keccak docs

* Fix typos

* Add examples

* Fix comments/docs

* chore: pin snark-verifier branch

* [fix] max_rows in BaseCircuitBuilder in Keccak Leaf Circuit (#152)

Fix max_rows in BaseCircuitBuilder inside Keccak Vanilla

* [chore] Remove Unnecessary Lookup in Keccak Coprocessor Leaf Circuit (#153)

* chore: fix fmt

* [fix] max_rows in BaseCircuitBuilder in Keccak Leaf Circuit (#152)

Fix max_rows in BaseCircuitBuilder inside Keccak Vanilla

* Remove lookup in Keccak Leaf circuit

* Revert "chore: pin snark-verifier branch"

This reverts commit 1bf36b1.

* [rename] (coprocessor, leaf) -> (component, shard) (#161)

* [rename] (coprocessor, leaf) -> (component, shard) (#161)

* [fix] Multiple Phase Lookup (#162)

Fix multiple phase lookup

* [fix] Multiple Phase Lookup (#162)

Fix multiple phase lookup

* [chore] add conversion `SafePrimitive` to `QuantumCell::Existing` (#169)

chore: add conversion `SafePrimitive` to `QuantumCell::Existing`

* fix: bad import on halo2-pse

* [chore] add conversion `SafePrimitive` to `QuantumCell::Existing` (#169)

chore: add conversion `SafePrimitive` to `QuantumCell::Existing`

* fix: bad import on halo2-pse

* chore: make `{Fixed,Var}LenBytes*` constructor public

It's convenient to be able to construct the structs from vectors of safe
bytes externally. Only unsafe-ness is `len <= max_len` is not checked.

* chore: make `{Fixed,Var}LenBytes*` constructor public

It's convenient to be able to construct the structs from vectors of safe
bytes externally. Only unsafe-ness is `len <= max_len` is not checked.

* chore(keccak): `format_requests` always returns true capacity (#171)

* [chore] derive `Hash` for `BaseCircuitParams` (#172)

chore: derive `Hash` for `BaseCircuitParams`

* [chore] impl `AsRef, AsMut` for `BaseCircuitBuilder` to self (#173)

chore: impl `AsRef, AsMut` for `BaseCircuitBuilder` to self

* [chore] impl `AsRef, AsMut` for `BaseConfig` to self (#174)

chore: impl `AsRef, AsMut` for `BaseConfig` to self

* [chore] Add getters to `KeccakComponentShardCircuit` (#178)

* chore: add getters to `KeccakComponentShardCircuit`

For example, it's useful to access `BaseCircuitBuilder` to read public instances.

* chore: `inputs` getter for `KeccakComponentShardCircuit`

* feat: remove getter for `BaseCircuitBuilder`

`BaseCircuitBuilder` is built during `synthesize` after raw vanilla
circuit synthesis, so it should not be accessed externally.

* [chore] Expose Keccak Packing (#180)

Expose Keccak packing

* [chore] Expose Keccak Format (#181)

Expose Keccak format_input

* [feat] basic dynamic lookup table gadget (#182)

* feat: basic dynamic lookup table gadget

* chore: fix imports

* [chore] expose `spec` in `PoseidonHasher` (#183)

chore: expose `spec` in `PoseidonHasher`

* [chore] fix halo2-pse build error (#184)

chore: fix halo2-pse build error

* [feat] expose keccak table loading/packing functions for external crate usage (#195)

* feat: expose `load_keccak_assigned_rows` for external crates to use

* feat: split `encode_inputs_from_keccak_fs` into

`pack_inputs_from_keccak_fs` and poseidon hashing part.
The packing part can be used separately from the Poseidon-specific part.

* chore: rename function

* [chore] add getters to `PoseidonCompactChunkInput` (#196)

chore: add getters to `PoseidonCompactChunkInput`

* [chore] fix deref after using CopyGetters (#197)

* chore: fix deref from get_copy

* chore: add missing docs from getters

* [feat] implement `CircuitExt` for `KeccakComponentShardCircuit` (#198)

* chore: import `snark-verifier-sdk`

* feat: implement `CircuitExt` for `KeccakComponentShardCircuit`

so it can be aggregated by `snark-verifier-sdk`

* chore: derive `Serialize` for keccak circuit params

* chore: fix `snark-verifier-sdk` version

* [chore] add `cargo audit` to CI (#207)

chore: add `cargo audit` to CI

Upgrade criterion version

* [fix] `FieldChip::range_check` should take `FieldPoint` instead of `UnsafeFieldPoint` (#209)

* fix: `FieldChip::range_check` should take `FieldPoint`

instead of `UnsafeFieldPoint`

* chore: fix audit-check CI

* chore: toggle CI on release branches

* [feat] update docs (#211)

* feat: update doc comments with function assumptions

* feat: update readme

* chore: fix CI

* [chore] fix dev graph tests (#212)

* chore: CI uses clippy all-targets

* fix: dev-graph tests (only works for halo2-pse)

Didn't bother refactoring halo2-axiom to support dev-graph

* [fix] `BasicDynLookupConfig` needs selector on advice table to prevent lookup poison (#206)

* fix: add fixed column to `BasicDynLookupConfig`

To prevent looking up into poisoned rows of `table`.

* feat: change `memory` example to use `BasicDynLookupConfig` for testing

* feat: change `BasicDynLookupConfig` to support zero key

* chore: move helper functions to `utils::halo2`

* [feat] add keccak circuit tests against Known Answer Test vectors (#213)

feat: add keccak circuit tests against Known Answer Test vectors

* [chore] fix documentation (#215)

* chore: fix keccak comment

* chore: remove redundant

* chore: fix test case description

* chore: fix documentation

* chore: add comment

* [chore] fix doc comment (#216)

chore: fix doc comment

* chore: fix halo2-pse compile

* fix: `TypeId` in `ContextTag` not stable across builds (#217)

* fix: use &str instead of TypeId in ContextTag

* chore: add warning to readme

* chore: fix comment

* chore: fix RAM test to use `&str` type id

* [chore] add crate prefix to `type_id`s (#218)

* chore: add crate prefix to `type_id`s

* fix: module_path! url

* chore: add type_id warning to `Context::new` and `ContextCell::new`

* chore: use halo2-axiom from crates.io

* chore: use poseidon-primitives from crates.io

* chore: Bump halo2-axiom to v0.4

* chore: add `get_mut` for keccak circuit params

* chore: bring back readme

---------

Co-authored-by: yulliakot <yulliakotel@gmail.com>
Co-authored-by: yuliakot <93175658+yuliakot@users.noreply.github.com>
Co-authored-by: Xinding Wei <weixinding@gmail.com>
Co-authored-by: PatStiles <33334338+PatStiles@users.noreply.github.com>
Co-authored-by: Jonathan Wang <jonathanpwang@users.noreply.github.com>
Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de>
Co-authored-by: PatStiles <pdstiles78@gmail.com>
Co-authored-by: mmagician <marcin.gorny.94@protonmail.com>
Co-authored-by: Xinding Wei <xinding@intrinsictech.xyz>
Co-authored-by: Antonio Mejías Gil <anmegi.95@gmail.com>
Co-authored-by: MonkeyKing-1 <67293785+MonkeyKing-1@users.noreply.github.com>
jonathanpwang added a commit that referenced this pull request Nov 30, 2023
* feat: upgrade overall `halo2-base` API to support future multi-threaded
assignments using our basic gate

* WIP: currently `gates::flex_gate` is updated and passes basic test

* BUG: `GateInstructions::idx_to_indicator` was missing a constraint to
check that the indicator witness was equal to 1 when non-zero.
* Previously the constraint ensured that `ind[i] = 0` when `idx != i`
  however `ind[idx]` could be anything!!!

* update: working benches for `mul` and `inner_product`

* feat: add `test_multithread_gates`

* BUG: `get_last_bit` did not do an `assert_bit` check on the answer

* this function was not used anywhere

* fix: `builder::assign_*` was not handling cases where two gates overlap
and there is a break point in that overlap
* we need to copy a cell between columns to fix

* feat: update `gates::range` to working tests and new API

* In keygen mode, the `CircuitBuilder` will clone the `ThreadBuilder`
  instead of `take`ing it because the same circuit is used for both vk
gen and pk gen. This could lead to more memory usage for pk gen.

* fix: change `AssignedValue` type to `KeccakAssignedValue` for
compatibility after halo2-base update

* Initial version 0.3.0 of halo2-ecc (#12)

* add multi-thread witness assignment support for `variable_base_msm`
  and `fixed_base_msm`
* batch size 100 MSM witness generation went from 500ms -> 100ms

* Sync with updates in `halo2_proofs_axiom`
* `assign_advice` not longer returns `Result` so no more `unwrap`

* Fix: assert uses of size hint in release mode (#13)

* remove `size_hint` in `inner_product_simple`

* change other uses of `size_hint` to follow with `assert_eq!` instead
  of `debug_assert_eq!`

* Fix: bit decomposition edge cases (#14)

* fix: change `debug_assert` in `decompose_u64_digits_limbs` to restrict
`bit_len < 64` and `decompose_biguint` to `64 <= bit_len < 128`
* add more comprehensive tests for above two functions

* Initial checkpoint for halo2-ecc v0.3.0 (#15)

* chore: clippy --fix

* Feat/add readme (#4)

* feat: add README

* feat: re-enable `secp256k1` module with updated tests

* chore: fix result println

* chore: update Cargo halo2_proofs_axiom to axiom/dev branch

* compatibility update with `halo2_proofs_axiom`

Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de>

* Fix: make `GateThreadBuilder` compatible with external usage (#16)

* chore: expose gate_builder.unknown

* feat: `GateThreadBuilder::assign_all` takes assigned_{advices,constants}
    as input instead of new hashmap, in case we want to constrain equalities
    for cells not belonging to this builder

* chore: update halo2-pse tag

* fix: `GateThreadBuilder::assign_all` now returns `HashMap`s of
    assigned cells for external equality constraints (e.g., instance cells,
    `AssignedCells` from chips not using halo2-lib).

* fix: `assign_all` was not assigning constants as desired: it was
    assigning a new constant per context. This leads to confusion and
    possible undesired consequences down the line.

* Fix: under-constrained `idx_to_indicator` (#17)

*fix(BUG): `GateChip::idx_to_indicator` still had soundness bug where at index
`idx` the value could be 0 or 1 (instead of only 1)

* feat: add some function documentation

* test(idx_to_indicator): add comprehensive tests
* both positive and negative tests

* Fix: soundness error in `FpChip::assert_eq` due to typo (#18)

* chore: update halo2-ecc version to 0.3.0

* fix(BUG): `FpChip::assert_equal` had `a` instead of `b` typo

* feat: add tests for `FpChip::assert_eq`
* positive and negative tests

* Remove redundant code and prevent race conditions (#19)

* feat: move `GateCircuitBuilder::synthesize` to `sub_synthesize` function
which also returns the assigned advices.

* reduces code duplication between `GateCircuitBuilder::synthesize` and
  `RangeCircuitBuilder::synthesize` and also makes it easier to assign
public instances elsewhere (e.g., snark-verifier)

* feat: remove `Mutex` to prevent non-deterministism

* In variable and fixed base `msm_par` functions, remove use of
  `Mutex<GateThreadBuilder>` because even the `Mutex` is not thread-
  safe in the sense that: if you let `Mutex` decide order
  that `GateThreadBuilder` is unlocked, you may still add Contexts to
  the builder in a non-deterministic order.

* fix: `fixed_base::msm_par` loading new zeros

* In `msm_par` each parallelized context was loading a new zero via
  `ctx.load_zero()`
* This led to using more cells than the non-parallelized version
* In `fixed_base_msm_in`, the if statement depending on
  `rayon::current_number_threads` leads to inconsistent proving keys
between different machines. This has been removed and now uses a fixed
number `25`.

* chore: use `info!` instead of `println` for params

* Allow `assign_all` also if `witness_gen_only = true`

* Fix: `inner_product_left_last` size hint (#25)

* Add documentation for halo2-base (#27)

* adds draft documentation for range.rs

* draft docs for lib.rs, utiils.rs, builder.rs

* fix: add suggested doc edits for range.rs

* docs: add draft documentation for flex_gate.rs

* fix: range.rs doc capitalization error

* fix: suggested edits for utils.rs docs

* fix: resolve comments for range.rs docs

* fix: resolve comments on flex_gate.rs docs

* fix: resolve comments for lib.rs, util.rs docs

* fix: resolve comments for builder.rs docs

* chore: use `info!` instead of `println` for params

* Allow `assign_all` also if `witness_gen_only = true`

* Fix: `inner_product_left_last` size hint (#25)

* docs: minor fixes

---------

Co-authored-by: PatStiles <pdstiles78@gmail.com>

* Smart Range Builder (#29)

* feat: smart `RangeCircuitBuilder`

Allow `RangeCircuitBuilder` to not create lookup table if it detects
that there's nothing to look up.

* feat: add `RangeWithInstanceCircuitBuilder`

* Moved from `snark-verifier-sdk`
* Also made this circuit builder smart so it doesn't load lookup table
  if not necessary
    * In particular this can also be used as a
      `GateWithInstanceCircuitBuilder`

* chore: derive Eq for CircuitBuilderStage

* fix: RangeConfig should not unwrap LOOKUP_BITS

* fix: `div_mod_var` when `a_num_bits <= b_num_bits` (#31)

* Feat: extend halo2 base test coverage (#35)

* feat: add flex_gate_test.rs and pos add() test

* feat: add pos sub() test

* feat: add pos neg() test

* feat: add pos mul() test

* feat: add pos mul_add() test

* feat: add pos mul_not() test

* feat: add pos assert_bit

* feat: add pos div_unsafe() test

* feat: add pos assert_is_const test

* feat: add pos inner_product() test

* feat: add pos inner_product_left_last() test

* feat: add pos inner_product_with_sums test

* feat: add pos sum_products_with_coeff_and_var test

* feat: add pos and() test

* feat: add pos not() test

* feat: add pos select() test

* feat: add pos or_and() test

* feat: add pos bits_to_indicator() test

* feat: add pos idx_to_indicator() test

* feat: add pos select_by_indicator() test

* feat: add pos select_from_idx() test

* feat: add pos is_zero() test

* feat: add pos is_equal() test

* feat: add pos num_to_bits() test

* feat: add pos lagrange_eval() test

* feat: add pos get_field_element() test

* feat: add pos range_check() tests

* feat: add pos check_less_than() test

* feat: add pos check_less_than_safe() test

* feat: add pos check_big_less_than_safe() test

* feat: add pos is_less_than() test

* feat: add pos is_less_than_safe() test

* feat: add pos is_big_less_than_safe() test

* feat: add pos div_mod() test

* feat: add pos get_last_bit() test

* feat: add pos div_mod_var() test

* fix: pass slices into test functions not arrays

* feat: Add pos property tests for flex_gate

* feat: Add positive property tests for flex_gate

* feat: add pos property tests for range_check.rs

* feat: add neg pranking test for idx_to_indicator

* fix: change div_mod_var test values

* feat(refactor): refactor property tests

* fix: fix neg test, assert_const, assert_bit

* fix: failing prop tests

* feat: expand negative testing is_less_than_failing

* fix: Circuit overflow errors on neg tests

* fix: prop_test_mul_not

* fix: everything but get_last_bit & lagrange

* fix: clippy

* fix: set LOOKUP_BITS in range tests, make range check neg test more

robust

* fix: neg_prop_tests cannot prank inputs

Inputs have many copy constraints; pranking initial input will cause all
copy constraints to fail

* fix: test_is_big_less_than_safe, 240 bits max

* Didn't want to change current `is_less_than` implementation, which in
  order to optimize lookups for smaller bits, only works when inputs
have at most `(F::CAPACITY // lookup_bits - 1) * lookup_bits` bits

* fix: inline doc for lagrange_and_eval

* Remove proptest for lagrange_and_eval and leave as todo

* tests: add readme about serial execution

---------

Co-authored-by: Jonathan Wang <jonathanpwang@users.noreply.github.com>

* fix(ecdsa): allow u1*G == u2*PK case (#36)

NOTE: current ecdsa requires `r, s` to be given as proper CRT integers

TODO: newtypes to guard this assumption

* fix: `log2_ceil(0)` should return `0` (#37)

* Guard `ScalarField` byte representations to always be little-endian (#38)

fix: guard `ScalarField` to be little-endian

* fix: get_last_bit two errors (#39)

2 embarassing errors:
* Witness gen for last bit was wrong (used xor instead of &)
* `ctx.get` was called after `range_check` so it was getting the wrong
  cell

* Add documentation for all debug_asserts (#40)

feat: add documentation for all debug_asserts

* fix: `FieldChip::divide` renamed `divide_unsafe` (#41)

Add `divide` that checks denomintor is nonzero.
Add documentation in cases where `divide_unsafe` is used.

* Use new types to validate input assumptions (#43)

* feat: add new types `ProperUint` and `ProperCrtUint`

To guard around assumptions about big integer representations

* fix: remove unused `FixedAssignedCRTInteger`

* feat: use new types for bigint and field chips

New types now guard for different assumptions on non-native bigint
arithmetic. Distinguish between:
- Overflow CRT integers
- Proper BigUint with native part derived from limbs
- Field elements where inequality < modulus is checked

Also add type to help guard for inequality check in
ec_add_unequal_strict

Rust traits did not play so nicely with references, so I had to switch
many functions to move inputs instead of borrow by reference. However to
avoid writing `clone` everywhere, we allow conversion `From` reference
to the new type via cloning.

* feat: use `ProperUint` for `big_less_than`

* feat(ecc): add fns for assign private witness points

that constrain point to lie on curve

* fix: unnecessary lifetimes

* chore: remove clones

* Better handling of EC point at infinity (#44)

* feat: allow `msm_par` to return identity point

* feat: handle point at infinity

`multi_scalar_multiply` and `multi_exp_par` now handle point at infinity
completely

Add docs for `ec_add_unequal, ec_sub_unequal, ec_double_and_add_unequal`
to specify point at infinity leads to undefined behavior

* feat: use strict ec ops more often (#45)

* `msm` implementations now always use `ec_{add,sub}_unequal` in strict
mode for safety
* Add docs to `scalar_multiply` and a flag to specify when it's safe to
  turn off some strict assumptions

* feat: add `parallelize_in` helper function (#46)

Multi-threading of witness generation is tricky because one has to
ensure the circuit column assignment order stays deterministic. To
ensure good developer experience / avoiding pitfalls, we provide a new
helper function for this.

Co-authored-by: Jonathan Wang <jonathanpwang@users.noreply.github.com>

* fix: minor code quality fixes (#47)

* feat: `fixed_base::msm_par` handles identity point (#48)

We still require fixed base points to be non-identity, but now handle
the case when scalars may be zero or the final MSM value is identity
point.

* chore: add assert for query_cell_at_pos (#50)

* feat: add Github CI running tests (#51)

* fix: ignore code block for doctest (#52)

* feat: add docs and assert with non-empty array checks (#53)

* Release 0.3.0 ecdsa tests (#54)

* More ecdsa tests

* Update mod.rs

* Update tests.rs

* Update ecdsa.rs

* Update ecdsa.rs

* Update ecdsa.rs

* chore: sync with release-0.3.0 and update CI

Co-authored-by: yulliakot <yulliakotel@gmail.com>
Co-authored-by: yuliakot <93175658+yuliakot@users.noreply.github.com>

* chore: fix CI

cannot multi-thread tests involving lookups due to environment variables

* fix: `prop_test_is_less_than_safe` (#58)

This test doesn't run any prover so the input must satisfy range check
assumption. More serious coverage is provided by
`prop_test_neg_is_less_than_safe`

* Add halo2-base readme (#66)

* feat: add halo2-base readme

* fix: readme formatting

* fix: readme edits

* fix: grammer

* fix: use relative links and formatting

* fix: formatting

* feat: add RangeCircuitBuilder description

* feat: rewording and small edits

---------

Co-authored-by: PatStiles <pdstiles78@gmail.com>

* fix: change all `1` to `1u64` to prevent unexpected overflow (#72)

* [Fix] Panic when dealing with identity point (#71)

* More ecdsa tests

* Update mod.rs

* Update tests.rs

* Update ecdsa.rs

* Update ecdsa.rs

* Update ecdsa.rs

* msm tests

* Update mod.rs

* Update msm_sum_infinity.rs

* fix: ec_sub_strict was panicing when output is identity

* affects the MSM functions: right now if the answer is identity, there
  will be a panic due to divide by 0 instead of just returning 0
* there could be a more optimal solution, but due to the traits for
  EccChip, we just generate a random point solely to avoid divide by 0
in the case of identity point

* Fix/fb msm zero (#77)

* fix: fixed_base scalar multiply for [-1]P

* feat: use `multi_scalar_multiply` instead of `scalar_multiply`

* to reduce code maintanence / redundancy

* fix: add back scalar_multiply using any_point

* feat: remove flag from variable base `scalar_multiply`

* feat: add scalar multiply tests for secp256k1

* fix: variable scalar_multiply last select

* Fix/msm tests output identity (#75)

* fixed base msm tests for output infinity

* fixed base msm tests for output infinity

---------

Co-authored-by: yulliakot <yulliakotel@gmail.com>

* feat: add tests and update CI

---------

Co-authored-by: yuliakot <93175658+yuliakot@users.noreply.github.com>
Co-authored-by: yulliakot <yulliakotel@gmail.com>

---------

Co-authored-by: yulliakot <yulliakotel@gmail.com>
Co-authored-by: yuliakot <93175658+yuliakot@users.noreply.github.com>

* fix: redundant check in `ec_sub_unequal`

* Add SafeType (#26)

* Add SafeType

* Refactor & add testing

* Add doc comment

* Refactor SafeChip

* Move gen_proof/check_proof to utils

* Fix merge issues

* feat(CI): switch to larger runner

* fix(builder): handle empty ctx with only equality constraints

* feat: add SafeAddress and SafeUint160 (#85)

* feat: add SafeAddress and SafeUint160

* fix incorrect byte size

* Merge release v0.3.0 into develop (#90)

* feat: `FpChip::range_check` now works with `max_bits < n * (k-1)` (#91)

* feat(base): range_check 0 bits by asserting is zero

* chore: add range_check 0 bits test

* feat(ecc): `FpChip::range_check` now works with `max_bits < n * (k-1)`

* Feat: test suite (#92)

* feat: stop using env var to pass around FLEX_GATE_CONFIG_PARAMS and

LOOKUP_BITS. Bad for testing (multi-threaded issues). Now we use
thread_local to have a global static for these config params that can be
passed around.

* chore: make utils folder and move some files

* Fix halo2 base tests naming (#76)

* feat: `BaseConfig` to switch between `FlexGateConfig` and `RangeConfig`

- `RangeCircuitBuilder` now uses `BaseConfig` to auto-decide whether to
  create lookup table or not.
    - In the future this should be renamed `BaseCircuitBuilder` or just
      `CircuitBuilder`, but for backwards compatibility we leave the name for now.
- `GateCircuitBuilder` no longer implements `Circuit` because we want to
  switch to having everyone just use `RangeCircuitBuilder`.
- Tests won't compile now because we still need to refactor

* feat: refactored halo2-base tests to use new test suite

* feat: remove use of env var in halo2-ecc

CI now can just run `cargo test`

* feat: remove use of env var from zkevm-keccak

* Add zkevm-keccak test to CI

* chore: fix CI

* chore: add lint to CI

* chore: make Baseconfig fns public

* fix(test): zkevm-keccak test should have `first_pass = SKIP_FIRST_PASS`

Currently with `first_pass = true`, it skips the first pass, but when
feature "halo2-axiom" is used, there is only one pass of `synthesize` so
the whole thing gets skipped. Mea culpa!

---------

Co-authored-by: Xinding Wei <weixinding@gmail.com>

* chore: make `bit_length` const function

* feat: add debugging functions (#99)

* feat: add debugging functions

Functions only available for testing:
* `ctx.debug_assert_false` for debug break point to search for other
  constrain failures in mock prover
* `assigned_value.debug_prank(prank_value)` to prank witness values for
  negative tests

* chore: code pretty

* chore: remove cfg(test) for debug functions

* feat(halo2-base): add `GateChip::pow_var` (#103)

* Use halo2curves v0.4.0 and ff v0.13 (#107)

* wip: change import to ff v0.13

* feat: remove `GateInstructions::get_field_element`

halo2curves now has `bn256-table` which creates table of small field
elements at compile time, so we should just use `F::from` always. This
also improves readability.

* chore: fix syntax and imports after update

* chore: add asm feature

* chore: workspace.resolver = 2

* chore: update ethers-core

* chore: add jemallocator feature to zkevm-keccak crate

* test: add bigger test case to keccak prover

* feat: use `configure_with_params`

remove `thread_local!` usage

* chore: bump zkevm-keccak version to 0.1.1

* feat: add `GateThreadBuilder::from_stage` for convenience

* chore: fixes

* fix: removed `lookup_bits` from `GateThreadBuilder::config`

* fix: debug_assert_false should load witness for debugging

* chore: use unreachable to document that Circuit::configure is never used

* chore: fix comment

* feat(keccak): use configure_with_params

* chore: fix halo2-pse errors

* chore: change halo2_proofs to main

* Add `sub_mul` to GateInstructions (#102)

* Add `sub_mul` to GateInstructions

* Add `sub_mul` prop test

* fix(test): `select_from_idx` wasn't calling the right function (#105)

* chore: add back RangeCircuitBuilder::config (#111)

* fix: `RangeCircuitBuilder::config` remember `lookup_bits`

* [Feat] Add Poseidon Hasher Chip (#110)

* Add Poseidon chip

* chore: minor fixes

* test(poseidon): add compatbility tests

Cherry-picked from #98

Co-authored-by: Antonio Mejías Gil <anmegi.95@gmail.com>

* chore: minor refactor to more closely match snark-verifier

https://github.com/axiom-crypto/snark-verifier/blob/main/snark-verifier/src/util/hash/poseidon.rs

---------

Co-authored-by: Xinding Wei <xinding@intrinsictech.xyz>
Co-authored-by: Jonathan Wang <31040440+jonathanpwang@users.noreply.github.com>
Co-authored-by: Antonio Mejías Gil <anmegi.95@gmail.com>

* feat: add VariableByteArray (#88)

* feat: add VariableByteArray

* fix: correct type in panic msg

* feat: make MAX_VAR_LEN const generic

* feat: add `SafeBool` and `SafeByte` types

These are very common so we have separate wrapper to avoid the extra length 1
vector heap allocation.

* wip: add VarLenBytes

* Refactor VarLenBytes
Add VarLenBytesVec and FixLenBytes
Fix tests

* Add unsafe methods for bytes
Address NITs

---------

Co-authored-by: Jonathan Wang <31040440+jonathanpwang@users.noreply.github.com>
Co-authored-by: Xinding Wei <xinding@intrinsictech.xyz>

* chore: reduce CI real prover load

* Rename PoseidonHasherChip to PoseidonHasher (#113)

Co-authored-by: Xinding Wei <xinding@intrinsictech.xyz>

* chore(safe_types): add conversion `SafeType` to/from `FixLenBytes`

* chore(safe_type): add `unsafe_to_safe_type` unsafe conversion fn

* feat: add `select_array_by_indicator` to `GateInstructions` (#115)

feat(base): add `select_array_by_indicator` to `GateInstructions`

* cleanup: use test-utils for benching (#112)

* cleanup: use test-utils for benching

* feat: add `{gen,check}_proof_with_instances`

* feat: add initial `bench_builder` cmd to `BaseTester`

* fix: cargo fmt

* fix(safe_types): `VarLenBytes` should allow `len == MAX_LEN` (#117)

* [feat] Add Poseidon Chip (#114)

* Add Poseidon hasher

* Fix test/lint

* Fix nits

* Fix lint

* Fix nits & add comments

* Add prover test

* Fix CI

* [chore] Reorg Folder Structure of hashes/zkevm (#118)

* chore: rename crate zkevm-keccak to zkevm-hashes

* fix: add `input_len` back to `KeccakTable`

* chore: move keccak specific constants to `keccak_packed_multi/util`

* Fix test

---------

Co-authored-by: Jonathan Wang <31040440+jonathanpwang@users.noreply.github.com>

* [fix] CI for zkevm hashes (#119)

Fix CI for zkevm hashes

* [chore] Split keccak implementation into multiple files (#120)

Split keccak implementation into multiple files

* feat: keccak constant visibility changes (#121)

feat: constant visibility changes

* [feat] Keccak Raw Output (#122)

* Replace raw keccak output instead of RLCOC

* Fix lint

* Add comments & improve expression performance

* Virtual region managers and dynamic lookup support (#123)

* feat: add `VirtualRegionManager` trait

Create `CopyConstraintManager` to manage global copy constraints.

* wip: separate `SinglePhaseGateManager` and `CopyConstraintManager`

`GateThreadBuilder` was very messy before

Todo:
- Split out lookup functionality

* wip: add `LookupAnyManager`

* wip: `RangeChip` with `LookupAnyManager`

reorg:
- previous `builder/threads` moved to `flex_gate/threads` because
it is all part of `FlexGateConfig` advice assignment logic
- `builder` moved to `range/circuit/builder.rs` as it is part of the
  assignment logic of `RangeCircuitBuilder`

* feat: working `BaseCircuitBuilder` backwards compatible

- `GateThreadBuilder` has become `MultiPhaseCoreManager`
- Some of the functionality has been moved into `BaseCircuitBuilder`,
  which is a generalization of `RangeCircuitBuilder`
- Some fixes on virtual managers because keygen calls `synthesize` twice
  (once for vk, once for pk) so can't drop

* fix: update halo2-ecc and sort `constant_equalities`

Sort `constant_equalities` to ensure deterministism.
Update `halo2-ecc` (mostly the tests) with new circuit builder format.

* fix: `LookupAnyManager` drop check `Arc` strong_count

* feat: add back single column lookup with selector

Special case: if only single advice column that you need to lookup, you
can create a selector and enable lookup on that column.
This means you add 1 selector column, instead of 1 advice column.

Only using this for `RangeConfig` and not generalizing it for now.

* feat: add example of dynamic lookup memory table

* Bump versions to 0.4.0

* chore: re-enable poseidon and safe_types

`Drop` for managers no longer panics because rust `should_panic` test
cannot handle non-unwinding panics.

* chore: remove `row_offset` from `assigned_advices`

This PR was merged: privacy-scaling-explorations/halo2#192

* chore: move `range::circuit` to `gates::circuit`

* nits: address review comments

* feat: add `num_instance_columns` to `BaseCircuitParams`

No longer a const generic

* chore(CI): use larger runner

* Use `raw_assign_{advice,fixed}` in keccak (#125)

* chore: switch `halo2_proofs` branch to `main`

* chore: use `raw_assign_{advice,fixed}` in keccak

* [feat] PoseidonHasher supports multiple inputs in compact format (#127)

* PoseidonHasher supports multiple inputs in compact format

* Add comments

* Remove unnecessary uses

* [feat] Expose Keccack Raw Inputs in Bytes instead of Input RLCs (#124)

* Expose Keccack raw inputs in bytes instead of input RLCs

* Fix column name in comments

* Add comments

* Compress 8 bytes of inputs into a single witness

* chore: add some comments

* Rewrite gates

* Fix comments & typos

* Fix naming

* Add comments

* Selector improvement

* Remove unused

---------

Co-authored-by: Jonathan Wang <31040440+jonathanpwang@users.noreply.github.com>

* Bump `zkevm-hashes` to v0.1.4

* chore: clippy fix

* Generic vertical gate assignment (#129)

* feat: make `single_phase::assign_with_constraints` generic

Use const generic for max rotations accessed by the vertical gate.
This way we can re-use the code for RLC gate.

* chore: make single_phase pub

* feat: add safety check for overlapping gates

* Add `deep_clone` to `BaseCircuitBuilder` (#131)

* chore: add convenience function to `BaseConfig`

* feat: add `deep_clone` to `BaseCircuitBuilder`

We sometimes want to clone `BaseCircuitBuilder` completely (for example
to re-run witness generation). The derived clone only clones the shared
references, instead of the underlying objects.

* fix: `SingleCorePhaseManager` should not create thread in constructor

Because the thread will default to phase 0.

* chore: make `new_context` public

* Convenience functions and fixes for multi-phase (#133)

* feat: add `clear` function to circuit builder and managers

* feat: add `BaseConfig::initialize`

* fix: break points for multiphase

* fix: clear should not change phase

* chore: remove dbg

* chore: add `BaseCircuitBuilder::set_k` fn

* fix: `CopyConstraintManager::clear` was dropping

* feat: impl `From<SafeByte>` for `AssignedValue`

* chore(poseidon): add `derive` statements

* fix(copy_constraints): backend permutation argument depends on order

Backend implementation of `constrain_equal` depends on the order in
which you add equality constraints, so it is not thread-safe...

* feat: add `left_pad` functions for var length arrays (#137)

* chore: use `PrimeField` for `OptimizedPoseidonSpec` (#139)

* chore: add getter functions to Poseidon spec (#140)

* feat: use `(TypeId, usize)` instead of `usize` for lookup tag (#142)

* chore: add `ContextTag` type alias

* feat(base): add `GateInstructions::inner_product_left` function (#143)

* chore: fix comments

* feat(base): add `GateInstructions::inner_product_left` function

* [feat] Keccak Coprocessor Leaf Circuit (#130)

* WIP

* chore: make `KeccakAssignedRow` fields public

* Refactor Keccak coprocessor circuit

* Optimize Keccak circuit MAX_DEGREE

* Fix comments

* Fix bug & typos

* Add testing & refactor folder structure

* Debugging

* Fix bugs

* Fix typo & bug

* feat(test): real prover tests use dummy input for keygen

* chore: make `LoadedKeccakF` public

* Also made `encoded_inputs_from_keccak_fs` public
* Both are useful for external use to make lookup tables in app circuits

* fix(keccak_leaf): review comments and optimization

* chore: use `gate` when `range` not necessary

* Move calculate base ciruit params out & Fix naming/comments

* Make Field-related parameter functions const

* feat: change `is_final` to `SafeBool`

* nit

* Fix typo

---------

Co-authored-by: Jonathan Wang <31040440+jonathanpwang@users.noreply.github.com>

* [feat] App Circuit Utils for Keccak Coprocessor (#141)

* Add keccak coprocessor encoding for VarLenBytesVec/FixLenBytesVec

* Fix naming/nits

* Fix nit

* [chore] Fix fmt (#144)

Fix fmt

* chore: add misc utility functions (#146)

* chore(keccak_leaf): make `generate_circuit_final_outputs` public

* chore: add misc utility functions

* feat(keccak): add `ingestion` module for Rust native input formatting (#147)

* chore(keccak): use `snark-verifier` native Poseidon for encoding (#148)

Currently only used for testing

* feat: optimize leaf poseidon with `hash_compact_chunk_inputs` (#149)

* [chore] cleanup code (#150)

chore: cleanup code

* chore: get halo2-pse working again

* chore: fix fmt

* [Doc] Keccak Doc (#145)

* Keccak docs

* Fix typos

* Add examples

* Fix comments/docs

* chore: pin snark-verifier branch

* [fix] max_rows in BaseCircuitBuilder in Keccak Leaf Circuit (#152)

Fix max_rows in BaseCircuitBuilder inside Keccak Vanilla

* [chore] Remove Unnecessary Lookup in Keccak Coprocessor Leaf Circuit (#153)

* chore: fix fmt

* [fix] max_rows in BaseCircuitBuilder in Keccak Leaf Circuit (#152)

Fix max_rows in BaseCircuitBuilder inside Keccak Vanilla

* Remove lookup in Keccak Leaf circuit

* [rename] (coprocessor, leaf) -> (component, shard) (#161)

* [fix] Multiple Phase Lookup (#162)

Fix multiple phase lookup

* [chore] add conversion `SafePrimitive` to `QuantumCell::Existing` (#169)

chore: add conversion `SafePrimitive` to `QuantumCell::Existing`

* fix: bad import on halo2-pse

* chore: make `{Fixed,Var}LenBytes*` constructor public

It's convenient to be able to construct the structs from vectors of safe
bytes externally. Only unsafe-ness is `len <= max_len` is not checked.

* [chore] add `cargo audit` to CI (#207)

chore: add `cargo audit` to CI

Upgrade criterion version

* [fix] `FieldChip::range_check` should take `FieldPoint` instead of `UnsafeFieldPoint` (#209)

* fix: `FieldChip::range_check` should take `FieldPoint`

instead of `UnsafeFieldPoint`

* chore: fix audit-check CI

* chore: toggle CI on release branches

* [feat] update docs (#211)

* feat: update doc comments with function assumptions

* feat: update readme

* chore: fix CI

* [chore] fix dev graph tests (#212)

* chore: CI uses clippy all-targets

* fix: dev-graph tests (only works for halo2-pse)

Didn't bother refactoring halo2-axiom to support dev-graph

* [feat] add keccak circuit tests against Known Answer Test vectors (#213)

feat: add keccak circuit tests against Known Answer Test vectors

* [chore] fix documentation (#215)

* chore: fix keccak comment

* chore: remove redundant

* chore: fix test case description

* chore: fix documentation

* chore: add comment

* [chore] fix doc comment (#216)

chore: fix doc comment

* fix: `TypeId` in `ContextTag` not stable across builds (#217)

* fix: use &str instead of TypeId in ContextTag

* chore: add warning to readme

* chore: fix comment

* [chore] add crate prefix to `type_id`s (#218)

* chore: add crate prefix to `type_id`s

* fix: module_path! url

* chore: add type_id warning to `Context::new` and `ContextCell::new`

* chore: use halo2-axiom from crates.io

* chore: use poseidon-primitives from crates.io

---------

Co-authored-by: Jonathan Wang <jonathanpwang@users.noreply.github.com>
Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de>
Co-authored-by: PatStiles <pdstiles78@gmail.com>
Co-authored-by: PatStiles <33334338+PatStiles@users.noreply.github.com>
Co-authored-by: yulliakot <yulliakotel@gmail.com>
Co-authored-by: yuliakot <93175658+yuliakot@users.noreply.github.com>
Co-authored-by: Xinding Wei <weixinding@gmail.com>
Co-authored-by: mmagician <marcin.gorny.94@protonmail.com>
Co-authored-by: Xinding Wei <xinding@intrinsictech.xyz>
Co-authored-by: Antonio Mejías Gil <anmegi.95@gmail.com>
Co-authored-by: MonkeyKing-1 <67293785+MonkeyKing-1@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant