Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: use G2 precomputed lines for Miller loop #930

Merged
merged 13 commits into from Nov 28, 2023
Merged

Conversation

ivokub
Copy link
Collaborator

@ivokub ivokub commented Nov 21, 2023

Description

This PR adds G2 line precomputations as in #874, #925, but initialized lazily and embeded inside G2 element. This lazy computation allows to have all the benefits of using ML with precomputed lines, but without having to change the existing interfaces and allowing for some inputs be provided using lines and other as points (as needed in Groth16).

Additionally, added a way to fix the verification key completely inside the circuit. This provides less versatility than providing the verification key as a witness, but allows for considerable decrease in the number of public witness elements, which matters for Solidity verifier. The decrease of constraints due to fixing VK inside circuit is not substantial compared to providing precomputed lines as our non-native arithmetic is still not sufficiently flexible to handle all cases of operations with constant arguments.

This PR is still not complete - the precomputations and witness assignments are implemented only for BW6-761 currently as it is most critical for our use cases. But I think extending the approach to BLS12-377 first and then to other curves is fairly straightforward. It is not copy-paste though as we have different extension towers and optimisations for line computations.

Type of change

  • New feature (non-breaking change which adds functionality)

How has this been tested?

  • TestPairFixedTestSolve - Miller Loop with input G2 element with precomputation
  • TestKZGVerificationEmulated3ConstantVk - constant KZG verification key
  • TestKZGVerificationEmulated3Precomputed - witness KZG verification key with precomputation
  • TestBW6InBN254 - witness G16 verification key without precomputation
  • TestBW6InBN254Precomputed - witness G16 verification key with precomputation
  • TestBW6InBN254Constant - constant G16 verification key

How has this been benchmarked?

All cases BW6 key emulated in BN254. R1CS constraint system. Decreasing order "no precomputation" -> "precomputed as witness" -> "precomputed as constant":

  • KZG verification 5389344 -> 4087448 -> 4066748
  • Groth16 verification 6086776 -> 4839928 -> 4748682

Checklist:

  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have added tests that prove my fix is effective or that my feature works
  • I did not modify files generated from templates
  • golangci-lint does not output errors locally
  • New and existing unit tests pass locally with my changes
  • Any dependent changes have been merged and published in downstream modules

@ivokub
Copy link
Collaborator Author

ivokub commented Nov 22, 2023

I haven't run the full set of benchmarks, but I think the initial results are indicative. Verifying Groth16 over BW6 in BN254 emulated using R1CS I have the decrease from 6086776 constraints to 4748682. This is the case when the verifying key is fully constant.

Additionally, this construction allows for the case of providing precomputed lines as a witness. This is a bit more versatile, allowing to verify different circuits, but then would have to figure out how to provide precomputed lines verifiably. If we provide them as a secret witness, then the outer prover could cheat, but providing as public witness adds 18k elements to public witness. In that case the circuit size is 4839928.

Similarly, for KZG verification only have 5389344->4087448->4066748 constraints.

@ivokub
Copy link
Collaborator Author

ivokub commented Nov 22, 2023

@yelhousni - I currently have implemented only for BW6-761 as imo this is most critical for PLONK compression BW6-761->BN254 (currently at 80M constraints without this PR). We could try to implement for all curves (second priority for BLS12-377), but I would rather try to merge this quickly and then do other curves separately in other PR.

@ivokub ivokub changed the title perf: use G2 precomputed lines for Miller loop (WIP) perf: use G2 precomputed lines for Miller loop Nov 22, 2023
@ivokub ivokub marked this pull request as ready for review November 22, 2023 12:57
@ivokub ivokub self-assigned this Nov 22, 2023
@ivokub ivokub added perf P1: High Issue priority: high zk-evm labels Nov 22, 2023
@ivokub ivokub added this to the v0.10.0 milestone Nov 22, 2023
@ivokub ivokub mentioned this pull request Nov 23, 2023
16 tasks
Copy link
Contributor

@yelhousni yelhousni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a great refactoring! it takes away the necessity to understand the fixed-arg vs. variable-arg pairing implementation for a user. Great job! I think we can merge and then add the other curves.

backend/plonk/bls12-377/marshal.go Show resolved Hide resolved
std/algebra/emulated/sw_bw6761/pairing.go Outdated Show resolved Hide resolved
@ivokub ivokub merged commit 6c05ea4 into master Nov 28, 2023
7 checks passed
@ivokub ivokub deleted the perf/g2-precomputed branch November 28, 2023 15:24
ivokub added a commit that referenced this pull request Nov 28, 2023
commit 6c05ea4
Author: Ivo Kubjas <ivo.kubjas@consensys.net>
Date:   Tue Nov 28 16:24:47 2023 +0100

    perf: use G2 precomputed lines for Miller loop (#930)

    * feat: add lazy line eval for Miller loop

    * chore: go mod

    * fix: DoublePairFixed order

    * refactor: remove fixed Q specialized methods

    * chore: serialize lines for KZG key

    * chore: go generate

    * docs: add init docs

    * feat: add fixed KZG verification key init

    * test: add constant and fixed VK test cases

    * test: use fixed init

    * feat: add fixed Groth16 verification

    * fix: unused import

    * refactor: merge last manual iteration

commit a99d198
Author: Ivo Kubjas <ivo.kubjas@consensys.net>
Date:   Tue Nov 28 16:20:44 2023 +0100

    feat: add PLONK in-circuit verifier (#880)

    * test: add recursion hash tests

    * fix: accumulate MSM result

    * refactor: take emulated element for additional data

    * fix: handled infinity point in native multi scalar exp

    * fix: use only nbBits when creating scalar

    * feat: add PLONK verifier

    * feat: PlaceholderVerifyingKey takes the vk as argument

    * feat: f -> scalarApi

    * feat: addition of computeIthLagrangeAtZeta

    * feat: bsb commitments are added to pi

    * refactor: PlaceholderProof takes the proof as argument

    * fix: compute ith lagrange ok, hashToField failing

    * fix: native short hash output size

    * feat: add bw6

    * docs: add package documentation

    * refactor: describe error in panic

    * refactor: init curve and pairing implicitly

    * refactor: remove comments

    * docs: add package examples

    * feat: add all supported witness assignments

    * test: add MSM test

    * fix: remove todo panic

    * feat: add option shortcuts

    * fix: include hash to field in shortcut option

    * feat: use only CCS for placeholder proof and verifyingkey

    * chore: typos and cleanup

    * docs: add KZG package documentation

    ---------

    Co-authored-by: Thomas Piellard <thomas.piellard@consensys.net>

commit 62b52ea
Merge: ec07217 97156f3
Author: Youssef El Housni <youssef.elhousni@consensys.net>
Date:   Fri Nov 24 10:44:33 2023 -0500

    Merge pull request #933 from Consensys/perf/karabina-cycloSq

    Perf: variant of the Karabina cyclotomic squaring

commit 97156f3
Author: Youssef El Housni <youssef.housni21@gmail.com>
Date:   Fri Nov 24 10:27:00 2023 -0500

    refactor: apply PR review suggestions

commit f52c4cb
Author: Youssef El Housni <youssef.housni21@gmail.com>
Date:   Thu Nov 23 01:50:41 2023 -0500

    perf(bls12-377): implement a variant of Karabina cyclo square

commit d7e8d78
Author: Youssef El Housni <youssef.housni21@gmail.com>
Date:   Wed Nov 22 23:28:26 2023 -0500

    perf(bw6): implement a variant of Karabina cyclo square

commit ec07217
Merge: 3aa2559 5479586
Author: Youssef El Housni <youssef.elhousni@consensys.net>
Date:   Wed Nov 22 18:16:46 2023 -0500

    Merge pull request #931 from Consensys/perf/bw6-finalExp

    Perf: optimize addition chains in BW6-761 final exponentiation

commit 5479586
Author: Youssef El Housni <youssef.housni21@gmail.com>
Date:   Wed Nov 22 13:07:50 2023 -0500

    perf(bw6/finalExp): replace Add(x,x) by MulConst(x,2)

commit 65cd6ee
Author: Youssef El Housni <youssef.housni21@gmail.com>
Date:   Tue Nov 21 21:39:55 2023 -0500

    fix(linter): ineffectual assignment

commit d948c7c
Author: Youssef El Housni <youssef.housni21@gmail.com>
Date:   Tue Nov 21 21:27:02 2023 -0500

    perf(bw6/finalExp): optimize addition chains

commit 3aa2559
Author: Gautam Botrel <gautam.botrel@gmail.com>
Date:   Mon Nov 20 14:03:52 2023 -0600

    feat: if we don't compress we don't need the dict (#929)
@yelhousni yelhousni mentioned this pull request Dec 6, 2023
20 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P1: High Issue priority: high perf zk-evm
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants