Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Goblin proof construction #3332

Merged
merged 42 commits into from
Nov 22, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
2fc62de
Squash
codygunton Nov 17, 2023
34c8dd1
Everything builds
codygunton Nov 17, 2023
6669d2b
Fix test
codygunton Nov 17, 2023
c8c3794
Re-link univariate evaluations
codygunton Nov 17, 2023
0c0ea53
Rename struct
codygunton Nov 17, 2023
7c3d80f
Revert shplonk
codygunton Nov 18, 2023
9dc88fb
Cheat... ~> TranslationEvaluations
codygunton Nov 18, 2023
1490b7d
Clean up
codygunton Nov 18, 2023
d9ebe64
Scrap test for now
codygunton Nov 18, 2023
fe01fd3
cleanup
codygunton Nov 18, 2023
ee565cf
Untemplate composer
codygunton Nov 19, 2023
981b9f4
Untemplate prover and verifier
codygunton Nov 19, 2023
b534033
Reintroduce warnings
codygunton Nov 19, 2023
c55c858
Some composer cleanup
codygunton Nov 19, 2023
835e77e
Remove one unneeded layer of copying.
codygunton Nov 19, 2023
03fc165
Use lambda
codygunton Nov 19, 2023
98272a1
1/3 reduction in compilation time.
codygunton Nov 19, 2023
87a4438
Revert "Remove one unneeded layer of copying."
codygunton Nov 19, 2023
6e2edd3
temp add -ftime-trace
codygunton Nov 19, 2023
32439fb
Make bb-tests run translator_vm_tests
codygunton Nov 19, 2023
2e5b735
Leave note on reverted block
codygunton Nov 19, 2023
c37a199
Link TODO to issue
codygunton Nov 20, 2023
b242bc7
Resolve some TODOs
codygunton Nov 20, 2023
95cb520
Separate translation check.
codygunton Nov 20, 2023
e83fa58
Refactor previous commit
codygunton Nov 20, 2023
f8ee996
Wrap opqueue in shared_ptr
codygunton Nov 20, 2023
7c5c08c
Remove comments
codygunton Nov 20, 2023
7700e9f
Pass Ops to ECCVM through OpQueue
codygunton Nov 20, 2023
f785cb1
Remove comment
codygunton Nov 20, 2023
3aafd14
Reference issue
codygunton Nov 20, 2023
64184aa
Leave notes
codygunton Nov 20, 2023
66f979b
Remove compilation profiling
codygunton Nov 20, 2023
8f6b899
some cleanup
codygunton Nov 20, 2023
2f80643
Cleanup
codygunton Nov 20, 2023
0d11fae
Initialize op_queue
codygunton Nov 21, 2023
183fb56
Delete notes
codygunton Nov 21, 2023
97373ff
Merge branch 'master' into cg/full-goblin
codygunton Nov 21, 2023
4e9002f
Update sumcheck after rebase.
codygunton Nov 21, 2023
058430e
Fix typos
codygunton Nov 22, 2023
075b0e4
Reintroduce engine
codygunton Nov 22, 2023
9c81e24
Merge remote-tracking branch 'origin/master' into cg/full-goblin
codygunton Nov 22, 2023
678a8b8
Merge branch 'master' into cg/full-goblin
codygunton Nov 22, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions barretenberg/cpp/scripts/bb-tests.sh
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ TESTS=(
srs_tests
sumcheck_tests
transcript_tests
translator_vm_tests
ultra_honk_tests
)
TESTS_STR="${TESTS[@]}"
Expand Down
1 change: 1 addition & 0 deletions barretenberg/cpp/src/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@ add_subdirectory(barretenberg/srs)
add_subdirectory(barretenberg/stdlib)
add_subdirectory(barretenberg/sumcheck)
add_subdirectory(barretenberg/transcript)
add_subdirectory(barretenberg/translator_vm)
add_subdirectory(barretenberg/ultra_honk)
add_subdirectory(barretenberg/wasi)

Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
#include "barretenberg/flavor/ecc_vm.hpp"
#include "barretenberg/flavor/goblin_translator.hpp"
#include "barretenberg/flavor/goblin_ultra.hpp"
#include "barretenberg/flavor/ultra.hpp"
Expand All @@ -7,12 +8,16 @@ namespace {
auto& engine = numeric::random::get_debug_engine();
}

using namespace proof_system::honk::sumcheck;

namespace proof_system::benchmark::relations {

using FF = barretenberg::fr;
using Fr = barretenberg::fr;
using Fq = grumpkin::fr;

template <typename Flavor, typename Relation> void execute_relation(::benchmark::State& state)
{
using FF = typename Flavor::FF;
using AllValues = typename Flavor::AllValues;
using SumcheckArrayOfValuesOverSubrelations = typename Relation::SumcheckArrayOfValuesOverSubrelations;

Expand All @@ -29,82 +34,118 @@ template <typename Flavor, typename Relation> void execute_relation(::benchmark:
}
}

void auxiliary_relation(::benchmark::State& state) noexcept
void ultra_auxiliary_relation(::benchmark::State& state) noexcept
{
execute_relation<honk::flavor::Ultra, AuxiliaryRelation<FF>>(state);
execute_relation<honk::flavor::Ultra, AuxiliaryRelation<Fr>>(state);
}
BENCHMARK(auxiliary_relation);
BENCHMARK(ultra_auxiliary_relation);

void elliptic_relation(::benchmark::State& state) noexcept
void ultra_elliptic_relation(::benchmark::State& state) noexcept
{
execute_relation<honk::flavor::Ultra, EllipticRelation<FF>>(state);
execute_relation<honk::flavor::Ultra, EllipticRelation<Fr>>(state);
}
BENCHMARK(elliptic_relation);
BENCHMARK(ultra_elliptic_relation);

void ecc_op_queue_relation(::benchmark::State& state) noexcept
void ultra_ecc_op_queue_relation(::benchmark::State& state) noexcept
{
execute_relation<honk::flavor::GoblinUltra, EccOpQueueRelation<FF>>(state);
execute_relation<honk::flavor::GoblinUltra, EccOpQueueRelation<Fr>>(state);
}
BENCHMARK(ecc_op_queue_relation);
BENCHMARK(ultra_ecc_op_queue_relation);

void gen_perm_sort_relation(::benchmark::State& state) noexcept
void ultra_gen_perm_sort_relation(::benchmark::State& state) noexcept
{
execute_relation<honk::flavor::Ultra, GenPermSortRelation<FF>>(state);
execute_relation<honk::flavor::Ultra, GenPermSortRelation<Fr>>(state);
}
BENCHMARK(gen_perm_sort_relation);
BENCHMARK(ultra_gen_perm_sort_relation);

void lookup_relation(::benchmark::State& state) noexcept
void ultralookup_relation(::benchmark::State& state) noexcept
{
execute_relation<honk::flavor::Ultra, LookupRelation<FF>>(state);
execute_relation<honk::flavor::Ultra, LookupRelation<Fr>>(state);
}
BENCHMARK(lookup_relation);
BENCHMARK(ultralookup_relation);

void ultra_permutation_relation(::benchmark::State& state) noexcept
{
execute_relation<honk::flavor::Ultra, UltraPermutationRelation<FF>>(state);
execute_relation<honk::flavor::Ultra, UltraPermutationRelation<Fr>>(state);
}
BENCHMARK(ultra_permutation_relation);

void ultra_arithmetic_relation(::benchmark::State& state) noexcept
{
execute_relation<honk::flavor::Ultra, UltraArithmeticRelation<FF>>(state);
execute_relation<honk::flavor::Ultra, UltraArithmeticRelation<Fr>>(state);
}
BENCHMARK(ultra_arithmetic_relation);

void translator_decomposition_relation(::benchmark::State& state) noexcept
{
execute_relation<honk::flavor::GoblinTranslator, GoblinTranslatorDecompositionRelation<FF>>(state);
execute_relation<honk::flavor::GoblinTranslator, GoblinTranslatorDecompositionRelation<Fr>>(state);
}
BENCHMARK(translator_decomposition_relation);

void translator_opcode_constraint_relation(::benchmark::State& state) noexcept
{
execute_relation<honk::flavor::GoblinTranslator, GoblinTranslatorOpcodeConstraintRelation<FF>>(state);
execute_relation<honk::flavor::GoblinTranslator, GoblinTranslatorOpcodeConstraintRelation<Fr>>(state);
}
BENCHMARK(translator_opcode_constraint_relation);

void translator_accumulator_transfer_relation(::benchmark::State& state) noexcept
{
execute_relation<honk::flavor::GoblinTranslator, GoblinTranslatorAccumulatorTransferRelation<FF>>(state);
execute_relation<honk::flavor::GoblinTranslator, GoblinTranslatorAccumulatorTransferRelation<Fr>>(state);
}
BENCHMARK(translator_accumulator_transfer_relation);

void translator_gen_perm_sort_relation(::benchmark::State& state) noexcept
{
execute_relation<honk::flavor::GoblinTranslator, GoblinTranslatorGenPermSortRelation<FF>>(state);
execute_relation<honk::flavor::GoblinTranslator, GoblinTranslatorGenPermSortRelation<Fr>>(state);
}
BENCHMARK(translator_gen_perm_sort_relation);

void translator_non_native_field_relation(::benchmark::State& state) noexcept
{
execute_relation<honk::flavor::GoblinTranslator, GoblinTranslatorNonNativeFieldRelation<FF>>(state);
execute_relation<honk::flavor::GoblinTranslator, GoblinTranslatorNonNativeFieldRelation<Fr>>(state);
}
BENCHMARK(translator_non_native_field_relation);

void translator_permutation_relation(::benchmark::State& state) noexcept
{
execute_relation<honk::flavor::GoblinTranslator, GoblinTranslatorPermutationRelation<FF>>(state);
execute_relation<honk::flavor::GoblinTranslator, GoblinTranslatorPermutationRelation<Fr>>(state);
}
BENCHMARK(translator_permutation_relation);

void eccvm_lookup_relation(::benchmark::State& state) noexcept
{
execute_relation<honk::flavor::ECCVM, ECCVMLookupRelation<Fq>>(state);
}
BENCHMARK(eccvm_lookup_relation);

void eccvm_msm_relation(::benchmark::State& state) noexcept
{
execute_relation<honk::flavor::ECCVM, ECCVMMSMRelation<Fq>>(state);
}
BENCHMARK(eccvm_msm_relation);

void eccvm_point_table_relation(::benchmark::State& state) noexcept
{
execute_relation<honk::flavor::ECCVM, ECCVMPointTableRelation<Fq>>(state);
}
BENCHMARK(eccvm_point_table_relation);

void eccvm_set_relation(::benchmark::State& state) noexcept
{
execute_relation<honk::flavor::ECCVM, ECCVMSetRelation<Fq>>(state);
}
BENCHMARK(eccvm_set_relation);

void eccvm_transcript_relation(::benchmark::State& state) noexcept
{
execute_relation<honk::flavor::ECCVM, ECCVMTranscriptRelation<Fq>>(state);
}
BENCHMARK(eccvm_transcript_relation);

void eccvm_wnaf_relation(::benchmark::State& state) noexcept
{
execute_relation<honk::flavor::ECCVM, ECCVMWnafRelation<Fq>>(state);
}
BENCHMARK(eccvm_wnaf_relation);

} // namespace proof_system::benchmark::relations
Original file line number Diff line number Diff line change
Expand Up @@ -319,9 +319,9 @@ template <typename Curve> class ZeroMorphProver_ {
auto& multilinear_challenge,
auto& commitment_key,
auto& transcript,
const std::vector<Polynomial>& concatenated_polynomials = {},
const std::vector<std::span<FF>>& concatenated_polynomials = {},
const std::vector<FF>& concatenated_evaluations = {},
const std::vector<std::vector<Polynomial>>& concatenation_groups = {})
const std::vector<std::vector<std::span<FF>>>& concatenation_groups = {})
{
// Generate batching challenge \rho and powers 1,...,\rho^{m-1}
FF rho = transcript.get_challenge("rho");
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -225,6 +225,17 @@ template <class Curve> class ZeroMorphWithConcatenationTest : public CommitmentT
// Initialize an empty BaseTranscript
auto prover_transcript = BaseTranscript<Fr>::prover_init_empty();

std::vector<std::span<Fr>> concatenated_polynomials_views;
for (auto& poly : concatenated_polynomials) {
concatenated_polynomials_views.emplace_back(poly);
}

std::vector<std::vector<std::span<Fr>>> concatenation_groups_views(concatenation_groups.size());
for (auto [group_of_polys, group_of_views] : zip_view(concatenation_groups, concatenation_groups_views)) {
for (auto& poly : group_of_polys) {
group_of_views.emplace_back(poly);
}
}
// Execute Prover protocol
ZeroMorphProver::prove(f_polynomials, // unshifted
g_polynomials, // to-be-shifted
Expand All @@ -233,9 +244,9 @@ template <class Curve> class ZeroMorphWithConcatenationTest : public CommitmentT
u_challenge,
this->commitment_key,
prover_transcript,
concatenated_polynomials,
concatenated_polynomials_views,
c_evaluations,
concatenation_groups);
concatenation_groups_views);

auto verifier_transcript = BaseTranscript<Fr>::verifier_init_empty(prover_transcript);

Expand Down
2 changes: 2 additions & 0 deletions barretenberg/cpp/src/barretenberg/common/benchmark.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@
#include <string>
#include <unistd.h>

#pragma GCC diagnostic ignored "-Wunused-result" // GCC13 hits this

namespace {
/**
* If user provides the env var BENCHMARK_FD write benchmarks to this fd, otherwise default to -1 (disable).
Expand Down
37 changes: 20 additions & 17 deletions barretenberg/cpp/src/barretenberg/eccvm/eccvm_prover.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -320,35 +320,38 @@ template <ECCVMFlavor Flavor> void ECCVMProver_<Flavor>::execute_transcript_cons
transcript.send_to_verifier("Translation:hack_commitment", commitment_key->commit(hack));

// Get the challenge at which we evaluate the polynomials as univariates
FF evaluation_challenge_x = transcript.get_challenge("Translation:evaluation_challenge_x");
evaluation_challenge_x = transcript.get_challenge("Translation:evaluation_challenge_x");

// Collect the polynomials and evaluations to be batched
const size_t NUM_UNIVARIATES = 6; // 5 transcript polynomials plus the constant hack poly
std::array<Polynomial, NUM_UNIVARIATES> univariate_polynomials = { key->transcript_op, key->transcript_Px,
key->transcript_Py, key->transcript_z1,
key->transcript_z2, hack };
std::array<FF, NUM_UNIVARIATES> univariate_evaluations;
for (auto [eval, polynomial] : zip_view(univariate_evaluations, univariate_polynomials)) {
eval = polynomial.evaluate(evaluation_challenge_x);
}
translation_evaluations.op = key->transcript_op.evaluate(evaluation_challenge_x);
translation_evaluations.Px = key->transcript_Px.evaluate(evaluation_challenge_x);
translation_evaluations.Py = key->transcript_Py.evaluate(evaluation_challenge_x);
translation_evaluations.z1 = key->transcript_z1.evaluate(evaluation_challenge_x);
translation_evaluations.z2 = key->transcript_z2.evaluate(evaluation_challenge_x);

// Add the univariate evaluations to the transcript
transcript.send_to_verifier("Translation:op", univariate_evaluations[0]);
transcript.send_to_verifier("Translation:Px", univariate_evaluations[1]);
transcript.send_to_verifier("Translation:Py", univariate_evaluations[2]);
transcript.send_to_verifier("Translation:z1", univariate_evaluations[3]);
transcript.send_to_verifier("Translation:z2", univariate_evaluations[4]);
transcript.send_to_verifier("Translation:hack_evaluation", univariate_evaluations[5]);
transcript.send_to_verifier("Translation:op", translation_evaluations.op);
transcript.send_to_verifier("Translation:Px", translation_evaluations.Px);
transcript.send_to_verifier("Translation:Py", translation_evaluations.Py);
transcript.send_to_verifier("Translation:z1", translation_evaluations.z1);
transcript.send_to_verifier("Translation:z2", translation_evaluations.z2);
transcript.send_to_verifier("Translation:hack_evaluation", hack.evaluate(evaluation_challenge_x));

// Get another challenge for batching the univariate claims
FF batching_challenge = transcript.get_challenge("Translation:batching_challenge");

// Collect the polynomials and evaluations to be batched
const size_t NUM_UNIVARIATES = 6; // 5 transcript polynomials plus the constant hack poly
std::array<Polynomial*, NUM_UNIVARIATES> univariate_polynomials = { &key->transcript_op, &key->transcript_Px,
&key->transcript_Py, &key->transcript_z1,
&key->transcript_z2, &hack };
std::array<FF, NUM_UNIVARIATES> univariate_evaluations;

// Constuct the batched polynomial and batched evaluation
Polynomial batched_univariate{ key->circuit_size };
FF batched_evaluation{ 0 };
auto batching_scalar = FF(1);
for (auto [eval, polynomial] : zip_view(univariate_evaluations, univariate_polynomials)) {
batched_univariate.add_scaled(polynomial, batching_scalar);
batched_univariate.add_scaled(*polynomial, batching_scalar);
batched_evaluation += eval * batching_scalar;
batching_scalar *= batching_challenge;
}
Expand Down
6 changes: 6 additions & 0 deletions barretenberg/cpp/src/barretenberg/eccvm/eccvm_prover.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
#include "barretenberg/commitment_schemes/gemini/gemini.hpp"
#include "barretenberg/commitment_schemes/shplonk/shplonk.hpp"
#include "barretenberg/flavor/ecc_vm.hpp"
#include "barretenberg/goblin/translation_evaluations.hpp"
#include "barretenberg/plonk/proof_system/types/proof.hpp"
#include "barretenberg/relations/relation_parameters.hpp"
#include "barretenberg/sumcheck/sumcheck_output.hpp"
Expand All @@ -22,6 +23,7 @@ template <ECCVMFlavor Flavor> class ECCVMProver_ {
using CommitmentLabels = typename Flavor::CommitmentLabels;
using Curve = typename Flavor::Curve;
using Transcript = typename Flavor::Transcript;
using TranslationEvaluations = barretenberg::TranslationEvaluations;

public:
explicit ECCVMProver_(std::shared_ptr<ProvingKey> input_key, std::shared_ptr<PCSCommitmentKey> commitment_key);
Expand All @@ -43,6 +45,8 @@ template <ECCVMFlavor Flavor> class ECCVMProver_ {

Transcript transcript;

TranslationEvaluations translation_evaluations;

std::vector<FF> public_inputs;

proof_system::RelationParameters<FF> relation_parameters;
Expand All @@ -62,6 +66,8 @@ template <ECCVMFlavor Flavor> class ECCVMProver_ {

Polynomial quotient_W;

FF evaluation_challenge_x;

sumcheck::SumcheckOutput<Flavor> sumcheck_output;
pcs::gemini::ProverOutput<Curve> gemini_output;
pcs::shplonk::ProverOutput<Curve> shplonk_output;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -227,7 +227,8 @@ TYPED_TEST_SUITE(ECCVMTranscriptTests, FlavorTypes);
*/
TYPED_TEST(ECCVMTranscriptTests, ProverManifestConsistency)
{
GTEST_SKIP() << "WORKTODO: update and reinstate after the protocol is finalized.";
GTEST_SKIP() << "TODO(https://github.com/AztecProtocol/barretenberg/issues/782): update and reinstate after the "
"protocol is finalized.";
using Flavor = TypeParam;

// Construct a simple circuit
Expand Down Expand Up @@ -256,7 +257,8 @@ TYPED_TEST(ECCVMTranscriptTests, ProverManifestConsistency)
*/
TYPED_TEST(ECCVMTranscriptTests, VerifierManifestConsistency)
{
GTEST_SKIP() << "WORKTODO: update and reinstate after the protocol is finalized.";
GTEST_SKIP() << "TODO(https://github.com/AztecProtocol/barretenberg/issues/782): update and reinstate after the "
"protocol is finalized.";

using Flavor = TypeParam;

Expand Down Expand Up @@ -310,7 +312,8 @@ TYPED_TEST(ECCVMTranscriptTests, ChallengeGenerationTest)

TYPED_TEST(ECCVMTranscriptTests, StructureTest)
{
GTEST_SKIP() << "WORKTODO: update and reinstate after the protocol is finalized.";
GTEST_SKIP() << "TODO(https://github.com/AztecProtocol/barretenberg/issues/782): update and reinstate after the "
"protocol is finalized.";

using Flavor = TypeParam;

Expand Down
2 changes: 1 addition & 1 deletion barretenberg/cpp/src/barretenberg/eccvm/eccvm_verifier.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -270,7 +270,7 @@ template <typename Flavor> bool ECCVMVerifier_<Flavor>::verify_proof(const plonk
univariate_opening_verified = PCS::verify(pcs_verification_key, batched_univariate_claim, transcript);
}

return multivariate_opening_verified && univariate_opening_verified;
return sumcheck_verified.value() && multivariate_opening_verified && univariate_opening_verified;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haven't we already verified the sumcheck by this time?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah but it felt safer to add this check (why not? also what if somehow the option doesn't have a value and the early return is missed? not sure what happens then).

}

template class ECCVMVerifier_<honk::flavor::ECCVM>;
Expand Down
4 changes: 0 additions & 4 deletions barretenberg/cpp/src/barretenberg/flavor/flavor.test.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,6 @@
#include <cstddef>
#include <gtest/gtest.h>

#pragma GCC diagnostic ignored "-Wunused-local-typedefs"
#pragma GCC diagnostic ignored "-Wunused-variable"

namespace proof_system::test_flavor {
TEST(Flavor, Getters)
{
Expand Down Expand Up @@ -139,7 +136,6 @@ TEST(Flavor, GetRow)
return std::vector<FF>({ FF::random_element(), FF::random_element() });
});
Flavor::ProverPolynomials prover_polynomials;
size_t poly_idx = 0;
for (auto [poly, entry] : zip_view(prover_polynomials.pointer_view(), data)) {
*poly = entry;
}
Expand Down
Loading