Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Deserialization for felts #50

Merged
merged 2 commits into from Mar 6, 2023
Merged

Deserialization for felts #50

merged 2 commits into from Mar 6, 2023

Conversation

ilitteri
Copy link
Collaborator

@ilitteri ilitteri commented Mar 6, 2023

Changes

  • Implement deserialize method for felt.
  • Implement deseriliaze method for felts.
  • Remove MulTerm & AddTerm Serialize implementations and replace them with derives.
  • Also derive Deserialize for all the structs.

@ilitteri ilitteri requested a review from jrchatruc March 6, 2023 21:28
@matias-gonz matias-gonz merged commit 6ca8204 into integration Mar 6, 2023
5 checks passed
@matias-gonz matias-gonz deleted the deserialization branch March 6, 2023 21:42
jrchatruc pushed a commit that referenced this pull request Mar 8, 2023
@ilitteri
Integration between Noir and gnark's Backend (#45)
9ed1439 
* Update `acvm`

* Automatically build go backend when compiling

* Fix `preprocess`

When preprocessing (a.k.a. compiling the circuit or generating proving and verifying keys) we have to sample random witnesses because we don't know the ones that the user is going to use.

Nevertheless, the ACIR circuit does know this. We should find a way of knowing this too because this won't let us sample random circuit variables (witnesses) because they're pre-set in the ACIR circuit.

* cargo fmt

* Encode proving and verifying keys and proof

* Add opcode resolution when preprocessing

* Fix example raw R1CS

* Add examples

* Update

- The ONE public variable must be set before every other variable
- It is not necessary to count the public and the secret variables when building the R1CS because the are already being counted after added
- The `sum` from the `AddTerm` is already the index of the variable in the term
- Cleanup some prints

* Fix public variables counting when building witnesses

* Cleanup prints

* Fix proving & verifying key serialization

* Add constraint for multiplication terms

* Cleanup prints

* Update values flattening function

* Update prover initial values

* Fix proving & verifying keys and proof serialization

* Fix dependencies

* Improve naming

* Add product to terms

* Use `0` as an index for the `ONE_WIRE`

Because it is declared as the first variable it refers to it

* Update `NbPublicVariables` sent to `witness.Fill`

* Replace old example with a simpler one

* Fix errors

* Add target for installing `nargo`

* Cleanup simple circuit example

* Fix `preprocess` random values sampling

* Add an integration test

* Rename `acvm_interop` -> `backend`

* Deserialization for felts (#50)

* Derive `Deserialize` and `Serialize` in fields

* Implement `Deserialize`-like methods for felts

* Remove comment
ilitteri added a commit that referenced this pull request Mar 9, 2023
@ilitteri
Fix R1CS building (#44)
ee40083 
* Add function that builds an R1CS with a RawR1CS

* Add function that fills the witnesses (private & public)

* Add imports

* Update main (for testing)

* Remove print

* Add test functions

* Add hex dependency

* Update naming

* Add `rand` as a dev-dependencie

* Fix parameter naming

* Fix export name

* Implement `TestFeltsSerialization`

* Implement `TestU64Serialization`

* Expose structs and modules (for testing)

* Add tests for felt, felt vector and u64 serialization

Integrated with Go

* cargo fmt

* Fix imports

* Fix comment

* Update felts serialization

* Update deps

* Update Makefile

Now we can either run all the tests or specify one. Also we can see the prints outputs with the new added flag

* Implement `AddTerm` with JSON deserialization

* Implement `MulTerm` with JSON deserialization

* Implement `RawGate` with JSON deserialization

* Implement `RawR1CS` with JSON deserialization

* Add `helpers` module

* Cleanup main function

* Move struct to a module

* `PublicInputs` is no longer a BTreeSet

* Implement `AddTerm` with JSON deserialization

* Implement `MulTerm` with JSON deserialization

* Implement `RawGate` with JSON deserialization

* Implement `RawR1CS` with JSON deserialization

* Add `helpers` module

* Cleanup main function

* Update deps

* Move struct to a module

* `PublicInputs` is no longer a BTreeSet

* Update function naming

* Improve testing using randomness

* Finished testing

- Added unchecked helper methods for deserializing
- The hole struct is asserted

* Update Makefile

* Enabled testing for Go

* Fix Makefile

* Fix Makefile

* Remove old serialize terms functions

* cargo fmt

* Add serialization integration tests

- MulTerm
- MulTerms
- AddTerm
- AddTerms
- RawGate (TODO)
- RawGates (TODO)
- RawR1CS (TODO)

* Expose `acir_to_r1cs` structs

* Implement `MulTerm` and `AddTerm` structs

* Update serialization

* `public_inputs` is no longer a `BTreeSet`

Now it is a `Vec<Witness>`

* Implement Go-side integration test functions

* cargo fmt

* Fix tests

* cargo clippy

* Implement `Display` and `Copy`

- `Copy` for `MulTerm` & `AddTerm`
- `Display` for every local struct

* Add missing integration tests

* Update `RawR1CS` fields types

`num_variables` & `num_constraints` from `usize` to `u64` to make it compatible with the Go API

* Change `write!` with `writeln!` & `Display` with `Debug`

* cargo fmt

* Fix imports

* cargo fmt

* Update `extern` function names

* Fix R1CS building

* Integration between Noir and gnark's Backend (#45)

* Update `acvm`

* Automatically build go backend when compiling

* Fix `preprocess`

When preprocessing (a.k.a. compiling the circuit or generating proving and verifying keys) we have to sample random witnesses because we don't know the ones that the user is going to use.

Nevertheless, the ACIR circuit does know this. We should find a way of knowing this too because this won't let us sample random circuit variables (witnesses) because they're pre-set in the ACIR circuit.

* cargo fmt

* Encode proving and verifying keys and proof

* Add opcode resolution when preprocessing

* Fix example raw R1CS

* Add examples

* Update

- The ONE public variable must be set before every other variable
- It is not necessary to count the public and the secret variables when building the R1CS because the are already being counted after added
- The `sum` from the `AddTerm` is already the index of the variable in the term
- Cleanup some prints

* Fix public variables counting when building witnesses

* Cleanup prints

* Fix proving & verifying key serialization

* Add constraint for multiplication terms

* Cleanup prints

* Update values flattening function

* Update prover initial values

* Fix proving & verifying keys and proof serialization

* Fix dependencies

* Improve naming

* Add product to terms

* Use `0` as an index for the `ONE_WIRE`

Because it is declared as the first variable it refers to it

* Update `NbPublicVariables` sent to `witness.Fill`

* Replace old example with a simpler one

* Fix errors

* Add target for installing `nargo`

* Cleanup simple circuit example

* Fix `preprocess` random values sampling

* Add an integration test

* Rename `acvm_interop` -> `backend`

* Deserialization for felts (#50)

* Derive `Deserialize` and `Serialize` in fields

* Implement `Deserialize`-like methods for felts

* Remove comment

* Fix ci
ilitteri added a commit that referenced this pull request Mar 14, 2023
@matias-gonz @ilitteri
Plonk Backend Implementation (#51)
4016972 
* Add function that builds an R1CS with a RawR1CS

* Add function that fills the witnesses (private & public)

* Add imports

* Update main (for testing)

* Remove print

* Add test functions

* Add hex dependency

* Update naming

* Add `rand` as a dev-dependencie

* Fix parameter naming

* Fix export name

* Implement `TestFeltsSerialization`

* Implement `TestU64Serialization`

* Expose structs and modules (for testing)

* Add tests for felt, felt vector and u64 serialization

Integrated with Go

* cargo fmt

* Fix imports

* Fix comment

* Update felts serialization

* Update deps

* Update Makefile

Now we can either run all the tests or specify one. Also we can see the prints outputs with the new added flag

* Implement `AddTerm` with JSON deserialization

* Implement `MulTerm` with JSON deserialization

* Implement `RawGate` with JSON deserialization

* Implement `RawR1CS` with JSON deserialization

* Add `helpers` module

* Cleanup main function

* Move struct to a module

* `PublicInputs` is no longer a BTreeSet

* Implement `AddTerm` with JSON deserialization

* Implement `MulTerm` with JSON deserialization

* Implement `RawGate` with JSON deserialization

* Implement `RawR1CS` with JSON deserialization

* Add `helpers` module

* Cleanup main function

* Update deps

* Move struct to a module

* `PublicInputs` is no longer a BTreeSet

* Update function naming

* Improve testing using randomness

* Finished testing

- Added unchecked helper methods for deserializing
- The hole struct is asserted

* Update Makefile

* Enabled testing for Go

* Fix Makefile

* Fix Makefile

* Remove old serialize terms functions

* cargo fmt

* Add serialization integration tests

- MulTerm
- MulTerms
- AddTerm
- AddTerms
- RawGate (TODO)
- RawGates (TODO)
- RawR1CS (TODO)

* Expose `acir_to_r1cs` structs

* Implement `MulTerm` and `AddTerm` structs

* Update serialization

* `public_inputs` is no longer a `BTreeSet`

Now it is a `Vec<Witness>`

* Implement Go-side integration test functions

* cargo fmt

* Fix tests

* cargo clippy

* Implement `Display` and `Copy`

- `Copy` for `MulTerm` & `AddTerm`
- `Display` for every local struct

* Add missing integration tests

* Update `RawR1CS` fields types

`num_variables` & `num_constraints` from `usize` to `u64` to make it compatible with the Go API

* Change `write!` with `writeln!` & `Display` with `Debug`

* cargo fmt

* Fix imports

* cargo fmt

* Update `extern` function names

* Fix R1CS building

* Update `acvm`

* Automatically build go backend when compiling

* Fix `preprocess`

When preprocessing (a.k.a. compiling the circuit or generating proving and verifying keys) we have to sample random witnesses because we don't know the ones that the user is going to use.

Nevertheless, the ACIR circuit does know this. We should find a way of knowing this too because this won't let us sample random circuit variables (witnesses) because they're pre-set in the ACIR circuit.

* cargo fmt

* Encode proving and verifying keys and proof

* Add opcode resolution when preprocessing

* Fix example raw R1CS

* Add examples

* Update

- The ONE public variable must be set before every other variable
- It is not necessary to count the public and the secret variables when building the R1CS because the are already being counted after added
- The `sum` from the `AddTerm` is already the index of the variable in the term
- Cleanup some prints

* Fix public variables counting when building witnesses

* Cleanup prints

* Fix proving & verifying key serialization

* Add constraint for multiplication terms

* Cleanup prints

* Update values flattening function

* Update prover initial values

* Fix proving & verifying keys and proof serialization

* Fix dependencies

* Improve naming

* Add product to terms

* Use `0` as an index for the `ONE_WIRE`

Because it is declared as the first variable it refers to it

* Update `NbPublicVariables` sent to `witness.Fill`

* Replace old example with a simpler one

* Fix errors

* Add target for installing `nargo`

* Cleanup simple circuit example

* Fix `preprocess` random values sampling

* Add an integration test

* Rename `acvm_interop` -> `backend`

* Deserialization for felts (#50)

* Derive `Deserialize` and `Serialize` in fields

* Implement `Deserialize`-like methods for felts

* Abstract groth16 backend structures

* Abstract plonk backend structures

* Add general backend structures and helpers

* Refactor old structure

* Refactor plonk structs using interfaces

* Arithmetic opcode (#52)

* Add OpcodeArithmetic unmarshal

* Change OpcodeArithmetic to ArithmeticOpcode

* Update target test-go

* Update gnark_backend_ffi/backend/plonk/structs/arithmetic_opcode_test.go

Co-authored-by: Ivan Litteri <67517699+ilitteri@users.noreply.github.com>

---------

Co-authored-by: Ivan Litteri <67517699+ilitteri@users.noreply.github.com>

* Directive opcode (#53)

* Add OpcodeArithmetic unmarshal

* Change OpcodeArithmetic to ArithmeticOpcode

* Update target test-go

* Update gnark_backend_ffi/backend/plonk/structs/arithmetic_opcode_test.go

Co-authored-by: Ivan Litteri <67517699+ilitteri@users.noreply.github.com>

* Add directive_opcode

---------

Co-authored-by: Ivan Litteri <67517699+ilitteri@users.noreply.github.com>

* Acir unmarshal (#54)

* Add acit unmarshal

* Add acir unmarshal test

* Finish acir test

* Update Makefile

* Refactor backend package (#56)

* Refactor backend package

* Update Makefile

* Add ACIR JSON example

* Implement buildSparseR1CS function

* Import plonk_backend

* Fix field visibility

* Plonk backend wrapper (#55)

* Move errors

* Move c_go_structures

* Add prove with meta

* Add prove_with_pk

* Add verify_with_meta

* Add verify_with_vk

* Add preprocess

* Add feature plonk

* Move EC config

* Remove unused config

* Expose num_constraints()

* Fix value serialization

* Fix map from_felt

* Fix `buildSparseR1CS`

* Fix Arithmetic unmarshal (#57)

* Remove unused print

* Fix mul term condition

* Fix acir deserialize (#58)

* Fix acir deserialize

* Fix opcode (variable of type plonk.OpcodeUnpacker) is not an interface

* Remove unused print

* Fix `add_term` unmarshal

* Fix `mul_term` unmarshal

* Rename `OpcodeUnpacker` to `Opcode`

* Fix `DirectiveOpcode` `UnmarshalJSON`

* Rename missing `OpcodeUnpacker` to `Opcode`

---------

Co-authored-by: Iván Litteri <ilitteri@fi.uba.ar>

* Integrate Plonk backends (#62)

* Move serialize module

* Update Go API parameters naming

* Fix values/public inputs serialization step

* Cleanup imports

* Fix naming

* Add new error

* Add serialize module to the general backend level

* Update default backend

* Implement Go backend functions

All but the Meta ones

* Update `buildSparseR1CS`

* Refactor naming

* Made `buildWitnesses` generic for all backends

* Add imports

* Add a Plonk example

* Save SRS in a file when Preprocessing

* Fix curve when sampling alpha

* Add srs serialization

* Refactor SRS loading

* Fix verify with vk

- From the Rust side now we use `encoded_felts` for encoding the public inputs
- From the Go side, now we filter the real public inputs that come from Noir. And we load the SRS and initialize it for the verifying key.

* Fix prove with pk

- From the Rust side, we fix the serialization.
- From the Go side we fix the values adding an extra value (one).

* Fix `PlonkVerifyWithVK` parameter order

* Abstract felts encoding

* Fix and refactor values handling when building R1CS

* Fix and refactor opcodes handling

* Fix `ProveWithPK`

* Fix `PlonkVerifyWithVK`

* Refactor `PlonkExample`

* Add examples

- 0 != 1
- 2 == 2

* Add ToRadixDirective

* Remove log prints

* Patch unhandled directives

* Update KZG size

* Add more test programs for the integration test

* Update `serialization_tests`

* Add feature flags

* cargo fmt

* Ignore Go job in CI

---------

Co-authored-by: matias-gonz <maigonzalez@fi.uba.ar>

* Merge with main branch

* Fix

* Refactor Go backend modules (#63)

* Refactor modules

* Made `HandleValues` generic for all css

* Add TODO comment

* Refactor plonk_preprocess

* Refactor plonk_verify_wth_verifying_key

* Refactor plonk_prove_with_pk

* Refactor srs.hex path

* Remove old file

* Refactor `ProveWithPK`

* Refactor `VerifyWithVK`

* Refactor `Preprocess`

* Cleanup

* Add TODO

* Update comment

---------

Co-authored-by: matias-gonz <maigonzalez@fi.uba.ar>

* Refactor Go backend modules (#63)

* Refactor modules

* Made `HandleValues` generic for all css

* Add TODO comment

* Refactor plonk_preprocess

* Refactor plonk_verify_wth_verifying_key

* Refactor plonk_prove_with_pk

* Refactor srs.hex path

* Remove old file

* Refactor `ProveWithPK`

* Refactor `VerifyWithVK`

* Refactor `Preprocess`

* Cleanup

* Add TODO

* Update comment

---------

Co-authored-by: matias-gonz <maigonzalez@fi.uba.ar>

* Add a TODO

* Update `README.md` (#64)

* Add a TODO

* Update README.md

* Fix `README.md`

* Update `README.md`

- Fix typos
- Update old texts

* Add Serialization section

* Fix bug when handling variables (#65)

* Update README.md

* Fix `README.md`

* Update `README.md`

- Fix typos
- Update old texts

* Add Serialization section

* Fix bug

---------

Co-authored-by: Iván Litteri <ilitteri@fi.uba.ar>
Co-authored-by: Ivan Litteri <67517699+ilitteri@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jrchatruc jrchatruc jrchatruc approved these changes

Assignees

@ilitteri ilitteri

@matias-gonz matias-gonz

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@ilitteri @jrchatruc @matias-gonz