[WIP] Input feed #8

bh2smith · 2018-11-30T14:22:43Z

New functionality includes

JSON to Serial R1CS and distributed R1CS
Plain Text to Serial R1CS

along with demo data.

Experimenting and Playing

Tests of these can be run in the command line (assuming installed dependencies) by

mvn -Dtest=distributedFromJSONTest test

and

mvn -Dtest=distributedFromTextTest test

To test on a third-party R1CS, one can edit the jsonFilePath and/or textFilePath in either of relations/DistributedInputFeedTest.java or relations/SerialInputFeedTest.java.

Note that one should consult the file format README in src/test/data/ before testing on your own format.

…son format

…ing() functionaly

# Conflicts: # src/main/java/relations/objects/LinearCombination.java

…nd VerificationKey

- JSON import now accepts string or long values in constraint dictionary.

- Offload some repeted code to function call - more general style for plain text input

fleupold

Great to see your progress! Looking at the size of this change we should have probably started sharing code earlier 😉.

I don't think the hash_transform files (e.g. hash_transform.qap.matrix_c) should be checked in. I like that you added simple examples to help understand what the python script does, but we shouldn't put our real use cases inside version control.

I think the python script in general, should not really be part of DIZK. It's unlikely that it will ever get merged in the main DIZK repository and could as well be a standalone repo that we as Gnosis manage.

To me, it looks like this PR contains a couple of different changes that would be nice to separate into their own PRs.

adding toString methods
pyscript to convert pepper into json (could become part of our dex-zksnarks repo or standalone)
distributedR1CSFromJSON + tests (I somehow can't find the implementation of that function)

pyscripts/r1cs/utils.py

src/main/java/profiler/generation/R1CSConstruction.java

src/main/java/relations/objects/LinearCombination.java

src/test/java/relations/DistributedInputFeedTest.java

fleupold

Cool, the R1CS file reader is really awesome. I think we should split this code up a little bit more since at the moment there are a lot of different readers added. I find it quite hard to understand what's going on.

src/main/java/relations/r1cs/FileToR1CS.java

# Conflicts: # src/main/java/profiler/scripts/upload_and_login.sh

…oesn't need to be in project directory

# Conflicts: # src/main/java/profiler/Profiler.java # src/main/java/profiler/scripts/launch.sh # src/main/java/profiler/scripts/upload_and_login.sh

…ed test data and update other test data accordingly

…feeders

…ended to the file path

…o true)

bh2smith added 23 commits November 22, 2018 20:10

introduce some python scripts to convert plaintext pepper output to j…

36ee6a2

…son format

update project dependencies, gitignore, readme and include dockerfile

9164ec0

Include basic toString() functionality for relevant R1CS classes

2011b0b

include test JSON data for roll_up exporter

5e775ea

extend LinearCombination to allow adding at arbitrary index and toStr…

e107275

…ing() functionaly

some style fixes and include serialSmallestExample function

43bb697

Include row index on Error and toString

a77c7ed

throw Error when profiler has wrong APP parameter.

e5422e1

[Temporary] generic R1CS doesn't need to have 1 as first input

956cf6f

update test data (inlude new pepper_out.json)

4168a9a

keep the first element is one assertion

58621f1

First satisfyable test case for InputFeed!

0f0645e

Merge branch 'r1cs-evaluation' into input_feed

2be1e60

# Conflicts: # src/main/java/relations/objects/LinearCombination.java

Update Test Data

ab0efb9

Introduce toString() functions for Proof, ProvingKey, ProvingKeyRDD a…

6f04086

…nd VerificationKey

update pyscripts and test data files

f741fe2

First functional PlainText input with README regarding input file format

c29f568

Update and include test files

6f454bb

First functional JSON to Distributed R1CS!

f269722

- Separate unit tests into Serial and Distributed.

31d963a

- JSON import now accepts string or long values in constraint dictionary.

- Begin construction of distributedR1CSFromText

35b315e

- Offload some repeted code to function call - more general style for plain text input

More basic input configuration for distributed input testing.

743c992

Update readme (JSON constraint values can be String or Integer).

7bb756f

bh2smith assigned fleupold and josojo Nov 30, 2018

Merge branch 'dev' into input_feed

c1ce3af

fleupold reviewed Nov 30, 2018

View reviewed changes

bh2smith added 2 commits December 1, 2018 10:15

implemented distributedR1CSFromText

24d1033

Avoid division by zero when numPartitions > numConstraints

82cf106

bh2smith added 30 commits December 18, 2018 13:37

Merge branch 'input_feed' into test_profiler

83a6199

# Conflicts: # src/main/java/profiler/scripts/upload_and_login.sh

new profiler scripting

d2fead3

Separate Profiler from InputProfiler and include serial Profiling

de0a44b

include distributed_1M and update InputProfiling

479183d

remove unused packages

0ddb44c

[lint] no more unchecked assignments!

6dbcf2c

update InputFeed Profiling - include serial and distributed

0e44293

Refactor class names to be CamelCase

9faa788

revert changes to Profiler as they are now included in InputProfiler

fead65c

[Saftey] Update profiling bash scripts so that key-pair information d…

55d15fc

…oesn't need to be in project directory

Merge branch 'input_feed' into test_profiler

6dd2f8f

# Conflicts: # src/main/java/profiler/Profiler.java # src/main/java/profiler/scripts/launch.sh # src/main/java/profiler/scripts/upload_and_login.sh

Index Error on profiler argument parser

3bc59f4

generalized input profiler scripts

93b8b9f

update test files

b00a2d3

update profile big instance script

d828712

Way better R1CS distribution!

6f7ba3a

include Full Cycle input Profiler in Input unit test.

18dcd5a

No need for abstract class in ditributed input feed. Delete some unus…

d2cac0b

…ed test data and update other test data accordingly

[Serial] remove unnecessary abstract class and clean up serial input …

1ce8e18

…feeders

unpacking script for dealing with zipped data files

b699f52

include boolean parameter for profiling when file:// needs to be prep…

ebfe015

…ended to the file path

Change the way aux is read (as line separate integers rather than space)

6deeead

update readme with witness file format

d840269

revert hard coded primary construction

b268a1c

no need to check satisfied in profiler (done when debug flag is set t…

785bb5a

…o true)

update file path to match with unpacking script.

75b2cd6

include common command guide for profiling scripts and environment setup

09dda65

typo in shell script

ff84ea4

public and aux inputs are now line separated instead of space separated

8dc5a0e

[BUG] bad parsing memory argument.

3ce1d0d

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] Input feed #8

[WIP] Input feed #8

bh2smith commented Nov 30, 2018 •

edited

fleupold left a comment •

edited

fleupold left a comment

[WIP] Input feed #8

Are you sure you want to change the base?

[WIP] Input feed #8

Conversation

bh2smith commented Nov 30, 2018 • edited

New functionality includes

Experimenting and Playing

fleupold left a comment • edited

Choose a reason for hiding this comment

fleupold left a comment

Choose a reason for hiding this comment

bh2smith commented Nov 30, 2018 •

edited

fleupold left a comment •

edited