Equihash updates. #4

davidlehn · 2017-09-15T18:15:12Z

No description provided.

- Expose default engine setting. - Proxy main calls to default engine.

- Refactor to make data easier to access. - Fix k check comment.

- Switch verify() to use a callback. - Process verify() with AsyncWorker. - Add more tests.

- Check value length is correct for k. - Check indices are distinct. - Check indices are ordered properly.

- Add API docs. - Use "seed" vs "inputs". - Use "solution" vs "value". - Add "nonceLength" option. - Add "maxNonces" option. - Add "nonce" option. - Return solution as native JS array. - Add sanity checks. - Allow any length seed. - Remove "Seed" class. - Partial handling of arbitrary length nonces. - Partial ability to start at any nonce. - Use streaming blake2b API. - Adapt tests that depended on older fixed sizes. - Comment out odd length seed tests. - Misc cleanups.

- Add "personal" bytes support, as used by reference blake2b implementation. - Add support function to create zcash personal bytes. - Minimize params passed around in addon. - Add all data to created Proof.

- Support buffers for nonces. - Allow solution as number if nonceLength provided. - Current nonce limitations that could be lifted: - size >= 4 bytes. - only first 4 bytes mutated as a counter. - Don't return seed in result proof.

dlongley · 2017-09-15T18:36:14Z

README.md

+- `personal`: buffer of "personal" bytes used in some blake2 initializations
+  (optional)
+- `nonce`: initial value of nonce (engine dependent type) (optional)
+- `maxNonces`: max number of nonces to check (optional)


Engine name specific options should be nested under the engine name. Potentially also nested under "engine".

dlongley · 2017-09-15T18:37:09Z

README.md

+`proof`:
+- `n`: equihash `n` parameter
+- `k`: equihash `k` parameter
+- `personal`: buffer of bytes used for "personal" bytes (engine dependent, optional)


Engine specific output should be nested under the engine name as a property. Potentially also nested under "engine".

Perhaps the engine name itself should also be included in the proof -- as an "algorithm". So "algorithm": "khot...". Then, from that, "algorithmParameters" can include any algorithm-specific parameters like "personal".

dlongley · 2017-09-15T18:38:00Z

README.md

+`proof`: same as output from `solve()`
+
+`options` (engine specific):
+- `personal`: buffer of "personal" bytes used in some blake2 initializations


See comments above regarding engine specific information.

dlongley · 2017-09-15T18:42:26Z

lib/index.js

+    return engine().verify(...arguments);
+  },
+  get PERSONALBYTES() {
+    return engine().PERSONALBYTES;


Seems engine/hash specific... why is this exposed as a core constant?

dlongley · 2017-09-15T18:43:39Z

lib/index.js

+  get PERSONALBYTES() {
+    return engine().PERSONALBYTES;
+  },
+  zcashPersonal(n, k) {


Maybe this should be _zcashPersonal as a private helper thing we're not exposing as part of the public API right now until we figure it out better. I would instead expect to find this, in the future, under .engine('zcash').PERSONAL_BYTES (we aren't implementing it now).

dlongley · 2017-09-15T18:48:37Z

lib/khovratovich/index.js

+    const b = Buffer.from(ab);
+    b.fill(0);
+    const dv = new DataView(ab);
+    dv.setUint32(0, nonce, true);


This is big-endian, yay 👍

The nonce is written here as little-endian. The nonce output is just arbitrary bytes so it doesn't really matter. This is just initializing the starting point to solve from and using little-endian as a convention mostly because zcash was doing so with its 256 bit nonce.

Yup, I was just crazy when I read this before. Endianness doesn't matter here, but it does matter with how the solution is serialized ... but that isn't done in this lib.

dlongley · 2017-09-15T18:50:51Z

lib/khovratovich/index.js

+    const stride = 1 << _k;
+    for(let i = 0; i < solution.length; i += (2 * stride)) {
+      if(solution[i] >= solution[i + stride]) {
+        return callback(null, false);


Why false here instead of an invalid solution? Where is the line between an "invalid" solution and a "false" one? I would have thought that a solution that is properly ordered, has no duplicates -- but is just wrong, is "false" ... anything else is invalid and generates an error. Not saying we have to do it that way, just saying that was my expectation.

It's open for debate where the line should be between an error and just a false return value. Or if it should all be one way or the other. Currently errors are mostly out of range or similar vs just bad solution data. Can revisit this as needed.

- Add 'algorithm' field, return in proof. - Add 'algorithmParamters' field, return in proof. - Move personalization to algorithmParamters. - Many updates to README. - Guess engine based on algorithm. - Move personalization constant to khovratovich engine. - Use more specific errors.

- Fix benchmark.js usage. Use onStart and onCycle to change seed. - Add predictable random values and crypto random values. - Print out more stats: min/max/mean/etc. - Due to large variations based on input, ensure at least 10 samples.

- Check solutions as they are found. - Order and return first solution that passes checks. - Small refactorings.

- Use stdint.h vs cstdint. - Define used endian.h macros.

C++11 initialization can cause crazy runtime behavior with older compilers. Change to use older initialization style.

davidlehn added 30 commits August 24, 2017 16:11

Simplify engine usage.

7b5ead5

- Expose default engine setting. - Proxy main calls to default engine.

Expose solution in big-endian format.

a45ce34

Add CHANGELOG.

3a5cc2e

Refactor, fix k check comment.

80e8910

- Refactor to make data easier to access. - Fix k check comment.

Add travis support.

381c867

Add more tests.

6236127

Use callback style for verify().

ff6ed34

- Switch verify() to use a callback. - Process verify() with AsyncWorker. - Add more tests.

Add travis badge.

47303f0

Use size_t.

4dade59

Use std::min.

e3462a4

Fix test names.

d2bfabe

Add .editorconfig.

ca23961

Reformat header.

fb8a079

Format code. Fix file header.

534132b

Formattting.

7479796

Style.

a211326

Update headers.

87806c1

Add verify checks.

730cf49

- Check value length is correct for k. - Check indices are distinct. - Check indices are ordered properly.

Order solution indices.

727a826

Use std::swap_ranges.

149c905

Cleanups.

d7ad3d3

Add more tests and test vectors.

1f24f4e

Add benchmark.

c6f9868

Use size_t.

c895ad5

Remove seed size comment.

a74b0ba

Add benchmark docs.

80c22ef

Add missing seed doc.

4fd8784

Add engine docs.

7f49d6e

Disable noop tests.

cd4ea19

davidlehn added 4 commits September 11, 2017 21:38

Refactoring and add "personal" bytes support.

730d4da

- Add "personal" bytes support, as used by reference blake2b implementation. - Add support function to create zcash personal bytes. - Minimize params passed around in addon. - Add all data to created Proof.

Add n=64,k=3 tests.

84ebeb5

Nonce and other updates.

42827bc

- Support buffers for nonces. - Allow solution as number if nonceLength provided. - Current nonce limitations that could be lifted: - size >= 4 bytes. - only first 4 bytes mutated as a counter. - Don't return seed in result proof.

Remove old constants.

712b79c

davidlehn requested review from dlongley, mattcollier and msporny September 15, 2017 18:15

dlongley requested changes Sep 15, 2017

View reviewed changes

davidlehn added 15 commits September 18, 2017 13:26

Keep nonce data in little-endian format.

77aaa45

Split out zcash code and vectors.

a45c7d2

Fix benchmark, check for basic errors.

e13c426

Add verifySync API.

501ed5a

Fix typos.

dc2c0d9

Check for solve errors.

e04ed92

Benchmark updates.

a37c7a7

- Fix benchmark.js usage. Use onStart and onCycle to change seed. - Add predictable random values and crypto random values. - Print out more stats: min/max/mean/etc. - Due to large variations based on input, ensure at least 10 samples.

Optimize checking and returning first solution.

9a2b212

- Check solutions as they are found. - Order and return first solution that passes checks. - Small refactorings.

Fix typo.

6491fa8

Add support to dump benchmark data.

172cd22

Return error if no solutions found.

5aa060e

Improve mac os x support.

1e1c568

- Use stdint.h vs cstdint. - Define used endian.h macros.

Change vector initialization.

8a91e15

C++11 initialization can cause crazy runtime behavior with older compilers. Change to use older initialization style.

Fix typos.

bcdb7d0

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Equihash updates. #4

Equihash updates. #4

Uh oh!

davidlehn commented Sep 15, 2017

Uh oh!

dlongley Sep 15, 2017

Uh oh!

dlongley Sep 15, 2017

Uh oh!

dlongley Sep 15, 2017

Uh oh!

dlongley Sep 15, 2017

Uh oh!

dlongley Sep 15, 2017

Uh oh!

dlongley Sep 15, 2017

Uh oh!

dlongley Sep 15, 2017

Uh oh!

davidlehn Sep 18, 2017

Uh oh!

dlongley Sep 18, 2017

Uh oh!

dlongley Sep 15, 2017

Uh oh!

davidlehn Sep 18, 2017

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Equihash updates. #4

Are you sure you want to change the base?

Equihash updates. #4

Uh oh!

Conversation

davidlehn commented Sep 15, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants