Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Poc 1 code review #2

Merged
merged 76 commits into from Nov 25, 2018
Merged
Show file tree
Hide file tree
Changes from 74 commits
Commits
Show all changes
76 commits
Select commit Hold shift + click to select a range
dbbcec3
Initial commit
avive Oct 18, 2018
3ae13fd
Initial implementation
avive Oct 18, 2018
6e5535a
Slight refactoring
avive Oct 18, 2018
8cdef19
Update types and interfaces
avive Oct 19, 2018
7bcb3fa
BinaryString type
avive Oct 19, 2018
7874dc2
Redesign BinaryString
avive Oct 19, 2018
f21829e
Initial tests and deps
avive Oct 19, 2018
e7cad72
Additional tests
avive Oct 19, 2018
c96e2c0
Verifier update
avive Oct 19, 2018
7407948
CreteNipChallenge
avive Oct 19, 2018
b314ddb
Update CreteNipChallenge
avive Oct 19, 2018
a5ebef9
Basic verifier challenge creation tests
avive Oct 19, 2018
670edbd
NipChallenge bug fixi
avive Oct 19, 2018
3a037c2
CreteNipChallenge update
avive Oct 19, 2018
bf82789
Cleanup
avive Oct 20, 2018
79101c6
Additional tests and cleanups
avive Oct 20, 2018
8414c65
Added computation of sibling ids in BN for a binary string id
avive Oct 20, 2018
d4453e2
TestBTSiblings
avive Oct 20, 2018
fa991d0
Update verifier
avive Oct 20, 2018
f32b226
Verifier.Verify() code complete - not tested yet
avive Oct 20, 2018
4dd7801
Updated verifier and cleaned up
avive Oct 20, 2018
4c51fff
goforamt project
avive Oct 20, 2018
ee89703
Refactoring - move some types to internal from shared
avive Oct 20, 2018
af0e1ad
Check for valid values of n
avive Oct 20, 2018
68bb9ea
Comment shared types
avive Oct 20, 2018
97e7885
Minor formatting and comments
avive Oct 20, 2018
18a742c
Added playground to main
avive Oct 20, 2018
6f79fc2
Additonal comments
avive Oct 20, 2018
5f3080d
Added validation of proof leaf label value
avive Oct 20, 2018
5ea94a6
Verifier bug fix
avive Oct 20, 2018
eef60d7
Added support for left siblings
avive Oct 20, 2018
ed8314d
Basic prover skeleton
avive Oct 20, 2018
ee8074f
Basic prover functionality
avive Oct 20, 2018
897ef69
SMProver MVP code complete
avive Oct 20, 2018
3912df7
prover code cleanup
avive Oct 20, 2018
159205a
goformat proj
avive Oct 20, 2018
b410d1f
Formatting
avive Oct 20, 2018
2119f36
Basic prover test passing
avive Oct 20, 2018
f8705b8
Inmemory dag
avive Oct 20, 2018
743e73c
Proof generation
avive Oct 20, 2018
fa8f34f
Comments
avive Oct 21, 2018
88b87a9
NIP verifiction test
avive Oct 21, 2018
5395da1
Simplify TestNip
avive Oct 21, 2018
85e589c
Additional tests
avive Oct 21, 2018
162b2f9
Goformat proj
avive Oct 22, 2018
bfc1e8b
Padding integration test !!
avive Oct 22, 2018
81025a2
Fix main and format
avive Oct 22, 2018
b9373f8
Cleanup
avive Oct 22, 2018
ac9a79a
Additional tests
avive Oct 22, 2018
e1d7fcc
Change verifier
avive Oct 22, 2018
23588cf
Use id.IsOdd()
avive Oct 22, 2018
4916ba9
Use bytes.Equal instead of bytes.Compare
avive Oct 22, 2018
c8e0499
Add readme testing notes
avive Oct 22, 2018
b8a683a
Reduce prints in tests
avive Oct 22, 2018
265721c
Use IsEven() instead of % == 0
avive Oct 22, 2018
d5a6ece
storage wip
avive Oct 22, 2018
35ebd03
Formatting
avive Oct 23, 2018
30c6192
Store (wip)
avive Oct 24, 2018
3ffc8a3
Basic store tests
avive Oct 24, 2018
1182e8e
Dag storage
avive Oct 24, 2018
eb99fa1
Fix bug in bstring so all tests pass
avive Oct 24, 2018
0533f34
Update test iterations
avive Oct 24, 2018
7044232
Use buffered writer to boost proover speed
avive Oct 24, 2018
d3ff03a
Store cleanup and tests
avive Oct 24, 2018
b0630f9
Dag computation progress %
avive Oct 24, 2018
3cdead8
Additional logging info
avive Oct 24, 2018
c156b25
Minor cleanups
avive Oct 25, 2018
b9867d8
Store optimizations
avive Oct 28, 2018
6ba3b0a
Best of both worlds: use LRU cache to compute leaf labels - much lowe…
avive Oct 29, 2018
71d677a
goforamt and minor comments
avive Oct 29, 2018
3b87e7b
Cleanup
avive Oct 29, 2018
2ec8cfe
Trivial optimziations - x4 pref improvements
avive Nov 2, 2018
a5f759c
Added scrypt-based hashing per Iddo's suggestion
avive Nov 3, 2018
2ca01a9
cmd n flag
Nov 4, 2018
88fc040
Various cleanups
avive Nov 4, 2018
9e137a7
Merge branch 'poc' of https://github.com/spacemeshos/poet-ref into poc
avive Nov 4, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
7 changes: 7 additions & 0 deletions .gitignore
@@ -0,0 +1,7 @@
poet-ref
.idea
.DS_Store
*.bin
*.prof
*.pdf
*.test
17 changes: 17 additions & 0 deletions README.md
@@ -1 +1,18 @@
# poet-ref

This is a private repo designed to create a refernce poet implementatkon for testing of the open source poet server.

The goal is to dsitribute a binary 'blac box' go component of a verifier that can be used to verify proofs created by a prover.

## Build

```
go get -u github.com/kardianos/govendor
govendor sync
go build
```

## Run the tests
```
go test ./...
```
202 changes: 202 additions & 0 deletions internal/bstring.go
@@ -0,0 +1,202 @@
package internal

import (
"crypto/rand"
"errors"
"math"
"math/big"
"strconv"
)

type SMBinaryStringFactory struct {
// cache map[uint64] map[uint]*SMBinaryString
// cache1 map[string] *SMBinaryString
}

func NewSMBinaryStringFactory() BinaryStringFactory {

return &SMBinaryStringFactory{
// make(map[uint64] map[uint]*SMBinaryString, 500),
// make(map[string]*SMBinaryString, 500),
}
}

type SMBinaryString struct {
v uint64 // stored value
d uint // number of binary digits to display
f *SMBinaryStringFactory
}

// digits must be at least as large to represent v
func (f *SMBinaryStringFactory) NewBinaryStringFromInt(v uint64, d uint) (BinaryString, error) {

// todo: only test in debug builds but not in production ones
/*
l := uint(bits.Len64(v))
if l > d {
return nil, errors.New("invalid digits. Digits must be large enough to represent v in bits")
}*/

res := &SMBinaryString{
v: v,
d: d,
f: f,
}

return res, nil
}

// Create a new BinaryString from a string of 0s and 1s, e.g. "00111"
// Returns an error if s is not a valid binary string, e.g. it contains chars
// other then 0 or 1
// Any leading 0s will be included in the result
func (f *SMBinaryStringFactory) NewBinaryString(s string) (BinaryString, error) {

var v uint64

if s != "" {
parsed, err := strconv.ParseUint(s, 2, 64)
if err != nil {
return nil, err
}
v = parsed
}

res := &SMBinaryString{
v: v,
d: uint(len(s)),
f: f,
}

return res, nil
}

// Create a new random d digits long BinaryString. e.g for digits = 4 "0110"
// d <= 63
func (f *SMBinaryStringFactory) NewRandomBinaryString(d uint) (BinaryString, error) {

if d > 63 {
return nil, errors.New("unsupported # of digits. must be less or equals to 64")
}

if d == 0 { // the only id with 0 digits is ""
return f.NewBinaryString("")
}

// generate a random number with d digits

// compute 2^d
max := uint64(math.Pow(2, float64(d)))
//max := uint64(math.Exp2(float64(d)))

maxBig := new(big.Int).SetUint64(max)

// max int with d digits is 2^d - 1. The following returns rnd in range [0...2^d-1]
rndBig, err := rand.Int(rand.Reader, maxBig)
if err != nil {
return nil, err
}

v := rndBig.Uint64()
return f.NewBinaryStringFromInt(v, d)
}

// returns list of siblings on the path from s the root assuming s is a node identifier in a full binary tree
func (s *SMBinaryString) GetBNSiblings(leftOnly bool) ([]BinaryString, error) {

// slice of siblings
var res []BinaryString

if s.v == 0 && s.d == 0 { // special case - dag root node
return res, nil
}

// current node pointer
var nodeId BinaryString

// initial value
nodeId = s

for {

// append node's sibling to result
siblingNode, err := nodeId.FlipLSB()
if err != nil {
return nil, err
}

if !leftOnly || siblingNode.IsEven() {
// we add to results if caller didn't request leftOnly
// or she did and the sibling is a left sibling (LSB == '0')
res = append(res, siblingNode)
}

// println("Adding sibling: ", siblingNode.GetStringValue())

// continue with the node's parent node
nodeId, err = nodeId.TruncateLSB()

if err != nil {
return nil, err
}

if len(nodeId.GetStringValue()) == 0 {
break
}
}

if len(res) == 0 && !leftOnly {
return res, errors.New("expected one or more siblings on the path to root")
}

return res, nil
}

// Returns a new BinaryString with the LSB truncated. e.g. "0010" => "001"
func (s *SMBinaryString) TruncateLSB() (BinaryString, error) {
return s.f.NewBinaryStringFromInt(s.v>>1, s.d-1)
}

// Flip LSB. e.g. "0010" => "0011"
func (s *SMBinaryString) FlipLSB() (BinaryString, error) {
return s.f.NewBinaryStringFromInt(s.v^1, s.d)
}

// Get string representation. e.g. "00011"
func (s *SMBinaryString) GetStringValue() string {

if s.d == 0 {
// special case - empty binary string
return ""
}

// binary string encoding of s.v without any leading 0s
res := strconv.FormatUint(s.v, 2)

// prepend any leading 0s if needed
n := s.d - uint(len(res))
for n > 0 {
res = "0" + res
n--
}

return res
}

// Get the binary value encoded in the string. e.g. 12
func (s *SMBinaryString) GetValue() uint64 {
return s.v
}

func (s *SMBinaryString) IsEven() bool {
return s.v%2 == 0
}

func (s *SMBinaryString) IsOdd() bool {
return !s.IsEven()
}

// return number of digits including leading 0s if any
func (s *SMBinaryString) GetDigitsCount() uint {
return s.d
}
166 changes: 166 additions & 0 deletions internal/bstring_test.go
@@ -0,0 +1,166 @@
package internal

import (
"github.com/stretchr/testify/assert"
"testing"
)

func TestBTSiblingsBasic(t *testing.T) {

const d = "00010001011010010001100111000010010101101111111111101110000110"
f := NewSMBinaryStringFactory()
b, err := f.NewBinaryString(d)
assert.NoError(t, err)

siblings, err := b.GetBNSiblings(false)
assert.NoError(t, err)

// we have this nice invariant regarding the # of siblings
// related to the height of b (and the # of bits in its id)
assert.Equal(t, len(d), len(siblings))

for _, s := range siblings {
println(s.GetStringValue())
}
}

func TestBTSiblings(t *testing.T) {

const d = "0101"
var expectedResults = [4]string{"0100", "011", "00", "1"}

f := NewSMBinaryStringFactory()
b, err := f.NewBinaryString(d)
assert.NoError(t, err)

siblings, err := b.GetBNSiblings(false)
assert.NoError(t, err)

// we have this nice invariant regarding the # of siblings
// related to the height of b (and the # of bits in its id)
assert.Equal(t, len(d), len(siblings))

for idx, s := range siblings {
assert.Equal(t, expectedResults[idx], s.GetStringValue())
}
}

func TestBTLeftSiblings(t *testing.T) {

const d = "0101"
var expectedResults = [2]string{"0100", "00"}

f := NewSMBinaryStringFactory()
b, err := f.NewBinaryString(d)
assert.NoError(t, err)

siblings, err := b.GetBNSiblings(true)
assert.NoError(t, err)

assert.Equal(t, len(expectedResults), len(siblings))

for idx, s := range siblings {
println(s.GetStringValue())
assert.Equal(t, expectedResults[idx], s.GetStringValue())
}
}

func TestBinaryString(t *testing.T) {

const d = "00010001011010010001100111000010010101101111111111101110000110"
f := NewSMBinaryStringFactory()
b, err := f.NewBinaryString(d)

assert.NoError(t, err)
assert.Equal(t, uint(len(d)), b.GetDigitsCount())
assert.Equal(t, d, b.GetStringValue())
}

func TestFlipLSB(t *testing.T) {

const d = "101011100110000111110011101010101110111101000000001011110101110"
f := NewSMBinaryStringFactory()

b, err := f.NewBinaryString(d)
assert.NoError(t, err)

b1, err := b.FlipLSB()
assert.NoError(t, err)

s := b.GetStringValue()
s1 := b1.GetStringValue()
assert.NotEqual(t, s[len(s)-1], s1[len(s1)-1])
}

func TestTruncateLSB(t *testing.T) {

const d = "101011100110000111110011101010101110111101000000001011110101110"
f := NewSMBinaryStringFactory()

b, err := f.NewBinaryString(d)
assert.NoError(t, err)

b1, err := b.TruncateLSB()
assert.NoError(t, err)

s := b.GetStringValue()
s1 := b1.GetStringValue()
assert.Equal(t, len(s), len(s1)+1)

// reconstruct s by appending s[LSB] to s1 - the truncated string
s3 := s1 + s[len(s)-1:]
assert.Equal(t, s, s3)

}

func TestRandomBinaryString(t *testing.T) {
f := NewSMBinaryStringFactory()
for i := 0; i < 10000; i++ {
const d = uint(63)
b, err := f.NewRandomBinaryString(d)
assert.NoError(t, err)
assert.Equal(t, uint(d), b.GetDigitsCount())
}
}

func TestInvalidBinaryString(t *testing.T) {
f := NewSMBinaryStringFactory()
// try to create a binary string from an invalid string - contains chars other than 0 and 1
_, err := f.NewBinaryString("0102")
assert.Error(t, err)
}

/*
func TestInvalidBinaryStringFromInt(t *testing.T) {
f := NewSMBinaryStringFactory()
// try to create from int with insufficient number of digits to encode int value to bits
_, err := f.NewBinaryStringFromInt(1025, 5)
assert.Error(t, err)
}*/

func TestBinaryStringFromInt(t *testing.T) {
f := NewSMBinaryStringFactory()

// 126 decimal is 1111101, requiring 7 binary digits
v := uint64(125)

b, err := f.NewBinaryStringFromInt(v, 7)
assert.NoError(t, err)
//println(b.GetStringValue())
assert.Equal(t, uint(7), b.GetDigitsCount())
assert.Equal(t, v, b.GetValue())

// encode to an 8 bits binary string
b, err = f.NewBinaryStringFromInt(v, 8)
assert.NoError(t, err)
//println(b.GetStringValue())
assert.Equal(t, uint(8), b.GetDigitsCount())
assert.Equal(t, v, b.GetValue())

// encode to a 63 bits binary string
b, err = f.NewBinaryStringFromInt(v, 63)
assert.NoError(t, err)
//println(b.GetStringValue())
assert.Equal(t, uint(63), b.GetDigitsCount())
assert.Equal(t, v, b.GetValue())
}