feat: Implement C-GO interface of dcurl #228

Fangop · 2020-04-03T14:01:47Z

These code are already formatted with gofmt
And, the naming policy is based on iotaledger/iota.go/pow for consistency.
- Though, I changed a little bit for more easily to read the code, like the name of mutex.

Here provides a C-GO interface for dcurl, in format of HORNET.
These API are based on dynamic linking.
We should add the path to libdcurl.so to LIBRARY_PATH and LD_LIBRARY_PATH.

LIBRARY_PATH
- For avoiding relative path in dcurl/cgo, I removed cgo flags with relative path to libdcurl.so. It'll leads to compiling error for c code in cgo.
  The solution is add the path/to/libdcurl.so to LIBRARY_PATH.
  When compiling, compiler will find dcurl in LIBRARY_PATH.

Under the path of dcurl:

$ export LIBRARY_PATH=$LIBRARY_PATH:$(pwd)/build
$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$(pwd)/build

Cause libdcurl.so is in /build after dcurl being built.

Providing 2 versions of API for the usage in HORNET, one is implemented with mutex lock.
And, the result is different from original dcurl, it returns only nonce rather than full 2673 trytes.

The usage is exactly same as api in iota.go/pow.

API usage

parameters

trytes: Has type Trytes, actually a string, is the trytes you want to calculate in PoW.
mwm: Has type int, represent minimum weight magnitude.
parallelism(not required): Has type int, the number of thread you want to use in this PoW calculation.

Output

nonce: Rather than full 2163 trytes, we use nonce here for the API format of HORNET.

Test package using GO test

This test will test for API DcurlProofOfWork(), which simultaneously test core API DcurlEntry().

Make sure your LIBRARY_PATH and LD_LIBRARY_PATH are ready.

Under the path of dcurl:

$ export LIBRARY_PATH=$LIBRARY_PATH:$(pwd)/build
$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$(pwd)/build

Test pakcage using GO test

Under the path of dcurl/cgo/pow.

$ go test
PASS
ok      github.com/DLTcollab/dcurl/cgo/pow    3.223s

Use -v for more information.

$ go test -v
=== RUN   TestDcurlProofOfWork
--- PASS: TestDcurlProofOfWork (3.32s)
    pow_test.go:17: Use dcurl to calculate PoW with 8 threads enabled and MWM = 14
    pow_test.go:25: Hashed trytes: BJCVLLRQHEB9MVDJBQVMCCK9ZHEARJLGUMYTHAUPGCGNTLXZZU9TFSOPUJXCWSULBXLUMTU9HCECZ9999
PASS
ok      github.com/DLTcollab/dcurl/cgo/pow    3.328s

Import dcurl/cgo/pow for other go files

Still, make sure your LIBRARY_PATH and LD_LIBRARY_PATH are ready.

$ export LIBRARY_PATH=$LIBRARY_PATH:path/to/libdcurl.so
$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:path/to/libdcurl.so

Use go get if your go files(one you want to import dcurl/cgo/pow) is not managed by go module.
- The url should be github.com/JulianATA/dcurl/cgo/pow now.
- After merged, it'll be github.com/DLTcollab/dcurl/cgo/pow

$ go get  github.com/JulianATA/dcurl/cgo/pow

Here is an example for how to import /dcurl/cgo/pow.

package main

import (
	. "github.com/iotaledger/iota.go/consts"
	"github.com/iotaledger/iota.go/curl"
	"github.com/julianATA/dcurl/cgo/pow"
)

func main() {
	rawTx := "999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999A9RGRKVGWMWMKOLVMDFWJUHNUNYWZTJADGGPZGXNLERLXYWJE9WQHWWBMCPZMVVMJUMWWBLZLNMLDCGDJ999999999999999999999999999999999999999999999999999999YGYQIVD99999999999999999999TXEFLKNPJRBYZPORHZU9CEMFIFVVQBUSTDGSJCZMBTZCDTTJVUFPTCCVHHORPMGCURKTH9VGJIXUQJVHK999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999"
	mwm := 14
	nonce, _ := pow.DcurlProofOfWork(rawTx, mwm, 8)
	rawTx = rawTx[:len(rawTx)-NonceTrinarySize/3] + nonce
	hashedTrytes := curl.MustHashTrytes(rawTx)
	println("Hashed trytes: " + hashedTrytes)
}

Name the file main.go

$ go run main.go
Hashed trytes: BJCVLLRQHEB9MVDJBQVMCCK9ZHEARJLGUMYTHAUPGCGNTLXZZU9TFSOPUJXCWSULBXLUMTU9HCECZ9999

Close #217

marktwtn · 2020-04-05T06:55:32Z

goDcurl/src/pow_dcurl/pow_dcurl_test.go

+	hashedTrytes := curl.MustHashTrytes(rawTx)
+
+	t.Log("hashed Trytes:", hashedTrytes)
+	if guards.IsTransactionHashWithMWM(hashedTrytes, 10) {


It should be 14, which is mwm.

Thanks!
Use the mwm variable with casting instead of an exact number now.

marktwtn · 2020-04-05T06:59:28Z

goDcurl/src/pow_dcurl/pow_dcurl_test.go

+		t.Log("Correctly hash with mwm")
+	} else {
+		t.Error("Wrong hash trytes")


Unify the log and error message.

Thanks!
Use "Correct hash with mwm" and "Wrong hash with mwm" now.

marktwtn · 2020-04-05T07:13:12Z

Fix commit message:
berfore using it. -> before using it.
Providing 2 versions of API -> The commit provides 2 versions of API
one implemented with mutex lock. -> one is implemented with mutex lock.
And, the result is -> And the result is

should aware of -> should be aware of
And, the type transfer between GOlang and C is tricky too. -> and the type transfer between GOlang and C is tricky, too.
So, implement a test in GOlang can make sure it work correctly in these 2 aspect.
-> Hence implement a test in GOlang can make sure it works correctly in these 2 aspect.
fallowing steps. -> following steps.

marktwtn · 2020-04-05T07:17:52Z

goDcurl/src/pow_dcurl/pow_dcurl_test.go

+	t.Log("Testing an 8 threads dcurl pow with 14 as mwm")
+	nonce, err := DcurlProofOfWork(rawTx, mwm, 8)
+	if err == nil {
+		t.Log("dcurl PoW successed")


Thanks, updated!

marktwtn · 2020-04-05T07:23:54Z

goDcurl/src/pow_dcurl/pow_dcurl_test.go

+	rawTx := "999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999A9RGRKVGWMWMKOLVMDFWJUHNUNYWZTJADGGPZGXNLERLXYWJE9WQHWWBMCPZMVVMJUMWWBLZLNMLDCGDJ999999999999999999999999999999999999999999999999999999YGYQIVD99999999999999999999TXEFLKNPJRBYZPORHZU9CEMFIFVVQBUSTDGSJCZMBTZCDTTJVUFPTCCVHHORPMGCURKTH9VGJIXUQJVHK999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999"
+
+	mwm := 14
+	t.Log("Testing an 8 threads dcurl pow with 14 as mwm")


Change to Use dcurl to calculate PoW with 8 threads enabled and MWM = 14

marktwtn · 2020-04-05T16:05:34Z

goDcurl/src/pow_dcurl/pow_dcurl_test.go

+
+	t.Log("hashed Trytes:", hashedTrytes)
+	if guards.IsTransactionHashWithMWM(hashedTrytes, uint(mwm)) {
+		t.Log("Correct hash wih mwm")


marktwtn · 2020-04-06T06:48:27Z

goDcurl/src/pow_dcurl/pow_dcurl_test.go

+	"github.com/iotaledger/iota.go/guards"
+)
+
+func TestDcrulProofOfWork(t *testing.T) {


TestDcurlProofOfWork

jserv · 2020-04-14T11:23:35Z

Can we use the directory name go instead?

marktwtn · 2020-04-14T11:25:58Z

Can we use the directory name go instead?

If possible, I prefer cgo since we have another directory named as jni.

Fangop · 2020-04-14T11:33:44Z

It could be dcurl/cgo/pow or dcurl/go/pow.

The only thing invariant is the pow package should be consistent with its folder name.

Fangop · 2020-04-14T23:54:39Z

@jserv @marktwtn
Here is some further information about directory name.
I thought cgo is better than go, since go mostly refers to pure go implementations.

jserv · 2020-04-15T01:27:17Z

I thought cgo is better than go, since go mostly refers to pure go implementations.

Let's move forward to cgo name scheme.

These API are based on dynamic linking. User should always check LD_LIBRARY_PATH and LIBRARY_PATH before using it. The commit provides 2 versions of API for the usage in HORNET, one is implemented with mutex lock. And the result is different from original dcurl, it returns nonce rather than full 2673 trytes.

Here is a test for C-GO interface based on GOlang testing package. The reasons to implement a test in GOlang are format and type. We should be aware of the format difference between HORNET and dcurl, and the transfer between GOlang and C is tricky, too. Hence implement a test in GOlang can make sure it works correctly in these 2 aspect. It'll check with following steps. First, the PoW works with or without errors. Then, the length of nonce should be 27. Hashing will check if nonce is correctly encoded. Last, the hashed trytes should be correct with mwm

GO module is a dependency management system for GO. It makes dependency information explicit and easier to manage.

Fangop · 2020-04-15T08:56:04Z

@jserv @marktwtn
The directory name has been changed, with a new go module name github.com/DLTcollab/dcurl/cgo.

jserv · 2020-04-15T09:01:53Z

cgo/pow/pow.go

@@ -0,0 +1,78 @@
+package pow


Should the filename be revised to cgo/pow/pow_dcurl.go?

Originally, I use pow_dcurl.go.
Under the module name dcurl.go or github.com/DLTcollab/dcurl/cgo now,
it should be clear that pow here is using dcurl.
Hence I thought the postfix could be redundant.

In iota.go/pow, they have different postfix for different pow implementations.

jserv · 2020-04-15T09:33:40Z

Let's move forward!

marktwtn reviewed Apr 5, 2020

View reviewed changes

marktwtn reviewed Apr 6, 2020

View reviewed changes

marktwtn approved these changes Apr 14, 2020

View reviewed changes

JulianATA added 3 commits April 15, 2020 16:46

feat: Use GO module to manage C-GO interface

a8084c1

GO module is a dependency management system for GO. It makes dependency information explicit and easier to manage.

jserv reviewed Apr 15, 2020

View reviewed changes

jserv merged commit 2930b63 into DLTcollab:develop Apr 15, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Implement C-GO interface of dcurl #228

feat: Implement C-GO interface of dcurl #228

Fangop commented Apr 3, 2020 •

edited

Loading

marktwtn Apr 5, 2020

Fangop Apr 5, 2020

marktwtn Apr 5, 2020

Fangop Apr 5, 2020

marktwtn commented Apr 5, 2020

marktwtn Apr 5, 2020

Fangop Apr 5, 2020

marktwtn Apr 5, 2020

marktwtn Apr 5, 2020

marktwtn Apr 6, 2020

jserv commented Apr 14, 2020

marktwtn commented Apr 14, 2020

Fangop commented Apr 14, 2020 •

edited

Loading

Fangop commented Apr 14, 2020

jserv commented Apr 15, 2020

Fangop commented Apr 15, 2020

jserv Apr 15, 2020

Fangop Apr 15, 2020

Fangop Apr 15, 2020

jserv commented Apr 15, 2020

feat: Implement C-GO interface of dcurl #228

feat: Implement C-GO interface of dcurl #228

Conversation

Fangop commented Apr 3, 2020 • edited Loading

API usage

parameters

Output

Test package using GO test

Import dcurl/cgo/pow for other go files

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

marktwtn commented Apr 5, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jserv commented Apr 14, 2020

marktwtn commented Apr 14, 2020

Fangop commented Apr 14, 2020 • edited Loading

Fangop commented Apr 14, 2020

jserv commented Apr 15, 2020

Fangop commented Apr 15, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jserv commented Apr 15, 2020

Fangop commented Apr 3, 2020 •

edited

Loading

Fangop commented Apr 14, 2020 •

edited

Loading