whisperv5: integrate whisper and implement API #14540

bas-vk · 2017-05-29T08:47:54Z

This PR holds several things:

it integrates whisper in the node. That means that whisper can be configured through command line flags or through a config file. It can be started by given one or more of the whisper flags. E.g. geth --shh.maxmessagesize <...> would enable whisper.

Whisper (EXPERIMENTAL) OPTIONS:
  --shh                       Enable Whisper
  --shh.maxmessagesize value  Max message size accepted (default: 1048576)
  --shh.pow value             Minimum POW accepted (default: 0.2)

implements the whisper v5 API.
adds the whisper/whisperv5/client package. This client works in a similar way as the ethclient package and can be used to build whisper go DApp's.

gluk256 · 2017-06-09T09:35:31Z

👍

gluk256

LGTM

holiman · 2017-06-13T12:34:25Z

whisper/whisperv5/whisper.go

+	w.statsMu.Lock()
+	defer w.statsMu.Unlock()
+
+	return w.stats


The locking here seems unnecessary, you're returning the w.stats object, so if the caller does modifications on it, these locks won't prevent that.

w.stats is returned by value which makes a copy.

bas-vk · 2017-06-15T09:53:55Z

Last commit removed commented out code.

fjl

Needs minor code changes.

fjl · 2017-06-21T07:22:08Z

whisper/whisperv5/client.go

+// You should have received a copy of the GNU Lesser General Public License
+// along with the go-ethereum library. If not, see <http://www.gnu.org/licenses/>.
+
+package whisperv5


The client should be in its own package, "whisper/shhclient" to remove the rpc dependency from whisper core.

fjl · 2017-06-21T07:23:01Z

whisper/whisperv5/config.go

+	}
+
+	Flags = []cli.Flag{WhisperEnabledFlag, MaxMessageSizeFlag, MinPOWFlag}
+)


I don't like adding CLI flags to library packages. We have some in internal/debug, but it should usually be avoided.

We can put them into "whisper/shhapi" for now.

fjl · 2017-06-21T07:23:56Z

whisper/whisperv5/envelope.go

@@ -62,7 +62,7 @@ func (e *Envelope) rlpWithoutNonce() []byte {

 // NewEnvelope wraps a Whisper message with expiration and destination data
 // included into an envelope for network forwarding.
-func NewEnvelope(ttl uint32, topic TopicType, aesNonce []byte, msg *SentMessage) *Envelope {
+func NewEnvelope(ttl uint32, topic TopicType, aesNonce []byte, msg *sentMessage) *Envelope {


Making sentMessage private means that no code outside package whisperv5 can call NewEnvelope. Not sure if that's good or not.

You can still create a sent message with whisperv5.NewSentMessage.

fjl · 2017-06-21T07:25:59Z

whisper/whisperv5/config.go

+	"gopkg.in/urfave/cli.v1"
+)
+
+//go:generate gencodec -type Config -formats toml -out gen_config.go


There is no need to use gencodec because none of its features (required fields, field type override, etc.) are used. Please remove the generated file and this line.

fjl · 2017-06-21T07:27:06Z

whisper/whisperv5/api_test.go

-	"github.com/ethereum/go-ethereum/common/hexutil"
-)
-
+/*


Please delete the file if these tests are no longer meaningful.

fjl · 2017-06-21T07:28:45Z

whisper/whisperv5/whisper.go


 	mailServer MailServer // MailServer interface
 }

 // New creates a Whisper client ready to communicate through the Ethereum P2P network.
-func New() *Whisper {
+func New(cfg *Config) *Whisper {
 	whisper := &Whisper{


Please add

if cfg == nil { cfg = DefaultConfig }

fjl · 2017-06-21T07:31:16Z

whisper/whisperv5/whisper.go

+	}
+	m2[idx] = val
+	s.vault.Store(m2)
+}


I don't get this type. Please explain. It smells like overengineering.
If you really need a fast concurrent map, use golang.org/x/sync/syncmap.

With the api it is possible to change several settings at runtime. This requires locks to prevent simultaneous updates and reads. I have replace the custom map with the syncmap as suggested.

fjl · 2017-06-21T07:34:27Z

whisper/whisperv5/topic.go

 	}
-	*t = BytesToTopic(b)
+	*t = BytesToTopic(data)


You can use hexutil.UnmarshalFixedText here.

fjl · 2017-06-21T07:38:13Z

whisper/whisperv5/api.go

+	InvalidPublicKeyErr    = errors.New("invalid public key")
+	InvalidSigningPubKey   = errors.New("invalid signing public key")
+	TooLowPoWErr           = errors.New("message rejected, PoW too low")
+	NoTopicsErr            = errors.New("missing topic(s)")


Error variables should be named ErrXXX.

fjl · 2017-06-21T07:43:22Z

General feedback about the API: There are lots of key management methods. These can be hard to use correctly. IMHO we should use the normal keystore for asymmetric keys and push responsibility to generate and manage symmetric keys to the user of the API.

bas-vk · 2017-06-21T11:03:46Z

Moving from the current key management to a "normal" keystore should be something that is done in a different PR. This one is already getting huge.

@gluk256 ping.

gluk256 · 2017-06-21T15:05:29Z

Please, let us keep focus on the current task. This PR has one very specific purpose: make the v5 working again. Refactoring could be done only after we will be able to build the project and call all the functions.

gluk256 · 2017-06-21T15:08:21Z

@fjl If you want, we can discuss the key management later, and refactor if necessity arise. But as long as whisper remains broken, we should concentrate on fixing it ASAP.

frozeman · 2017-06-22T14:05:30Z

Some small fixes necessary after testing:

shh_info: message needs renaming to messages in the returned object
sometimes generated sym keys don't get stored or something, but creating filters fail with unknown symkey key (even tho they were created before), and shh_hasSymKey returns undefined (should at least return false)
i also get errors like specify either a symetric or a asymmetric key when creating filters, even tho i have generated an specified a symKeyID and shh.hasSymKey(id) returns true

bas-vk · 2017-06-23T07:24:00Z

shh_info: message needs renaming to messages in the returned object

Renamed

sometimes generated sym keys don't get stored or something, but creating filters fail with unknown symkey key (even tho they were created before), and shh_hasSymKey returns undefined (should at least return false)

The only possible return value of shh_hasSymKey is a bool. Would be nice get an example script that reproduces the issue.

i also get errors like specify either a symetric or a asymmetric key when creating filters, even tho i have generated an specified a symKeyID and shh.hasSymKey(id) returns true

I expect that you use an invalid field name but an example script would help. Here is an example with symmetric keys that uses filters.

frozeman · 2017-06-23T09:27:23Z

The last 2 errors were a problem on my side. From what i tested it looks like working. 👍

frozeman · 2017-06-23T10:17:10Z

Once this is merged, we can add it to the geth console internally: web3/web3.js#902
The files in this PR in the distilled folder can be used.

frozeman · 2017-06-23T14:48:34Z

Those files can be used: Please include them and test the API and basic functions inside the console.
https://github.com/ethereum/web3.js/tree/develop/dist

karalabe · 2017-06-26T10:39:46Z

cmd/geth/usage.go

@@ -188,17 +188,18 @@ var AppHelpFlagGroups = []flagGroup{
 		}, debug.Flags...),
 	},
 	{
+		Name:  "Whisper (EXPERIMENTAL)",


Please make "Whisper" all uppercase. That's how everything else is.

bas-vk added the whisper label May 29, 2017

bas-vk force-pushed the whisper-api branch 11 times, most recently from 65aa33d to 7f00d42 Compare June 5, 2017 10:05

bas-vk force-pushed the whisper-api branch 3 times, most recently from 9371b12 to 86dfaa8 Compare June 9, 2017 08:19

bas-vk added the review label Jun 9, 2017

gluk256 approved these changes Jun 9, 2017

View reviewed changes

bas-vk force-pushed the whisper-api branch 2 times, most recently from 5276ad1 to 9625710 Compare June 13, 2017 09:49

bas-vk mentioned this pull request Jun 13, 2017

Add whisper RPC simulation ethereum/hive#55

Closed

holiman reviewed Jun 13, 2017

View reviewed changes

whisperv5: integrate whisper and add whisper RPC simulator

b58a501

bas-vk force-pushed the whisper-api branch from 9625710 to b58a501 Compare June 15, 2017 09:53

fjl requested changes Jun 21, 2017

View reviewed changes

bas-vk added 3 commits June 21, 2017 10:11

whisper: move ShhClient to its own package

36e7963

whisper: remove gencodec override for config

767dc6c

whisper: remove obsolete api tests

3d66ba5

bas-vk added 5 commits June 21, 2017 10:24

whisper: fallback to default config if none is specified

b6b0e00

whisper: renamed errors

4a1d516

whisper: move flags from whisper package to utils

7a11e86

whisper: use syncmap for dynamic configuration options

a4e4c76

whisper: use hexutil.UnmarshalFixedText for topic parsing

c62d542

fjl approved these changes Jun 22, 2017

View reviewed changes

whisper: renamed Info#Message to Info#Messages

b6e99de

karalabe reviewed Jun 26, 2017

View reviewed changes

cmd/geth: fix whisper flag group capitalization

ea1d182

karalabe merged commit feb2932 into ethereum:master Jun 26, 2017

karalabe added this to the 1.6.7 milestone Jun 26, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

whisperv5: integrate whisper and implement API #14540

whisperv5: integrate whisper and implement API #14540

bas-vk commented May 29, 2017 •

edited

Loading

gluk256 commented Jun 9, 2017

gluk256 left a comment

holiman Jun 13, 2017

bas-vk Jun 15, 2017

bas-vk commented Jun 15, 2017

fjl left a comment •

edited

Loading

fjl Jun 21, 2017

fjl Jun 21, 2017 •

edited

Loading

fjl Jun 21, 2017

fjl Jun 21, 2017

bas-vk Jun 21, 2017

fjl Jun 21, 2017

fjl Jun 21, 2017

fjl Jun 21, 2017

fjl Jun 21, 2017 •

edited

Loading

bas-vk Jun 21, 2017 •

edited

Loading

fjl Jun 21, 2017

fjl Jun 21, 2017

fjl commented Jun 21, 2017 •

edited

Loading

bas-vk commented Jun 21, 2017 •

edited

Loading

gluk256 commented Jun 21, 2017 •

edited

Loading

gluk256 commented Jun 21, 2017

frozeman commented Jun 22, 2017 •

edited

Loading

bas-vk commented Jun 23, 2017

frozeman commented Jun 23, 2017

frozeman commented Jun 23, 2017

frozeman commented Jun 23, 2017

karalabe Jun 26, 2017

whisperv5: integrate whisper and implement API #14540

whisperv5: integrate whisper and implement API #14540

Conversation

bas-vk commented May 29, 2017 • edited Loading

gluk256 commented Jun 9, 2017

gluk256 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bas-vk commented Jun 15, 2017

fjl left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fjl Jun 21, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fjl Jun 21, 2017 • edited Loading

Choose a reason for hiding this comment

bas-vk Jun 21, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fjl commented Jun 21, 2017 • edited Loading

bas-vk commented Jun 21, 2017 • edited Loading

gluk256 commented Jun 21, 2017 • edited Loading

gluk256 commented Jun 21, 2017

frozeman commented Jun 22, 2017 • edited Loading

bas-vk commented Jun 23, 2017

frozeman commented Jun 23, 2017

frozeman commented Jun 23, 2017

frozeman commented Jun 23, 2017

Choose a reason for hiding this comment

bas-vk commented May 29, 2017 •

edited

Loading

fjl left a comment •

edited

Loading

fjl Jun 21, 2017 •

edited

Loading

fjl Jun 21, 2017 •

edited

Loading

bas-vk Jun 21, 2017 •

edited

Loading

fjl commented Jun 21, 2017 •

edited

Loading

bas-vk commented Jun 21, 2017 •

edited

Loading

gluk256 commented Jun 21, 2017 •

edited

Loading

frozeman commented Jun 22, 2017 •

edited

Loading