Import external watch-only addresses #204

tuxcanfly · 2015-03-02T18:23:56Z

Added RPC commands to import external watch-only addresses both with and without pubkeys. WIP on finishing test coverage and docs.

Refs #192

davecgh · 2015-03-19T14:52:55Z

waddrmgr/db.go

@@ -549,14 +581,14 @@ func putNumAccounts(tx walletdb.Tx, numAccounts uint32) error {
 // the common parts.
 func deserializeAddressRow(addressID, serializedAddress []byte) (*dbAddressRow, error) {
 	// The serialized address format is:
-	//   <addrType><account><addedTime><syncStatus><rawdata>
+	//   <addrType><account><addedTime><syncStatus><watchingOnly><rawdata>


Why do we need a watching only flag as well as an adtWatching address type? It would be ideal to avoid changing this format because, if you do, you'll have to write upgrade code and modify every existing address in the database which I think we really don't want to do.

After thinking about it some more, I think having the extra field is more flexible. However, at the moment there is some redundancy with the adtWatching type that doesn't quite feel right.

If we look at it from the perspective of what a user will need to do for watching-only purposes:

Import BIP0032 public key chains

Import "loose" public keys (those which are not a part of BIP0032 key chain)

Import the hash160 associated with a pay-to-script-hash

Considering that, the first case is an adtChain with a watching-only flag, the second is an adtImport with a watching-only flag, and the third is an adtScript with a watching-only flag.

Updated to only use the watchingOnly flag and removed adtWatching type. Importing "loose" public keys and P2SH hashes works using the importpubkey and importaddress commands. Now working on importing public key chains.

Now working on importing BIP0032 public key chains.

davecgh · 2015-03-23T17:21:21Z

waddrmgr/address.go

-	internal         bool
-	compressed       bool
-	used             bool
+	publicAddress


What is the purpose of splitting this out?

I thought it might be logical to separate the structs for loose vs keychain addrs for future e.g if we wanted to add a method specific to loose addrs.

davecgh · 2015-03-23T17:28:01Z

Since the format of existing addresses in the db is being changed here, the db version needs to be bumped and it needs upgrade code to handle the new serialization. You should be able to do it without requiring the private passphrase by specifically targetting the new field offset.

tuxcanfly · 2015-03-23T19:35:56Z

Yeah, we will need a upgrade to initialize the watchingOnly field, I guess it can be inferred by checking that pubkey is not nil.

tuxcanfly · 2015-03-24T14:04:20Z

@davecgh Added upgrade path, addressed items reviewed above. Tested that we support importing pubkeys, p2pkh and p2sh addresses now.

davecgh · 2015-03-27T20:08:03Z

waddrmgr/db.go

@@ -902,35 +927,48 @@ func deserializeImportedAddress(row *dbAddressRow) (*dbImportedAddressRow, error
 		dbAddressRow: *row,
 	}

-	pubLen := binary.LittleEndian.Uint32(row.rawData[0:4])
+	pkHashLen := binary.LittleEndian.Uint32(row.rawData[0:4])


Please stick with the existing offset approach. There is less chance of error when updating the code with that approach.

Reminder to take care of this. I don't particularly care for offset variables, but to be consistent with the rest of the package, it should create the offset var first, and then slice between [offset:offset+fieldlen].

They are superior to 4+4+8+len(thing)+x+y+len(z.foo). And then each field has to repeat that from the last, etc.

EDIT: The recent conversation from the ppcd folks having to deal with all those magic offsets everywhere is a prime example of how they help as well.

With variable-sized data I agree

jrick · 2015-05-01T20:26:17Z

waddrmgr/address.go

@@ -62,11 +65,11 @@ type ManagedPubKeyAddress interface {
 	ManagedAddress

 	// PubKey returns the public key associated with the address.
-	PubKey() *btcec.PublicKey
+	PubKey() (*btcec.PublicKey, error)


I think we should change the signature not to introduce an error (for the not-exists case) but to return the public key and a boolean for whether the pubkey is compressed or not. Since watching addresses can be imported without the pubkey, not having the pubkey exist is now a totally valid state, and the caller must determine whether not existing is an error or not.

For the not-exists case, we would either need another (a second) bool return, or return a nil pubkey. I'm leaning towards the additional return to avoid situations where the caller forgets to check for the nil case, since the Go type system isn't strong enough to enforce this.

OK, we already have a db field called compressed on managedAddress which we are using for determining whether the pubkey is compressed or not. If we change the signature we might want to remove that field to avoid ambiguity.

Also, the PubKey signature is consistent with PrivKey which returns ErrWatchingOnly when private key is not available.

jrick · 2015-05-04T15:10:11Z

rpcserver.go

+	}
+
+	var compressed bool
+	switch n := len(pubKeyBytes); n {


I think this whole switch can be removed. The correct length is checked by ParsePubKey below. All you would need to pass to ImportPubKey is the bool len(pubKeyBytes) == btcec.PubKeyBytesLenCompressed.

I kind of think that the btcec type should save whether the pubkey was compressed or not, but oh well.

Thanks, removed the swtich and simplified this.

tuxcanfly · 2015-05-11T15:09:48Z

OK, updated PubKey to return compressed as well as ok. Refactored related methods accordingly.

jrick · 2015-05-14T18:42:58Z

The changed methods need better documentation to describe what all the return values are. Either the interface definition can use named returns to document the values, or comments should be added to describe what each bool means.

tuxcanfly · 2015-05-15T15:00:17Z

Thanks, using named returns in the interface and also updated docs about the return values.

In 6ee1f9b, Manager.Address was updated to convert PK addr to PKH addr so that the db is always keyed by script hash. This commit fixes ImportAddress to use the same key while fetching the addr from the db.

ghost · 2016-08-27T07:29:11Z

What's the status on this? The corresponding PRs btcsuite/btcd#279 and btcsuite/btcjson#41 were merged a long time ago.

Did #192 "supersede" this PR?

jrick · 2016-08-29T13:42:38Z

There are several conflicts preventing this from being merged. There will be even more conflicts if decred/dcrwallet#315 is backported to btcwallet.

We're also moving away from the idea of importing keys into a HD wallet, both to simplify our code and to make the entire wallet recoverable from just the seed. Would need a discussion about if we still even want this feature.

tuxcanfly force-pushed the import-address branch 3 times, most recently from 1a1eb59 to 7f91bd7 Compare March 7, 2015 13:00

davecgh mentioned this pull request Mar 11, 2015

Refactoring to create a 'wallet' package #213

Merged

tuxcanfly force-pushed the import-address branch from 7f91bd7 to cfbd1cb Compare March 12, 2015 18:30

davecgh reviewed Mar 19, 2015
View reviewed changes

tuxcanfly force-pushed the import-address branch from cfbd1cb to a04ef44 Compare March 19, 2015 18:07

davecgh mentioned this pull request Mar 20, 2015

Implement votingpool's startwithdrawal #178

Merged

tuxcanfly force-pushed the import-address branch from 3e647c4 to 7d2bf04 Compare March 20, 2015 19:34

davecgh reviewed Mar 23, 2015
View reviewed changes

tuxcanfly force-pushed the import-address branch 3 times, most recently from 348af4d to c634317 Compare March 23, 2015 18:52

tuxcanfly force-pushed the import-address branch 2 times, most recently from 3a3c1eb to b0e29ec Compare March 24, 2015 14:00

tuxcanfly force-pushed the import-address branch 3 times, most recently from 0d839f6 to 4f843cb Compare March 25, 2015 16:12

davecgh reviewed Mar 27, 2015
View reviewed changes

tuxcanfly force-pushed the import-address branch 3 times, most recently from 8ea8bed to 8847744 Compare May 1, 2015 20:07

jrick reviewed May 1, 2015
View reviewed changes

tuxcanfly force-pushed the import-address branch from 8847744 to b854a06 Compare May 2, 2015 03:19

jrick reviewed May 4, 2015
View reviewed changes

tuxcanfly force-pushed the import-address branch 2 times, most recently from 1019144 to 7b3e16f Compare May 5, 2015 19:07

tuxcanfly force-pushed the import-address branch 2 times, most recently from c1be9db to e670502 Compare May 14, 2015 08:12

tuxcanfly force-pushed the import-address branch from e670502 to 057597f Compare May 15, 2015 08:34

tuxcanfly force-pushed the import-address branch from 7a447bd to c8edcc6 Compare May 26, 2015 15:47

jrick mentioned this pull request May 28, 2015

Add spendable field to listunspent result. #288

Merged

tuxcanfly force-pushed the import-address branch from c8edcc6 to 3d6cc83 Compare August 24, 2015 12:07

tuxcanfly added 5 commits November 13, 2015 17:53

Added RPC cmds to import watching-only addresses

e657c53

Modify PubKey, ExportPubKey to return ok instead of err

b9119b9

Refactor PubKey related methods

dc40ca5

Updated PubKey to use named returns, add doc

cc6b4c9

Updated ImportAddress in accordance with 6ee1f9b

1444b51

In 6ee1f9b, Manager.Address was updated to convert PK addr to PKH addr so that the db is always keyed by script hash. This commit fixes ImportAddress to use the same key while fetching the addr from the db.

tuxcanfly force-pushed the import-address branch from ba7c06e to 1444b51 Compare November 13, 2015 12:23

Import external watch-only addresses #204

Import external watch-only addresses #204

tuxcanfly commented Mar 2, 2015

davecgh Mar 19, 2015

davecgh Mar 19, 2015

tuxcanfly Mar 23, 2015

davecgh Mar 23, 2015

tuxcanfly Mar 23, 2015

davecgh commented Mar 23, 2015

tuxcanfly commented Mar 23, 2015

tuxcanfly commented Mar 24, 2015

davecgh Mar 27, 2015

jrick May 4, 2015

davecgh May 4, 2015

jrick May 4, 2015

jrick May 1, 2015

tuxcanfly May 4, 2015

jrick May 4, 2015

tuxcanfly May 4, 2015

tuxcanfly commented May 11, 2015

jrick commented May 14, 2015

tuxcanfly commented May 15, 2015

ghost commented Aug 27, 2016 •

edited by ghost

jrick commented Aug 29, 2016 •

edited

Import external watch-only addresses #204

Are you sure you want to change the base?

Import external watch-only addresses #204

Conversation

tuxcanfly commented Mar 2, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

davecgh commented Mar 23, 2015

tuxcanfly commented Mar 23, 2015

tuxcanfly commented Mar 24, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tuxcanfly commented May 11, 2015

jrick commented May 14, 2015

tuxcanfly commented May 15, 2015

ghost commented Aug 27, 2016 • edited by ghost

jrick commented Aug 29, 2016 • edited

ghost commented Aug 27, 2016 •

edited by ghost

jrick commented Aug 29, 2016 •

edited