Auction info not showing all bids

[randomlogin]

Reviewing this: #553
I've encountered situation where auction related info was different for different wallets, for example getbids or getauctioninfo.
One wallet does not shows actions from other wallets.
Reproduce script: https://gist.github.com/randomlogin/d110b89a55152551a99ba459e3f8896d

[pinheadmz]

This is expected behavior. HSD only watches for a name after its first participation in an auction.

If Alice opens an auction, her wallet will collect all the bids from now on.
Let's say then Alice places a bid.
If Bob then bids, his wallet only starts watching name when he places his first bid. He will not know about Alice's bid.
Then Charlie bids.
Alice, Bob and Charlie will all know about Charlie's bid. But Charlie will not know about Alice and Bob's bids.

This is precisely why rpc importname was added: #408

If you add an importname+ rescan command between lines 22 and 23 of your bash script, I think you'll see it work.

[Anunayj]

I can reproduce this on master branch, this is the output i got:

checking from the first wallet

null
[
  {
    "name": "sense",
    "nameHash": "cd9a545509d93b62b8d6c63d979080ab58aeb57129092636de22ec6d58246071",
    "prevout": {
      "hash": "a0888fde7012eb0d4c28fe104469b35f402e5b904a32226659208151f6e59e89",
      "index": 0
    },
    "lockup": 21000000,
    "blind": "4205043b373e719ef5986219646b00384a248929658d16334315b1515d373fef",
    "own": false
  },
  {
    "name": "sense",
    "nameHash": "cd9a545509d93b62b8d6c63d979080ab58aeb57129092636de22ec6d58246071",
    "prevout": {
      "hash": "de83b833e1abc13289595da2ce24c6183c15021c2c03196f6c39b7a47a5fd96b",
      "index": 0
    },
    "value": 1000000,
    "lockup": 1000000,
    "blind": "ed158e12dce393cd57f60ebafdc4f29e50d7c7ee435a8f4f2580047cc297e806",
    "own": true
  }
]


checking from the second wallet

null
[
  {
    "name": "sense",
    "nameHash": "cd9a545509d93b62b8d6c63d979080ab58aeb57129092636de22ec6d58246071",
    "prevout": {
      "hash": "a0888fde7012eb0d4c28fe104469b35f402e5b904a32226659208151f6e59e89",
      "index": 0
    },
    "value": 20000000,
    "lockup": 21000000,
    "blind": "4205043b373e719ef5986219646b00384a248929658d16334315b1515d373fef",
    "own": true
  }

]

The second Wallet seems to be oblivious of the first bid, rescanning didn't help.
The weird thing is even tho getbids does not show the bid (even after i reveal from other wallet), getreveals does show the reveal.

[randomlogin]

I cannot agree it's expected behaviour for the user.
If a user makes a bid, he should be aware of all bids, even done before him.
I'm for changing how it works.

[Anunayj]

This is what I get on importname

$ hsw-cli --network=regtest rpc importname sense 2050 
Name already exists.

Rescanning didn't show the output either

$ hsw-cli --network=regtest --id=asdfg rescan 0
Rescanning...

$ hsw-cli --network=regtest rpc getbids sense
[]

$ hsw-cli  --network=regtest rpc getbids sense                 
[
  {
    "name": "sense",
    "nameHash": "cd9a545509d93b62b8d6c63d979080ab58aeb57129092636de22ec6d58246071",
    "prevout": {
      "hash": "a0888fde7012eb0d4c28fe104469b35f402e5b904a32226659208151f6e59e89",
      "index": 0
    },
    "value": 20000000,
    "lockup": 21000000,
    "blind": "4205043b373e719ef5986219646b00384a248929658d16334315b1515d373fef",
    "own": true
  }
]

[Anunayj]

Meanwhile getreveals does work:

$ hsw-cli --network=regtest rpc getreveals sense
[
  {
    "name": "sense",
    "nameHash": "cd9a545509d93b62b8d6c63d979080ab58aeb57129092636de22ec6d58246071",
    "prevout": {
      "hash": "477b7b24c94acc4f8747c039ddf1915d22e7b02dbfde1ac3c7e40203c31083be",
      "index": 0
    },
    "value": 1000000,
    "height": 2065,
    "own": false
  },
  {
    "name": "sense",
    "nameHash": "cd9a545509d93b62b8d6c63d979080ab58aeb57129092636de22ec6d58246071",
    "prevout": {
      "hash": "db4ef401db44fb032c8e2ad869a844ca8396f1cbfb3f7b97f824a646dca74f1b",
      "index": 0
    },
    "value": 20000000,
    "height": 2064,
    "own": true
  }
]

[Anunayj]

I cannot agree it's expected behaviour for the user.
If a user makes a bid, he should be aware of all bids, even done before him.
I'm for changing how it works.

Well as of right now re-scanning is supposed to rescan for all other bids, which makes sense to me. Though I have no idea if I did something wrong, but re-scanning didn't help the situation for some reason?

[pinheadmz]

@Anunayj getreveals works because those reveals were broadcast after the wallet started to "watch" the name.

The wallet does not index every transaction for every name all the time, it has an internal watchlist. A name is only added to that wacthlist when the wallet itself sends an auction-related transaction.

The only way to know what bids happened in the past is to rescan -- but a simple rescan won't work because a rescan starts with the wallet reverting blocks and rewinding history, then playing it forward again. When you rewind history, you revert the moment where the wallet started watching a name! So as you rescan forwards again, you continue to ignore bids until the wallet itself participates in the auction. In #499 I have a commit that may change this behavior, but a rescan will still be required.

Again, this is precisely why rpc importname was added, and a button was added to Bob Wallet to "rescan auction". Because of the wallet architecture, an importname action MUST be executed before the wallet's first bid or it won't work.

Because a rescan is a disruptive action, it is not automatic. Applications like Namebase and hnsnetwork.com probably index all auction activity for all names all the time. A user interested in managing all that extra data can have the user experience you're discussing, but I don't think that should be the default for hsd.

[Anunayj]

I see, though is there a way to fix it if i have done that? bid before importing name.

[pinheadmz]

I modified your script a bit, here's basically a refactor showing your initial UX:

export HSD_NETWORK=regtest

#create 2 wallets, populate them with some hns
hsw-cli mkwallet qwerty
hsw-cli mkwallet asdfg

hsw-rpc selectwallet qwerty
hsd-rpc generatetoaddress 200 `hsw-rpc getnewaddress`
hsw-rpc selectwallet asdfg
hsd-rpc generatetoaddress 200 `hsw-rpc getnewaddress`


#make bids
hsw-rpc selectwallet qwerty
hsw-rpc sendopen sad
hsd-rpc generatetoaddress 6 `hsw-rpc getnewaddress`
hsw-rpc sendbid sad 1 1
hsd-rpc generatetoaddress 1 `hsw-rpc getnewaddress`

hsw-rpc selectwallet asdfg
hsw-rpc sendbid sad 2 2
hsd-rpc generatetoaddress 10 `hsw-rpc getnewaddress`

#view the bids from different wallets
printf "\n\nchecking from the first wallet\n\n"
hsw-rpc selectwallet qwerty 
hsw-rpc getbids sad 
printf "\n\nchecking from the second wallet\n\n"
hsw-rpc selectwallet asdfg 
hsw-rpc getbids sad 

...and here is modified with importname:

export HSD_NETWORK=regtest

#create 2 wallets, populate them with some hns
hsw-cli mkwallet qwerty
hsw-cli mkwallet asdfg

hsw-rpc selectwallet qwerty
hsd-rpc generatetoaddress 200 `hsw-rpc getnewaddress`
hsw-rpc selectwallet asdfg
hsd-rpc generatetoaddress 200 `hsw-rpc getnewaddress`


#make bids
hsw-rpc selectwallet qwerty
hsw-rpc sendopen sad
hsd-rpc generatetoaddress 6 `hsw-rpc getnewaddress`
hsw-rpc sendbid sad 1 1
hsd-rpc generatetoaddress 1 `hsw-rpc getnewaddress`

hsw-rpc selectwallet asdfg

hsw-rpc importname sad
hsw-cli rescan
sleep 2

hsw-rpc sendbid sad 2 2
hsd-rpc generatetoaddress 10 `hsw-rpc getnewaddress`

#view the bids from different wallets
printf "\n\nchecking from the first wallet\n\n"
hsw-rpc selectwallet qwerty 
hsw-rpc getbids sad 
printf "\n\nchecking from the second wallet\n\n"
hsw-rpc selectwallet asdfg 
hsw-rpc getbids sad 

output:

checking from the first wallet

null
[
  {
    "name": "sad",
    "nameHash": "8f1071d276171eb4d91816405718eabecfb11067794f1f402146f4983c130218",
    "prevout": {
      "hash": "189910c5032eab81d138c2c2aead866d5f1af820889d2ec7bf7a54fd8efc5888",
      "index": 0
    },
    "lockup": 2000000,
    "blind": "0595e9cbe22e1458a200c6f719c8fa0939496ed61fc4793c4d1b0ab70e10e79c",
    "own": false
  },
  {
    "name": "sad",
    "nameHash": "8f1071d276171eb4d91816405718eabecfb11067794f1f402146f4983c130218",
    "prevout": {
      "hash": "d8b858385b8be89361a480d8fe38e77249d3fcb0a97d4fc787085bb784fc7b14",
      "index": 0
    },
    "value": 1000000,
    "lockup": 1000000,
    "blind": "fdf16449c9971f97436df3232b064246cae8fbcc53a16d52562794a80d841cba",
    "own": true
  }
]


checking from the second wallet

null
[
  {
    "name": "sad",
    "nameHash": "8f1071d276171eb4d91816405718eabecfb11067794f1f402146f4983c130218",
    "prevout": {
      "hash": "189910c5032eab81d138c2c2aead866d5f1af820889d2ec7bf7a54fd8efc5888",
      "index": 0
    },
    "value": 2000000,
    "lockup": 2000000,
    "blind": "0595e9cbe22e1458a200c6f719c8fa0939496ed61fc4793c4d1b0ab70e10e79c",
    "own": true
  },
  {
    "name": "sad",
    "nameHash": "8f1071d276171eb4d91816405718eabecfb11067794f1f402146f4983c130218",
    "prevout": {
      "hash": "d8b858385b8be89361a480d8fe38e77249d3fcb0a97d4fc787085bb784fc7b14",
      "index": 0
    },
    "lockup": 1000000,
    "blind": "fdf16449c9971f97436df3232b064246cae8fbcc53a16d52562794a80d841cba",
    "own": false
  }
]

[pinheadmz]

I see, though is there a way to fix it if i have done that? bid before importing name.

This might be worth looking in to, perhaps the commit I linked to from the other PR would allow this. Or the original logic of importname can be updated.

[Anunayj]

ig changing importname would be a a good idea? Do a rescan if a height is given?

[pinheadmz]

We want to avoid automatically rescanning I think, so for example in the Bob Wallet implementation it's still two commands but the user only clicks one button. It only needs to rescan a few days work of blocks also, so it's not a huge drag.

I think it is worth looking in to enabling importname to work even after the user submits a late bid. I think the commit I referenced above might be the only thing needed, and then the "name already exists" check can probably be removed. Please experiment with this if you have time!

[Anunayj]

Ok just checked, PR #499 indeed does fix this issue, doing a rescan shows the second bid. Requiring a rescan for seeing old bids makes sense too.
I think the "name already exists" check should stay since importname should only do what it says, add a name to the watchlist. and rescan only if the rescan height is given. It would make sense to rescan manually, if you already imported a name without rescanning back then.

[pinheadmz]

closed by #604