Blockchain.info API support

[lacksfish]

Fully functional blockchain.info API

How to use

Change the source in joinmarket.cfg to

blockchain_source = blockchain.info

Then, tumble as usual

Notes

The pushtx() function in the BlockchaininfoInterface class is functional but could need a quick review. I left my findings and questions in the comments.

The interface got an extra make_bci_request() function for directing requests to the API. Includes API error handling for common errors.

[AdamISZ]

I'm going to pull and try to test now. Is anyone else testing at all? @lacksfish what about you, could you give info on any tests you've done?

Note we now have a test_blockr.py; but I only put in a test for testnet there, still, in principle, it would be desirable for PR commiters to deliver tests with the code. I do appreciate that this is a particularly difficult one, but still.

[AdamISZ]

First test, OK - can run wallet-test against real wallets and get correct info, although I noted a difference of 19s for bc.i vs 4s for blockr.io, don't know how repeatable that is. I will try a sendpayment.

Second test: tried wallet-tool again with a big wallet (my main yg wallet), results not so good:
bc.i took 350 seconds, and didn't find all the coins (missed two addresses in mixdepth 2)
blockr took 37 and did find all coins.
I'm looking carefully at the delta, at first sight there looks to be a problem with it not searching far enough forwards on each branch. I notice that even for the other branches (that have correct amounts) it doesn't go as far forwards. Investigating.

The timing delta is partly due to timeouts as the bc.i version inserted several 60 second waits, so less concerned about that.

[AdamISZ]

Took a bit of burrowing, but I think I found the problem. The sync_addresses code implicitly assumes that the list of data returned is in the same order as the address list passed to the API, but it isn't for bci it seems. See here: https://github.com/JoinMarket-Org/joinmarket/pull/481/files#diff-3b408ed85f23411e81db7c0ae451edfdR188 (it's the same in the blockr code of course).

Continuing to do tests; if I got that right, it'll be fiddly but it can be fixed.

[AdamISZ]

Yes, that solved it. @lacksfish you can add insert here this code:

new_data = []
for a in addrs:
    new_data.append([d for d in data if d['address']==a][0])

and then the next line obviously becomes:

for addr in new_data:

With this edit the balance is reported correctly. Without it you will get something unpredictable because the ending value of last_used_addr in each loop iteration depends on the ordering of the list that bci delivers (I don't know if it's random or sorted somehow).

Edit: of course some extra sanity check might make sense there, also there might be a cleverer way to re-sort the list without looping, I just couldn't think of one offhand.

[lacksfish]

Sorry for the late replay, am quite busy at the moment. I will look into it in the coming two weeks.

[coveralls]

Coverage decreased (-4.5%) to 73.555% when pulling 9cff309 on lacksfish:master into c2ec1dd on JoinMarket-Org:develop.

[lacksfish]

AdamISZ, added your solution and also updated my repository.

[coveralls]

Coverage decreased (-4.5%) to 73.595% when pulling 051329c on lacksfish:master into c2ec1dd on JoinMarket-Org:develop.

[AdamISZ]

Seems to have picked up a bunch of old stuff, something's gone wrong. By the way I notice a lot of it is joinmarket-qt stuff which shouldn't be here (it's supposed to be only in the gui branch).

[coveralls]

Coverage decreased (-4.4%) to 73.6% when pulling cfff7f0 on lacksfish:master into aba76d5 on JoinMarket-Org:develop.

[coveralls]

Coverage decreased (-4.5%) to 73.56% when pulling cfff7f0 on lacksfish:master into aba76d5 on JoinMarket-Org:develop.

[lacksfish]

I'd be very thankful for someone testing this updated version of the API Interface, I implemented AdamISZ's fix for properly sorting the data array in sync_addresses()

[AdamISZ]

@lacksfish did you see my earlier comment? This needs to be rebased somehow.

[AdamISZ]

I've added this (although had to fix up the bugfix above, didn't seem to have been done properly) in fa61dbd ; of course it would be much better to give proper attribution but in the absence of cleaning up this PR I just went ahead.

Apart from that attribution issue, the other issue is that it's dreadfully slow (even not on Tor, but of course especially on Tor as you regularly get endpoint limit reached and waits). I see it as something to use in extremis when nothing else is available.

Closing; @lacksfish let me know if you want to fix this up somehow so you can get attribution.