Insight API broadcast_tx [17]

[papacarp]

This PR is ready for testing on the broadcast Tx endpoint
[17] send_tx

Also a new standard API failure packet was included as described in PR #479.

Valid Successful Response

{
    "rawtx": "18a4eeed058c2266512863d03f79651cd38d94d6d682ae3f1e4aad0178c6998f"
}

Failure Response

{
    "status": 404,
    "url": "/insight-api/tx/send",
    "error": "SendRawTransaction failed: -1: Rejected transaction 18a4eeed058c2266512863d03f79651cd38d94d6d682ae3f1e4aad0178c6998f: orphan transaction 18a4eeed058c2266512863d03f79651cd38d94d6d682ae3f1e4aad0178c6998f references outputs of unknown or fully-spent transaction 1d3fe2879c60c040fc5b3426aa305b29565aa27035726eaccbb4970b6971de84"
}

Creating a raw transaction for testing

Below is a powershell script for creating a new raw transaction using dcrctl. YMMV.

$unspentobj = ./dcrctl.exe --testnet /wallet listunspent | ConvertFrom-Json

$unspent = $unspentobj | Select-Object -Property * -ExcludeProperty ("txtype","address","account","scriptPubKey","amount","confirmations","spendable") | ConvertTo-Json -Compress

$unspent = "[" + $unspent.replace('"','\"') + "]"

$sendto = '{\"TsWjioPrP8E1TuTMmTrVMM2BA4iPrjQXBpR\":0.1,\"Tsmsr99vrJTxMMtnhmLT3okH6UJji8g5KVU\":7.4}'

$rawtransaction = ./dcrctl.exe --testnet /wallet createrawtransaction $unspent $sendto

$rawsignedtx = ./dcrctl.exe --testnet /wallet signrawtransaction $rawtransaction| ConvertFrom-Json

$rawsignedtx."hex"

[chappjc]

This one is more of a 400 Bad Request: https://httpstatuses.com/400

[chappjc]

Let's check that rawHexTx does not exceed the max allowable size.

[chappjc]

This one would be a 422 (unprocessable entity), but the message is OK.

[chappjc]

We'll also want to have a clear plan for rate limiting on the insight API worked out. There are plenty of middlewares out there that should fit the bill.

[papacarp]

PR has been rebased and updated based on review comments. It is recommended we merge this as our lead Insight API PR and I can rebase all the other Insight API PRs accordingly.

1. Error messages have been updated to a standard strategy we will use across all Insight API. Dialog and recommendations are shown below.
2. RawTx post extraction moved to middleware.
3. Rate limiter added. Rate was set to 1 request/second. See tests below:

Error Message Recommendation
I have analyzed most of the responses from bitpay and added them to my spreadsheet I'm using to track differences. There are some inconsistencies, but the following rules seem to apply:

200 OK
400 Bad Request - checksum mismatches, invalid addresses/block hash, etc, outside of range expected for block numbers
404 Not Found - blank data, or valid request, but no data found for address or tx or block hash, etc.
503 Service Unavailable - if there is no block or address param posted to txs endpoint.  i think this is a bug

The responses are usually content type: html/text with a text body describing error. The exception is for url data that is not present the endpoint will respond with 404, json with a packet including an error message (such as: insight.bitpay.com/api/block/)

Our existing mainnet API has some inconsistencies as well.

200 OK
400 Bad Request - unknown, or invalid blocks, invliad tx (not hex for example)
404 Not Found - not found tx, 
500 Internal Server Error - unknown blocks for txs by block or txs by address

Recommendation:

1. We should regex and checksum rulecheck all addresses, block hash, txs, block indexes, rawtx values. Any API call with invalid values will result in a "400 Bad Request" with html/text content-type and a text description of error.
2. For all calls that are valid if we are unable to find that tx or block hash then it will result in a 404 Not Found response with html/text content-type and the text "Not found". Note, an address with no transactions will NOT be considered 404 as it may just not have transactions yet.
3. All other responses are 200 OK, JSON content type and JSON packet. single value responses like /balance are considered valid JSON as well and will also have JSON content type.

Rate Limit Testing
Using postman I sent 20 transactions to dcrdata as fast as possible. Rate limiter is acting as intended.

[chappjc]

Nice work. I'll try to get this in tonight

[papacarp]

Great! I'll rebase and re-align the other Insight PR's later today.