Signrawtransaction can throw "TX decode failed" even if "decoderawtransaction" doesn't

[Sjors]

Perpaps I'm using the wrong syntax for signrawtransaction, but otherwise I might have found a bug.

For example this unsigned transaction gets decoded successfully (remove the line-breaks):

curl --user USERNAME --data-binary '{"id":"t0",
"method": "decoderawtransaction",
"params": 
["0100000002d354be7cd5426bfbe70517b934b8764d17ad7f93e2b8db868211851d4404740f010000001976a91423376070c7b24da64b435c71613053800494ab1c88acffffffffe49a65da5abe3edd6e5157327fe794a7c75befecaaf18fefd3154dbb4527d6d6010000001976a91423376070c7b24da64b435c71613053800494ab1c88acffffffff0240420f00000000001976a914c8a73488183dd49f63a11dea0a3b242ae70942d288ac10ae2201000000001976a91423376070c7b24da64b435c71613053800494ab1c88ac0000000001000000"] 
}' http://127.0.0.1:8332/

But when I try to sign it, it throws an exception:

curl --user USERNAME --data-binary '{"id":"t0",
"method": "signrawtransaction",
"params": 
[
 "0100000002d354be7cd5426bfbe70517b934b8764d17ad7f93e2b8db868211851d4404740f010000001976a91423376070c7b24da64b435c71613053800494ab1c88acffffffffe49a65da5abe3edd6e5157327fe794a7c75befecaaf18fefd3154dbb4527d6d6010000001976a91423376070c7b24da64b435c71613053800494ab1c88acffffffff0240420f00000000001976a914c8a73488183dd49f63a11dea0a3b242ae70942d288ac10ae2201000000001976a91423376070c7b24da64b435c71613053800494ab1c88ac0000000001000000",
  [],
  []
] 
}' http://127.0.0.1:8332/

{"result":null,"error":{"code":-22,"message":"TX decode failed"},"id":"t0"}

Now there might be something wrong with my transaction, but I would except signrawtransaction to at least throw a different error than decoderawtransaction if the latter successfully decodes it.

Looking at the source code, the decode method does this and throws the "TX decode failed" message if it doesn't work:

CTransaction tx;
    try {
        ssData >> tx;
    }
    catch (std::exception &e) {
        throw JSONRPCError(RPC_DESERIALIZATION_ERROR, "TX decode failed");

The decode method is similar but performs one extra step. My guess is that that is where the error occurs in my case, but I haven't tested this.

vector<CTransaction> txVariants;
   while (!ssData.empty())
   {
       try {
           CTransaction tx;
           ssData >> tx;
           txVariants.push_back(tx);

[Sjors]

Another transaction where this happens is the example transaction here. I get this error even if I include the private key and previous transaction id:

curl --user USERNAME --data-binary '{"id":"t0", "method": "signrawtransaction", "params": [
"0100000001eccf7e3034189b851985d871f91384b8ee357cd47c3024736e5676eb2debb3f2010000001976a914010966776006953d5567439e5e39f86a0d273bee88acffffffff01605af405000000001976a914097072524438d003d23a2f23edb65aae1bb3e46988ac0000000001000000", 
["f2b3eb2deb76566e7324307cd47c35eeb88413f971d88519859b1834307ecfec"], 
["2g82vgrZTviKG5sN1g2VM7FHgHTm16ej4gmr8ECMzab6"]
]}' http://127.0.0.1:8332/

Again, decoderawtransaction has no issues with this transaction.

[gavinandresen]

I added this code to decoderawtransaction:

if (!ssData.empty()) {
     fprintf(stderr, "Extra data in ssData: %s\n",
            HexStr(ssData.begin(), ssData.end()).c_str());
}

... and get:
Extra data in ssData: 01000000

Instead of ignoring extra bytes, decoderawtransaction should at the very least warn about them.

[Sjors]

Legend! That's extremely helpful. Now I'm able to sign my own transaction. The sample transaction results in an "invalid private key" message, but I can live with that.

I'm a bit puzzled about why these last 4 bytes aren't allowed. Is step 13 in this tutorial wrong? Or does the QT client add them automatically before it signs the transaction?

[gavinandresen]

Don't know why that tutorial says to add a hash code to the transaction, I think it is wrong (but their python code is correct). See https://en.bitcoin.it/wiki/Transaction for the format.

[Sjors]

Not the final transaction, but the intermediate form that is signed. I'm afraid I'll have to dive in the client C code to figure out what exactly is going on. My goal is to do the signing in ruby:

https://bitcointalk.org/index.php?topic=202271.0

[sipa]

@Sjors I think you're confusing the transaction itself with its intermediate form used for signing. Among many other modifications, the hashtype is indeed appended to a transaction before computing the hash being signed, but this hashtype field is not part of the transaction itself. There are other modifications made, like clearing the other inputs of a transaction, and replacing the input script being signed with the output script being redeemed.

Just to be clear, sendrawtransaction/createrawtransaction/... take transactions. The modifications made to compute the hash being signed are done on-the-fly when necessary (they are different for the different inputs, for example, so you can't provide a single intermediate form anyway).