Support Request: Need help migrating from old Txb to Psbt

[JeffWScott]

Deprecation Warning: TransactionBuilder will be removed in the future. (v6.x.x or later) Please use the Psbt class instead. Examples of usage are available in the transactions-psbt.js integration test file on our Github. A high level explanation is available in the psbt.ts and psbt.js files as well.

So I got this warning.
I currently build transactions this way.

const tx = bitcoin.Transaction.fromHex(rawTransaction);
const txb = bitcoin.TransactionBuilder.fromTransaction(tx, networkObj)

I don't see a way to build a psbt from a rawTransaction. Is this being phased out?

[junderw]

Is this being phased out?

Yes.

• There shouldn't really be a use case for importing a completed raw transaction into PSBT, since you can't sign a completed transaction.
• For partially signed / unsigned transactions, the flow of use TXB to make TX -> pass to next signer and import into TXB via TX -> sign -> buildIncomplete() and toHex(), will change to use PSBT to make the transaction -> toBase64() -> fromBase64() -> sign -> toBase64() ... etc.

If you can tell me a good use case for allowing importing of Transaction objects to create the base for a PSBT, let me know.

Thanks.

Examples are here: (Tests were migrated to TypeScript)

bitcoinjs-lib/test/integration/transactions.spec.ts

Lines 75 to 160 in 41bf2cd

	it('can create (and broadcast via 3PBP) a typical Transaction', async () => {
	// these are { payment: Payment; keys: ECPair[] }
	const alice1 = createPayment('p2pkh');
	const alice2 = createPayment('p2pkh');
	// give Alice 2 unspent outputs
	const inputData1 = await getInputData(
	5e4,
	alice1.payment,
	false,
	'noredeem',
	);
	const inputData2 = await getInputData(
	7e4,
	alice2.payment,
	false,
	'noredeem',
	);
	{
	const {
	hash, // string of txid or Buffer of tx hash. (txid and hash are reverse order)
	index, // the output index of the txo you are spending
	nonWitnessUtxo, // the full previous transaction as a Buffer
	} = inputData1;
	assert.deepStrictEqual({ hash, index, nonWitnessUtxo }, inputData1);
	}
	// network is only needed if you pass an address to addOutput
	// using script (Buffer of scriptPubkey) instead will avoid needed network.
	const psbt = new bitcoin.Psbt({ network: regtest })
	.addInput(inputData1) // alice1 unspent
	.addInput(inputData2) // alice2 unspent
	.addOutput({
	address: 'mwCwTceJvYV27KXBc3NJZys6CjsgsoeHmf',
	value: 8e4,
	}) // the actual "spend"
	.addOutput({
	address: alice2.payment.address, // OR script, which is a Buffer.
	value: 1e4,
	}); // Alice's change
	// (in)(5e4 + 7e4) - (out)(8e4 + 1e4) = (fee)3e4 = 30000, this is the miner fee
	// Let's show a new feature with PSBT.
	// We can have multiple signers sign in parrallel and combine them.
	// (this is not necessary, but a nice feature)
	// encode to send out to the signers
	const psbtBaseText = psbt.toBase64();
	// each signer imports
	const signer1 = bitcoin.Psbt.fromBase64(psbtBaseText);
	const signer2 = bitcoin.Psbt.fromBase64(psbtBaseText);
	// Alice signs each input with the respective private keys
	// signInput and signInputAsync are better
	// (They take the input index explicitly as the first arg)
	signer1.signAllInputs(alice1.keys[0]);
	signer2.signAllInputs(alice2.keys[0]);
	// If your signer object's sign method returns a promise, use the following
	// await signer2.signAllInputsAsync(alice2.keys[0])
	// encode to send back to combiner (signer 1 and 2 are not near each other)
	const s1text = signer1.toBase64();
	const s2text = signer2.toBase64();
	const final1 = bitcoin.Psbt.fromBase64(s1text);
	const final2 = bitcoin.Psbt.fromBase64(s2text);
	// final1.combine(final2) would give the exact same result
	psbt.combine(final1, final2);
	// Finalizer wants to check all signatures are valid before finalizing.
	// If the finalizer wants to check for specific pubkeys, the second arg
	// can be passed. See the first multisig example below.
	assert.strictEqual(psbt.validateSignaturesOfInput(0), true);
	assert.strictEqual(psbt.validateSignaturesOfInput(1), true);
	// This step it new. Since we separate the signing operation and
	// the creation of the scriptSig and witness stack, we are able to
	psbt.finalizeAllInputs();
	// build and broadcast our RegTest network
	await regtestUtils.broadcast(psbt.extractTransaction().toHex());
	// to build and broadcast to the actual Bitcoin network, see https://github.com/bitcoinjs/bitcoinjs-lib/issues/839
	});

[junderw]

There is a hacky workaround. It would involve creating a BIP174 object and passing it an ITransaction interface, then passing that BIP174 object to bitcoinjs PSBT constructor.

but tbh migrating to PSBT is not a 1:1 migration and requires understanding PSBT and changing the transaction signing flow to accommodate for it.

If Transaction + TransactionBuilder is a nokia feature phone, PSBT is like the first iPhone. It takes a bit of getting used to. But it is extremely expandable.

[junderw]

hackey workaround, but it will wipe out your input scriptSigs and witnesses, and you will need to run updateInput on each input with their required info for signing.

Since there's a limit to the info a Transaction can hold as compared to a PSBT, it might be less work to rework everything to use PSBT rather than trying to retrofit PSBT onto a raw transaction based API. (If you'd like help on how to do this you can reach me on keybase)

const bitcoin = require('bitcoinjs-lib');
const Transaction = bitcoin.Transaction;
const Psbt = bitcoin.Psbt;

// from bip174
const PsbtBase = require('bip174').Psbt;

// needed to create bip174 object
class PsbtTransaction {
  constructor(buffer = Buffer.from([2, 0, 0, 0, 0, 0, 0, 0, 0, 0])) {
    this.tx = Transaction.fromBuffer(buffer);
    this.tx.ins.forEach(input => {
      input.script = Buffer.allocUnsafe(0);
      input.witness = [];
    });
    Object.defineProperty(this, 'tx', {
      enumerable: false,
      writable: true,
    });
  }

  getInputOutputCounts() {
    return {
      inputCount: this.tx.ins.length,
      outputCount: this.tx.outs.length,
    };
  }

  addInput(input) {
    if (
      input.hash === undefined ||
      input.index === undefined ||
      (!Buffer.isBuffer(input.hash) &&
        typeof input.hash !== 'string') ||
      typeof input.index !== 'number'
    ) {
      throw new Error('Error adding input.');
    }
    const hash =
      typeof input.hash === 'string'
        ? Buffer.from(input.hash, 'hex').reverse()
        : input.hash;
    this.tx.addInput(hash, input.index, input.sequence);
  }

  addOutput(output) {
    if (
      output.script === undefined ||
      output.value === undefined ||
      !Buffer.isBuffer(output.script) ||
      typeof output.value !== 'number'
    ) {
      throw new Error('Error adding output.');
    }
    this.tx.addOutput(output.script, output.value);
  }

  toBuffer(): Buffer {
    return this.tx.toBuffer();
  }
}

const txBuffer = Buffer.from(rawTransaction, 'hex');
const psbt = new Psbt({}, new PsbtBase(new PsbtTransaction(txBuffer)));

[JeffWScott]

@junderw

If you can tell me a good use case for allowing importing of Transaction objects to create the base for a PSBT, let me know.

So the scenario is that their is another system that creates the transaction as REST API (simple P2PKH transactions). It does it by feeding some endpoint data to a backend python service that does the "change addressing and fees" and UTXO discovery using APIs. It then creates a raw transaction with that information and sends it back to the app. The app holds the user's private keys so it signs the transaction and then sends it back to the API server which publishes the transaction (because it has all the connections to do so). The REST API supports other networks as well, like ethereum for example. So you can just send "Coin, To, From and Value" to the API service and it will return to you the raw transaction that needs to be signed (for ETH that includes all the gas and nonce information for example). The app signs it and posts it to the "publish" endpoint (like "Coin Name, Signed_TX").

With ETH it's as simple as the app just preforming the following:

- var tx = etheruemTX(raw_transaction)
- tx.sign(privatekey)
- Post to API

We have the app working currently for bitcoin as well, but on a WAY older version of bitcoinjs-lib. I'm trying to upgrade the package currently. I took the project over from someone who was previously using txb to accomplish the signing of the transaction, but because the bitcoinjs-lib version is so old even that TXB logic doesn't work on the latest version. TXB being depreciated means I'm not about to redo the TXB logic, only to be in the same boat next time we upgrade dependencies. So it PSBT is the way forward, than so be it.

That said, the raw bitcoin transaction string DOES have all the inputs and outputs and scripts (simple P2PKH transactions) needed to make the transaction. So I believe starting from the raw transaction is still an appropriate place for us. No need to redo all the work already done.

If I can't create a PSBT directly from raw_tx then maybe I need to create the PSBT from scratch. I could create a TX object with the raw_transaction and then just iterate through that object to build a PSBT. Once I have the PSBT built, validated and signed I can get the transaction hex and send it back to the API server to publish.

Does that seem like a logical conclusion?

You have posted a great deal of info. I appreciate the help. I'll dig in and see if I can make head or tails of it.

[junderw]

ok, so let me double check your constraints:

1. All related scripts are P2PKH.
2. API sends you an unsigned raw transaction hex string.
3. Client sends back signed raw transaction hex string.

In order to sign Psbts for P2PKH, you not only need the hash (txid) and index (vout) of the output being spent, but you also need the entire raw transaction that contains the output you are spending.

If you switch the script to segwit based (ie. P2SH-P2WPKH etc) all you need in addition to what you already have is the exact satoshi amount of each input. (Once you have that, all you need to do is create the output for your script using your publicKey (which your signing app should have, obviously, it has the privateKey) and the amount to create a witnessUtxo object to pass to the Psbt input.

Either way, you will need more information than the API is currently giving you.

To be honest, TransactionBuilder is pretty insecure. Reason being that there is actually no checks that your input value is what you expect, and if the API is compromised, for instance a miner could add an extremely large input so that your miner fee will grow extremely large... since TransactionBuilder doesn't require amounts for inputs, and even if you pass it, it has no way to verify.

Psbt changes that, by requiring the entire raw transaction you are pointing to, the Psbt class checks the hash of the raw transaction that it matches the hash in the input, then checks the output pointed to by the index and verifies the actual value in the transaction, preventing such attacks.

For segwit, since you sign the input value, it only needs the value and the scriptPubkey (called witnessUtxo in the API) to properly sign with Psbt.

By increasing security, we increased the burden on unsigned transaction creators in exchange for giving signers enough info to make sure they are signing what they believe they are signing.

So possible ways forward:

1. API supports PSBT, includes the raw transactions of the spent outputs for each input, and sends the client a psbt. signer logic is 3 lines max.
2. API doesn't support PSBT, BUT somehow returns an array of raw transactions for each input alongside the raw transaction. signing logic is maybe 10-20 lines of actual procedures and maybe a new class (like my example) which after creating the Psbt, will call updateInput with the nonWitnessUtxo (the raw transaction of the spent output). Then signing is a 1-2 line ordeal.
3. Modify Psbt to be less secure and not require the spent output's transaction. Which no one really wants.

Ethereum had the blessing of hindsight, luckily, and not being UTXO-based.

IMO Satoshi should have made the raw transaction serialization contain input amounts and sign them and should have separated the witnesses from the transaction hash through some sort of normalization, removing malleability... then P2PKH wouldn't need the whole transaction.

But here we are. lol.

If you need any specific help and want to share code you're not comfortable sharing publicly let me know on keybase.

[JeffWScott]

@junderw Thanks for the detailed response. I did mess around with it today and came to the same conclusion. I was thinking I might do #2. I'll give it another go next week. If I run into problems I'll hit you up on keybase. Thanks again.

[junderw]

If you have any more problems reopen this issue. Cheers!

[3s3s]

Hello
Can you help me to migrate this code to Psbt? This is case for custom redeem script (for example OP_DROP OP_DROP OP_CHECKSIG)

async function send(keyPair, txb, redeemScript)
{
  const tx = txb.buildIncomplete();

  const signatureHash = IsWitness() ?
      tx.hashForWitnessV0(0, redeemScript, 1, bitcoin.Transaction.SIGHASH_ALL) :
      tx.hashForSignature(0, redeemScript, bitcoin.Transaction.SIGHASH_ALL);

  const signature = bitcoin.script.signature.encode(keyPair.sign(signatureHash), bitcoin.Transaction.SIGHASH_ALL);

  IsWitness() ?
      tx.setWitness(i, [
        signature,
        Buffer.from('00', 'hex'),
        Buffer.from('11', 'hex'),
        redeemScript
      ]) :
      tx.setInputScript(i, bitcoin.script.compile([
        signature,
        Buffer.from('22', 'hex'),
        Buffer.from('33', 'hex'),
        redeemScript
      ]));


  return await exports.broadcast(tx.toHex());
}

[junderw]

Psbt doesn't support finalization of arbitrary scripts, so you should just use the Transaction API for now.

[junderw]

@3s3s I have a pull request for adding the ability to use a custom finalizer.
Check it out and tell me if you have any questions about it.

#1491

[monokh]

One use case for creating a PSBT based on a raw transaction is for fee bumping. Something like Psbt.fromTransaction would be cool but still have plenty of headaches because of the need to provide input values and adjust the outputs.

At this point my process will be to create a Transaction object and then carry over the inputs, outputs etc to the PSBT object.