chore(payment): Implement silent payments

[jvgelder]

Draft for now looking for a bit of feedback on how we can expose the functionality to the developer

Implements bip352 and exposes the following functions:

• scanForSilentPayments: to find if given outputs belong to our sp address
• deriveOutput: to create a new silent payment output
• encodeSilentPaymentAddress
• decodeSilentPaymentAddress

Note it tests against vectors taken from: https://github.com/bitcoin/bips/blob/master/bip-0352/send_and_receive_test_vectors.json

TODO:

• Decide how we expose scanning and deriving to the developer i guess via the p2sp object ?
• Decide whether we return the found hex endcoded or Uint8Array output
• Figure out of we can remove modN32 and subBE as they might exist?
• Add derivation logic
• Replace getPubkeyFromInputTS from test with something that exists?

Do note that when this bitcoin-core/secp256k1#1698 gets merged and we make it available in https://github.com/bitcoinjs/tiny-secp256k1 part of this code becomes redundant but at least we already have the interfaces in place.

Bluewallet also has a TS implementation : https://github.com/BlueWallet/SilentPayments/blob/master/src/index.ts

[junderw]

I've been in discussions on the best way to handle this, and I think it would be a better idea to pass in a cached key.

Instead of calculating the sum of the inputs in real time every time, The payment should only take a special key ie. inputAggregate or something with an easier name. Then separate from the Payments API we would offer a helper function that can calculate the aggregate key from given inputs array.

This is primarily because our friends over at BlueWallet see a future where apps like electrs or mempool etc will pre-calculate the aggregate keys and index them for us....

But... yeah... ballooning Payments API into something that encompasses a whole transaction seems like a bit much. At that point we might as well turn it into a method on the Psbt.

[junderw]

@Overtorment if you'd like to chime in, feel free.

[jvgelder]

I've been in discussions on the best way to handle this, and I think it would be a better idea to pass in a cached key.

Instead of calculating the sum of the inputs in real time every time, The payment should only take a special key ie. inputAggregate or something with an easier name. Then separate from the Payments API we would offer a helper function that can calculate the aggregate key from given inputs array.

This is primarily because our friends over at BlueWallet see a future where apps like electrs or mempool etc will pre-calculate the aggregate keys and index them for us....

But... yeah... ballooning Payments API into something that encompasses a whole transaction seems like a bit much. At that point we might as well turn it into a method on the Psbt.

ye maybe use the following logic:

• If shared secret S?: Uint8Array and inputTweak?: Uint8Array are set we go straight to deriveOutputs
• else if inputs?: Input[] and privKeys?: Array<{ priv: Uint8Array; isXOnly: boolean }> we do calculate the tweak and shared secret
• else we throw an error.

1. Is there a use case to allow the user to provide A_sum instead of the private keys?
2. Besides returning outputs we may also want to return a transaction that can be signed or is this not the place to do that?
3. Similar to the point above do we want to return signed outputs?

[Overtorment]

ill take a look.

btw we have a TS implementation of SP here: https://github.com/BlueWallet/SilentPayments/

[jvgelder]

I've been in discussions on the best way to handle this, and I think it would be a better idea to pass in a cached key.
Instead of calculating the sum of the inputs in real time every time, The payment should only take a special key ie. inputAggregate or something with an easier name. Then separate from the Payments API we would offer a helper function that can calculate the aggregate key from given inputs array.
This is primarily because our friends over at BlueWallet see a future where apps like electrs or mempool etc will pre-calculate the aggregate keys and index them for us....
But... yeah... ballooning Payments API into something that encompasses a whole transaction seems like a bit much. At that point we might as well turn it into a method on the Psbt.

ye maybe use the following logic:

* If shared secret `S?: Uint8Array` and `inputTweak?: Uint8Array` are set we go straight to `deriveOutputs`

* else if `inputs?: Input[]` and `privKeys?: Array<{ priv: Uint8Array; isXOnly: boolean }>` we do calculate the tweak and shared secret

* else we throw an error.


1. Is there a use case to allow the user to provide A_sum instead of the private keys?

2. Besides returning outputs we may also want to return a transaction that can be signed or is this not the place to do that?

3. Similar to the point above do we want to return signed outputs?

Allow various cases for deriving of outputs :

bitcoinjs-lib/ts_src/payments/p2sp.ts

Line 89 in 4f52dde

lazy.prop(o, 'outputs', () => {

[jvgelder]

This is something I am unsure about.

1. Should outputs be the ones create to send?
2. Where and how do we store resulting UTXO when we scanned?

[jvgelder]

ill take a look.

btw we have a TS implementation of SP here: https://github.com/BlueWallet/SilentPayments/

Ye I mentioned on the top :) (thought I referred to to something inside the core bluewallet app).
Reason for this PR is that I think most of the code belongs in bitcoinjslib (see the use of tagged hash for example).
If you think you have more elegant parts feel free to point out / overwrite some of my code here ( I am not that familiar with typescript)

[junderw]

ye maybe use the following logic:

• If shared secret S?: Uint8Array and inputTweak?: Uint8Array are set we go straight to deriveOutputs
• else if inputs?: Input[] and privKeys?: Array<{ priv: Uint8Array; isXOnly: boolean }> we do calculate the tweak and shared secret
• else we throw an error.

1. Is there a use case to allow the user to provide A_sum instead of the private keys?
2. Besides returning outputs we may also want to return a transaction that can be signed or is this not the place to do that?
3. Similar to the point above do we want to return signed outputs?

I would like to repeat what I said before:

ballooning Payments API into something that encompasses a whole transaction seems like a bit much. At that point we might as well turn it into a method on the Psbt.

It seems like your reply is ignoring this and talking about placing all of that inside the p2sp Payment API still... am I misunderstanding your response? Could you please clarify?

[jvgelder]

ye maybe use the following logic:

• If shared secret S?: Uint8Array and inputTweak?: Uint8Array are set we go straight to deriveOutputs
• else if inputs?: Input[] and privKeys?: Array<{ priv: Uint8Array; isXOnly: boolean }> we do calculate the tweak and shared secret
• else we throw an error.

1. Is there a use case to allow the user to provide A_sum instead of the private keys?
2. Besides returning outputs we may also want to return a transaction that can be signed or is this not the place to do that?
3. Similar to the point above do we want to return signed outputs?

I would like to repeat what I said before:

ballooning Payments API into something that encompasses a whole transaction seems like a bit much. At that point we might as well turn it into a method on the Psbt.

It seems like your reply is ignoring this and talking about placing all of that inside the p2sp Payment API still... am I misunderstanding your response? Could you please clarify?

• scratch point 2 and 3 indeed.

For the inputs/outputs i was thinking of a bare minimal set of data, the parsing of the transactions into receiving inputs would be done elsewhere. I think the lazy output property kinda reflects this now right? or do you still think even that is too much here?

[Overtorment]

i feel a bit out of element here, every time i get distracted from SP i completely forget how they work and getting familiar again means re-reading all the specs again, and it takes time.

are there any specific questions for me? if yes, dumb it down for me.
otherwise, the standalone lib https://github.com/BlueWallet/SilentPayments/ does the job well, and embeds nicely into whatever js library for bitcoin you are going to use (not necessarily bitcoinjs-lib, more modern js apps choose to use scure-btc-signer instead )

[jvgelder]

i feel a bit out of element here, every time i get distracted from SP i completely forget how they work and getting familiar again means re-reading all the specs again, and it takes time.

are there any specific questions for me? if yes, dumb it down for me. otherwise, the standalone lib https://github.com/BlueWallet/SilentPayments/ does the job well, and embeds nicely into whatever js library for bitcoin you are going to use (not necessarily bitcoinjs-lib, more modern js apps choose to use scure-btc-signer instead )

no worries thanks for making me aware of scure-btc-signer also mentioned the blue wallet repo there as there was an issue open for silent payments too.

[junderw]

This PR is getting to be way out of scope for this library.

Wipe everything and start from the beginning.

What is your goal? Don't be vague. "To support BIP352" is vague. Describing a list of all the things you did before planning anything is also vague... it doesn't tell me your intentions.

1. What are your intentions?
2. What are the requirements?
3. What are the specific changes you need to fill each of the requirements?

Let's start there.

[junderw]

Stop coding. Discussions only for now.

[jvgelder]

1. I try to get Silent payments in this lib so that its easy for anyone already depending on this library to support spending and receiving from and to silent payment addresses
2. We need:

• to be able to generate a silent payment address to receive funds on
• to be able to spend to a silent payment address
• To be able to find payments to our silent payment addresses

• Silent payment address decoding / encoding
• Silent output derivation
• Scanning transactions to find payments that belong to one of our silent payment addresses
• Optionally we can have an external provider make us aware of UTXOs which belong to us

Note that the silent payment address looks one way when you share it and different on chain as the addresses are derived from the silent payment address.

[junderw]

1. Silent payment address decoding / encoding
   1. Currently the address module does output > address, address > output conversions. This will be difficult to include here since the output > address doesn't have a direct 1:1 mapping. address > output could be possible if we create a separate function for SP only, and pass in the information necessary. However, the dependency graph of necessary information to create the output is complicated and lends itself well to the Payments API's lazy getters. So if you have t_k and B_spend, you don't need to pass in an array of A_n pubkeys or privkeys at all.
2. Silent output derivation
   1. This can be done with Payments API via lazy getters, but the problem is the dependency graph has two "paths" the sender (private a_n public B_spend) and the receiver (Public A_n private b_spend) which we should DEFINITELY support both paths, while validating for nonsensical paths.
3. Scanning transactions + external provider (3 & 4)
   1. I think this is where we start to get out of scope.

---

I think that perhaps instead of using the existing Payments API, we should create a new SilentPayment class that doesn't extend Payment, it just exists on its own.

It contains either a pub or priv of both spend and scan keys. It should have a static constructor method fromStealthAddress(s: string): SilentPayment that creates one with both public keys.

Starting from this baseline, we could try to make this class interact with Transaction and Psbt.

I don't think any code in bitcoinjs-lib should try to connect to RPC or some external service. It should be a simple data in data out API. ie. pass in a Transaction and Uint8Array[] (where each scriptPubkey Uint8Array corresponds to the same index input of the Transaction) Or maybe as simple as inputs, outputs (the outputs spent by the inputs)

This might even be better suited for its own library that depends on bitcoinjs-lib... you could do that and just upstream the minimal changes you need from us to better support your library.

I'm still open to including it in bitcoinjs-lib, but one thing I'm a bit unsure about is the Psbt support.

For sending TO a stealth address, there should really be stealth payment info embedded in the Psbt so that we can update the output addresses if someone adds an input to a partially signed Psbt.

I believe the BIP352 authors are currently discussing how to deal with Psbt...

However, we DID release P2TR support without Psbt support initially (and it was very hacky and no one liked it) so I'm open to all ideas.

Please discuss a concrete plan for an API (ie. like a d.ts style pseudo-type-declaration just to give a rough idea) before you start coding though so we don't waste your time.

If you decide to go the separate library route, feel free to mention me on any issues you have over there and I'll be happy to join the discussion.

[jvgelder]

I think its good if the base is this in library as I assume it will get more eyeballs and checks from more people.

I tried to come with a very basic layout but its tougher than I thought, I dont want to expose the developer to too much of the internals. Should be as simple as I want to send 0.2 to silent payment address sp213123...

initial spending and receiving layout

class SilentPayment{
   
      /**
      * Encode a Silent payment address for others to spend to
      * Encodes spend and scan public keys into a Bech32m Silent Payment address.
      * @param {Uint8Array} [B_spend] - spend public key.
      * @param {Uint8Array} [B_scan] - public scan key.
      * @param {number} [version] - Optional version number of the silent payment scheme.
      * @param network - testing, regtest or prod
      * @returns bech32m encoded string
      */
      export function encodeSilentPaymentAddress(
          B_spend: Uint8Array,
          B_scan: Uint8Array,
          version = 0,
          network: Network = BITCOIN_NETWORK,
      ): string 

      // TODO what kind of output do we want here ?
      fuction createOutput(inputs : Input[],  recipients :  {silentAddr: string, value : number}[]) :  ?
      
     /**
     * Convenience function to create outputs from a transaction
     * Note: Silent payment depend on inputs, so if any input changes the output also changes
     * This function removes all other outputs already set on the transaction to prevent outputs created on different input 
     * set.
     * @param recipients - list of silent payment recipients
     * @param tx - Partially build transactions already containing the required inputs
     * @returns a transaction with 
     */
     export function createSilentPaymentTransaction(recipients :  {silentAddr: string, value : number}[], tx: Transaction) : Transaction
     
     /**
     Storage interface for silent address and generated in/output addresses
     Used to display to the user which on chain addresses belong to which silent payment addresses
     */
     interface SilentPaymentOutputGroup{
        silentPaymentAddress : string
       sent : {txid : string, vout: number, pub_key : string, destSilentPaymentAddress : string}[],
       received : {txid : string, vout: number, pub_key : string, originSilentPaymentAddress?: string}[]
     }
}

Scanning

I think we may want to move scanning to a different PR and first focus on spending and sending.
However i did create a few options for scanning results sorted by trust level as it may influence how we structure code.
Hopefully external service are created on top of the low trust model so that privacy is preserved
For doing business with entities that require travel rule it might be convenient to share the shared_secret S
This gives on chain privacy and gives regulatory compliance

class SilentPayment{


      /**
      verify if given outputs really belong to us
      @param potentialOutputs - List of outputs potentially to be for us
      @param S - shared secret
      @param B_spend - public spend key
      @param b_scan - private scan key
      @returns a list of SilentPaymentOutputs that can be used to construct an actual p2tr output or Pbst
      */
      verifyScanResult(potentialOutputs : Output[], S: Uint8Array, B_spend : Uint8Array,  b_scan : Uint8Array) : SilentPaymentOutput []

      /**
      scan if the given outputs belong to us
      @param  tk - tweaked hash
      @param potentialOutputs - List of outputs potentially to be for us
      @param B_spend - public spend key ( to check if 
      @param b_scan - private scan key
      @returns a list of SilentPaymentOutputs that can be used to construct an actual bip341 output
      */
      scan(tk: Uint8Array, potentialOutputs : Output[], B_spend : Uint8Array,  b_scan : Uint8Array) : SilentPaymentOutput []

      /**
      check if a given transaction contains UTXO for our key public key B_scan
      @param  decodedTx - decoded transaction
      @param B_spend - public spend key
      @param b_scan - private scan key
      */
      scan(decodedTx: Transaction, B_spend : Uint8Array,  b_scan : Uint8Array) : SilentPaymentOutput []
      /**
      check if a given transaction contains UTXO for our key public key B_scan
      @param  rawTx - raw transaction bytes
      @param B_spend - public spend key
      @param b_scan - private scan key
      @returns a list of SilentPaymentOutputs that can be used to construct an actual p2tr output or Pbst
      */
      scan( rawTx: Uint8Array,  B_spend : Uint8Array, b_scan : Uint8Array) : SilentPaymentOutput []
      
     
}

[junderw]

createOutput

1. We would need the private keys to every eligible Input's pubkey.

createSilentPaymentTransaction

1. Same as createOutput, we need some way to represent the fact that we have all the private keys for all eligible inputs' pubkeys.

SilentPaymentOutputGroup

I didn't understand the use case for this, nor how a hypothetical function that output this interface would work.

scan

Without the previous outputs (pointed at by the input) it's impossible to know the public key we need for creating the aggregate pubkey.

---

I am thinking more along the lines of:

1. SilentPayment contains 2 keys as attributes on the class. These can both be either private or public.
2. If both are public. It has methods to output a stealth address. It has a method to generate an output script based on passing in the aggregate private key and lowest lexical outpoint as an argument.
3. If scan is private, enable methods for scanning a given transaction + previous outputs array. Maybe add intermediate methods that can handle intermediate states (ie just passing in the shared secret, or just passing in the aggregate pubkey and lowest serialized outpoint etc.
4. If spend is private, add a method to output a labelled stealth address (since we have the spend private we can change the Bm pubkey) also add a method to output the private key for a given output we own (similar to the scan function without spend private, but in the spend private case the private key for the output should be given so that we can spend the transaction like normal.

I think this would be a good minimal interface to start with.