Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bitcoind for onchain #317

Closed
wants to merge 38 commits into from
Closed
Show file tree
Hide file tree
Changes from 28 commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
d7b3016
Adding temporary bitcoindHotWalletClient
jotapea Jul 9, 2021
91efc67
Starting to add bitcoind functions that will replace lnd equivalents …
jotapea Jul 9, 2021
d65b6b0
Bitcoind sendToAddress is in BTC
jotapea Jul 9, 2021
f25b124
Getting the fee from getTransaction directly
jotapea Jul 9, 2021
e2a4b73
Logs showing most of bitcoind values working (tests pass)
jotapea Jul 9, 2021
4055d24
Revert "Logs showing most of bitcoind values working (tests pass)"
jotapea Jul 9, 2021
b2cb90b
Prettier modifications
jotapea Jul 9, 2021
f892602
Disable unused variable warning
jotapea Jul 9, 2021
efa79b6
Use bitcoindHotWalletClient instead of bitcoindDefaultClient
jotapea Jul 9, 2021
f53c0e4
Keep imports in alphabetical order
jotapea Jul 9, 2021
722eaac
Add (temporary?) PayOnChainClient class
jotapea Jul 12, 2021
b42e713
Add BitcoindClient class and one line switch
jotapea Jul 12, 2021
846163a
Switch back to lnd client
jotapea Jul 12, 2021
eaae5d9
Comments of failed bitcoind tests
jotapea Jul 12, 2021
c9f5023
Removing try catch from test
jotapea Jul 12, 2021
931930a
Name the default bitcoind wallet 'default'
jotapea Jul 12, 2021
7a8c996
Prettier
jotapea Jul 12, 2021
d2b9993
Adding test to funding new hot wallet fails
jotapea Jul 12, 2021
f191f71
Comment out failing test
jotapea Jul 12, 2021
2195777
Rename default test and remove hot wallet
jotapea Jul 14, 2021
7b3014d
Rename test from fund bitcoind to outside
jotapea Jul 14, 2021
d75007f
Rename onchain_funding to lnd_onchain_funding
jotapea Jul 14, 2021
74cc217
Add test that creates hot bitcoind wallet
jotapea Jul 14, 2021
5e85a45
One more location to set the "default" wallet
jotapea Jul 14, 2021
ddcef66
Tests pass deposit to bitcoind test commented out
jotapea Jul 14, 2021
2ff75ba
Add bitcoind client version of onchain methods
jotapea Jul 15, 2021
f5403dc
Add alternate bitcoind_onchain_funding function
jotapea Jul 15, 2021
0d34550
Enable bitcoind client and funding tests TODO
jotapea Jul 15, 2021
a5d0433
Revert "Enable bitcoind client and funding tests TODO"
jotapea Jul 15, 2021
cdb2130
Fix: Define type for transactions_bitcoind
jotapea Jul 15, 2021
8c2c32d
Onchain receive test passes
jotapea Jul 16, 2021
3137c35
Temporary removal
jotapea Jul 16, 2021
a88636e
Revert "Temporary removal"
jotapea Jul 16, 2021
9eed585
Rename class avoid conflict with BitcoindClient
jotapea Jul 16, 2021
2daa2dc
Rename 'default' wallet to 'outside'
jotapea Jul 16, 2021
2d47cb9
Rename to bitcoindHotWalletClient
jotapea Jul 16, 2021
d20f6de
Keep default bitcoind client while having outside
jotapea Jul 16, 2021
038be28
Temporary remove
jotapea Jul 16, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
301 changes: 288 additions & 13 deletions src/OnChain.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,12 @@ import {
TransactionRestrictedError,
ValidationError,
} from "./error"
import { customerPath, lndAccountingPath, onchainRevenuePath } from "./ledger/ledger"
import {
bitcoindAccountingPath,
customerPath,
lndAccountingPath,
onchainRevenuePath,
} from "./ledger/ledger"
import { getActiveOnchainLnd, getLndFromPubkey } from "./lndUtils"
import { lockExtendOrThrow, redlock } from "./lock"
import { baseLogger } from "./logger"
Expand All @@ -33,10 +38,12 @@ import { UserWallet } from "./userWallet"
import {
amountOnVout,
bitcoindDefaultClient,
bitcoindHotClient,
btc2sat,
LoggedError,
LOOK_BACK,
myOwnAddressesOnVout,
sat2btc,
} from "./utils"

export const getOnChainTransactions = async ({
Expand All @@ -59,14 +66,139 @@ export const getOnChainTransactions = async ({
}
}

export const getOnChainTransactionsBitcoind = async ({
incoming, // TODO?
}: {
incoming: boolean
}) => {
try {
// TODO? how many transactions back?
const transactions: [] = await bitcoindHotClient.listTransactions(null, 1000)
return transactions
} catch (err) {
const error = `issue fetching bitcoind transaction`
baseLogger.error({ err, incoming }, error)
throw new LoggedError(error)
}
}

///////////
abstract class PayOnChainClient {
client

static clientPayInstance(): PayOnChainClient {
// * ALSO change updatePending *
// return new LndOnChainClient()
return new BitcoindClient()
}

abstract getBalance(): Promise<number>

abstract getEstimatedFee(
sendTo?: {
address: string
tokens: number
}[],
): Promise<number>

// return txid
abstract sendToAddress(address: string, amount: number): Promise<string>

abstract getTxnFee(id: string): Promise<number>
}

class LndOnChainClient extends PayOnChainClient {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is a highly coupled class with getActiveOnchainLnd. Maybe is better pass the lnd client as parametter (the same for BitcoindClient)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The main purpose of the classes at the moment is for ease of development purposes. After the bitcoind client works as a replacement to the lnd client, then we could reconsider the purpose and structure of these classes.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but you have only one instance (line 68) and in the same file

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this still relevant @dolcalmi ?

constructor() {
super()
this.client = getActiveOnchainLnd().lnd
}

async getBalance(): Promise<number> {
const { chain_balance: onChainBalance } = await getChainBalance({ lnd: this.client })
return onChainBalance
}

async getEstimatedFee(
sendTo: {
address: string
tokens: number
}[],
): Promise<number> {
const { fee: estimatedFee } = await getChainFeeEstimate({
lnd: this.client,
send_to: sendTo,
})
return estimatedFee
}

async sendToAddress(address: string, amount: number): Promise<string> {
const { id } = await sendToChainAddress({ address, lnd: this.client, tokens: amount })
return id
}

async getTxnFee(id: string): Promise<number> {
const outgoingOnchainTxns = await getOnChainTransactions({
lnd: this.client,
incoming: false,
})
// eslint-disable-next-line
let fee
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why are fee and fee_ necessary?

const [{ fee: fee_ }] = outgoingOnchainTxns.filter((tx) => tx.id === id)
// eslint-disable-next-line
fee = fee_
return fee
}
}

// eslint-disable-next-line
class BitcoindClient extends PayOnChainClient {
constructor() {
super()
this.client = bitcoindHotClient
}

async getBalance(): Promise<number> {
const onChainBalance2Btc = await this.client.getBalance()
const onChainBalance2 = btc2sat(onChainBalance2Btc)
return onChainBalance2
}

// TODO! FIX
// eslint-disable-next-line
async getEstimatedFee(sendTo?: any): Promise<number> {
return Promise.resolve(1000)
// // TODO! estimatedFee2: {"errors":["Insufficient data or no feerate found"],"blocks":2}
// const confTarget = 1 // same with 1 // 6
// // TODO: estimate_mode
// const estimatedFee2 = await this.client.estimateSmartFee(confTarget)
// return estimatedFee2
}

async sendToAddress(address: string, amount: number): Promise<string> {
const id2 = await this.client.sendToAddress(address, sat2btc(amount))
return id2
}

async getTxnFee(id: string): Promise<number> {
const txn = await this.client.getTransaction(id) //, null, true) // verbose true
const fee2 = btc2sat(-txn.fee) // fee comes in BTC and negative
return fee2
}
}
///////////

export const OnChainMixin = (superclass) =>
class extends superclass {
constructor(...args) {
super(...args)
}

async updatePending(lock): Promise<void> {
await Promise.all([this.updateOnchainReceipt(lock), super.updatePending(lock)])
// await Promise.all([this.updateOnchainReceipt(lock), super.updatePending(lock)])
await Promise.all([
this.updateOnchainReceiptBitcoind(lock),
super.updatePending(lock),
])
}

async getOnchainFee({
Expand Down Expand Up @@ -217,16 +349,17 @@ export const OnChainMixin = (superclass) =>
throw new TransactionRestrictedError(error, { logger: onchainLogger })
}

const { lnd } = getActiveOnchainLnd()
// const { lnd } = getActiveOnchainLnd()
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dolcalmi Notice that this is the same call that is now in line 90 (this.client = getActiveOnchainLnd().lnd), so it should work just like it used to.

const clientPayInstance = PayOnChainClient.clientPayInstance() // lnd-onchain or bitcoind

const { chain_balance: onChainBalance } = await getChainBalance({ lnd })
const onChainBalance = await clientPayInstance.getBalance()

let estimatedFee, id, amountToSend

const sendTo = [{ address, tokens: checksAmount }]
const sendTo = [{ address, tokens: checksAmount }] // only required with lnd

try {
;({ fee: estimatedFee } = await getChainFeeEstimate({ lnd, send_to: sendTo }))
estimatedFee = await clientPayInstance.getEstimatedFee(sendTo)
} catch (err) {
const error = `Unable to estimate fee for on-chain transaction`
onchainLogger.error({ err, sendTo, success: false }, error)
Expand Down Expand Up @@ -284,7 +417,7 @@ export const OnChainMixin = (superclass) =>

return lockExtendOrThrow({ lock, logger: onchainLogger }, async () => {
try {
;({ id } = await sendToChainAddress({ address, lnd, tokens: amountToSend }))
id = await clientPayInstance.sendToAddress(address, amountToSend)
} catch (err) {
onchainLogger.error(
{ err, address, tokens: amountToSend, success: false },
Expand All @@ -295,12 +428,7 @@ export const OnChainMixin = (superclass) =>

let fee
try {
const outgoingOnchainTxns = await getOnChainTransactions({
lnd,
incoming: false,
})
const [{ fee: fee_ }] = outgoingOnchainTxns.filter((tx) => tx.id === id)
fee = fee_
fee = await clientPayInstance.getTxnFee(id)
} catch (err) {
onchainLogger.fatal({ err }, "impossible to get fee for onchain payment")
fee = 0
Expand Down Expand Up @@ -329,6 +457,7 @@ export const OnChainMixin = (superclass) =>

// TODO/FIXME refactor. add the transaction first and set the fees in a second tx.
await MainBook.entry(memo)
// TODO: this is no longer from Lightning
.credit(lndAccountingPath, sats - this.user.withdrawFee, metadata)
.credit(onchainRevenuePath, this.user.withdrawFee, metadata)
.debit(this.user.accountPath, sats, metadata)
Expand Down Expand Up @@ -395,6 +524,45 @@ export const OnChainMixin = (superclass) =>
return address
}

async getOnChainAddressBitcoind(): Promise<string> {
// TODO
// another option to investigate is to have a master key / client
// (maybe this could be saved in JWT)
// and a way for them to derive new key
//
// this would avoid a communication to the server
// every time you want to show a QR code.

let address

const onchainClient = bitcoindHotClient
// TODO?
const pubkey = "TODO" // pubkey: process.env[`${input.name}_PUBKEY`] || exit(98),

try {
address = await onchainClient.getNewAddress()
} catch (err) {
const error = `error getting bitcoind onchain address`
this.logger.error({ err }, error)
throw new LoggedError(error)
}

try {
this.user.onchain.push({ address, pubkey })
await this.user.save()
} catch (err) {
const error = `error storing new onchain address to db`
throw new DbError(error, {
forwardToClient: false,
logger: this.logger,
level: "warn",
err,
})
}

return address
}

async getOnchainReceipt({ confirmed }: { confirmed: boolean }) {
const pubkeys: string[] = this.user.onchain_pubkey
let user_matched_txs: GetChainTransactionsResult["transactions"] = []
Expand Down Expand Up @@ -473,6 +641,60 @@ export const OnChainMixin = (superclass) =>
return user_matched_txs
}

// eslint-disable-next-line
async getOnchainReceiptBitcoind({ confirmed }: { confirmed?: boolean }) {
// TODO? confirmed?
const pubkeys: string[] = this.user.onchain_pubkey
let user_matched_txs: GetChainTransactionsResult["transactions_bitcoind"] = []

for (const pubkey of pubkeys) {
// TODO: optimize the data structure
const addresses = this.user.onchain
.filter((item) => (item.pubkey = pubkey))
.map((item) => item.address)

// const onchainClient = bitcoindHotClient

const incoming_txs = await getOnChainTransactionsBitcoind({ incoming: true })

// bitcoind transactions:
// [
// {
// "address": "bcrt1qs2gudhr2c6vk5gjt338ya82y5tr78kd0xf9ht5",
// "category": "receive",
// "amount": 1.02443592,
// "label": "",
// "vout": 1,
// "confirmations": 9,
// "blockhash": "6bd2ec8a3c04ecfd96ceac13d894f2641cfb622a3dc144cf5723108d169428f9",
// "blockheight": 120,
// "blockindex": 1,
// "blocktime": 1626221313,
// "txid": "962e18e6c819742211185333825e3159f67980c0ad30120bb6a05f3bbed492ea",
// "walletconflicts": [],
// "time": 1626221313,
// "timereceived": 1626221313,
// "bip125-replaceable": "no"
// }
// ]

let incoming_filtered: GetChainTransactionsResult["transactions_bitcoind"]

// TODO? not doing any filtering for now...
// eslint-disable-next-line
incoming_filtered = incoming_txs

user_matched_txs = [
...incoming_filtered.filter(
// only return transactions for addresses that belond to the user
(tx) => _.intersection(tx.address, addresses).length > 0,
),
]
}

return user_matched_txs
}

async getTransactions() {
const confirmed: ITransaction[] = await super.getTransactions()

Expand Down Expand Up @@ -648,4 +870,57 @@ export const OnChainMixin = (superclass) =>
},
)
}

async updateOnchainReceiptBitcoind(lock?) {
const user_matched_txs = await this.getOnchainReceiptBitcoind({ confirmed: true })

const type = "onchain_receipt"

await redlock(
{ path: this.user._id, logger: baseLogger /* FIXME */, lock },
async () => {
// FIXME O(n) ^ 2. bad.
for (const matched_tx of user_matched_txs) {
// has the transaction has not been added yet to the user account?
//
// note: the fact we fiter with `account_path: this.user.accountPath` could create
// double transaction for some non customer specific wallet. ie: if the path is different
// for the dealer. this is fixed now but something to think about.
const mongotx = await Transaction.findOne({
accounts: this.user.accountPath,
type,
hash: matched_tx.txid,
})

if (!mongotx) {
const sats = btc2sat(matched_tx.amount)
const fee = Math.round(sats * this.user.depositFeeRatio)

const metadata = {
currency: "BTC",
type,
hash: matched_tx.txid,
pending: false,
...UserWallet.getCurrencyEquivalent({ sats, fee }),
payee_addresses: [], // TODO?
}

await MainBook.entry()
.credit(onchainRevenuePath, fee, metadata)
.credit(this.user.accountPath, sats - fee, metadata)
.debit(bitcoindAccountingPath, sats, metadata)
.commit()

const onchainLogger = this.logger.child({
topic: "payment",
protocol: "onchain",
transactionType: "receipt",
onUs: false,
})
onchainLogger.info({ success: true, ...metadata })
}
}
},
)
}
}