Zeromq bitcoind subscriptions #390

jotapea · 2021-07-29T05:41:29Z

PR for Listen to bitcoind with ZeroMQ #346.

src/bitcoind.ts

bodymindarts · 2021-07-29T14:51:01Z

src/bitcoind.ts

+    txid: string
+    include_watchonly?: boolean
+  }): Promise<InWalletTransaction> {
+    return await this.client.getTransaction({ txid, include_watchonly })


Why are you sometimes using the default client and sometimes this.client? I think I'd prefer to always use this.client and pass in the default client via constructor if that works.

this.client is also the instance variable name used in the BitcoindWalletClient because it actually is a client just like in the BitcoindClient class. The only difference is that in BitcoindWalletClient it was instantiated with the wallet argument. This is the way for the BitcoindWalletClient instances to contact the same "default" (main) client while being "loaded" and ready to invoke wallet rpc calls.

src/bitcoindSubscribers.ts

bodymindarts · 2021-07-29T14:53:10Z

test/integration/01-setup/01-connection.spec.ts

@@ -21,6 +27,22 @@ it("connects to bitcoind", async () => {
  expect(chain).toEqual("regtest")
 })

+it("detect active bitcoind zeromq notifications", async () => {
+  const notifications = await bitcoindClient.getZmqNotifications()
+  expect(notifications.length).toEqual(4) // 2 per topic


Where is the code that creates these notifications? This is very implicit and order dependent. Perhaps you could make this idempotent like so:

const beforeLength = await bitcoindClient.getZmqNotifications().length // execute some tx const notifications = await bitcoindClient.getZmqNotifications() expect(notifications.length).toEqual(beforLength + 2)

Enabling zeromq is set in bitcoind configuration, and it was already enabled. So that call is not being affected by this branch.

I don't understand. This is a new test right? So can we try to make any new tests we add be idempotent?

It is a newly added test yes, but is not actually testing anything that has a "before" and "after". It is just detecting if the bitcoind client has the zeromq notifications enabled.

bodymindarts · 2021-07-29T14:56:11Z

src/bitcoindSubscribers.ts

+    for await (const [topic, msg] of sock) {
+      if (topic.toString() === subscribeTopic) {
+        const rawTx = msg.toString("hex")
+        const tx = await bitcoindDefaultClient.decodeRawTransaction({ hexstring: rawTx })


can we always pass in a client and not ever use the default client except on the top level when we inject it? This would remove coupling to global state and improve testability.

jotapea · 2021-07-29T17:15:55Z

@bodymindarts The way I am approaching it is that the wallet-clients are separate from the "loading of" or "total of" loaded wallets in the default (main) client. So, to use a wallet, you actually need a new wallet-client for this. Also the classes are done in a way that their methods don't overlap (at least for now).

jotapea · 2021-07-30T15:11:09Z

@nicolasburtey @bodymindarts I think this is ready to get merged, at least from the perspective of testing. I am not sure where else changes need to be done for production compatibility (i.e. equivalent to the opening of the ports in docker-compose).

bodymindarts

The PR looks good but still you are coupling to global state which we are trying to move away from for better testability / code structure / modularisation etc...

bodymindarts · 2021-08-02T17:36:02Z

src/core/specter-wallet.ts

@@ -267,7 +267,7 @@ export class SpecterWallet {
      throw new Error(err)
    }

-    const tx = await this.bitcoindClient.getTransaction({
+    const tx = await bitcoindDefaultClient.getTransaction({


Can we please avoid coupling to global state / objects. No singletons... use constructor injection.

bodymindarts · 2021-08-02T17:36:06Z

src/services/bitcoind/subscribers.ts

+    for await (const [topic, msg] of sock) {
+      if (topic.toString() === BITCOIND_EVENTS.RAW_TX) {
+        const rawTx = msg.toString("hex")
+        const tx = await bitcoindDefaultClient.decodeRawTransaction({ hexstring: rawTx })


Can we please not reference singletons / depend on global state?

jotapea · 2021-08-03T01:11:56Z

@bodymindarts bitcoindDefaultClient is now fully removed.

bitcoindSubscribers module includes logging messages test functions receiveRawBlockSubscriber and receiveRawTxSubscriber

TODO: not using SpecterWalletConstructorArgs as I am not sure the best way to add BitcoindClient type

- getBalancesDetail and getBalance bitcoind index utility functions receive BitcoindClient as parameter - replaced with OnChainMixin instance variable TODO?

bodymindarts

Please remove the TODO comments. If you are unsure then ask in PR discussion don't commit this noise to the code.
Other than that its looking fine.

bodymindarts · 2021-08-05T07:58:34Z

src/core/on-chain/index.ts

@@ -61,10 +61,14 @@ export const getOnChainTransactions = async ({
 export const OnChainMixin = (superclass) =>
  class extends superclass {
    readonly config: UserWalletConfig
+    // TODO? is this ok here?


Why the comment in code? Please remove

bodymindarts · 2021-08-05T07:58:44Z

src/core/on-chain/index.ts


    constructor(...args) {
      super(...args)
      this.config = args[0].config
+      // TODO? is this ok here?


This shouldn't be committed to code

@bodymindarts Comment removed, and I assume it is ok to add the bitcoind client as an instance variable to the "mixin", at least for now.

bodymindarts · 2021-08-05T07:59:16Z

src/core/specter-wallet.ts

  readonly logger: Logger
  readonly config: SpecterWalletConfig

-  constructor({ logger, config }: SpecterWalletConstructorArgs) {
+  // TODO? Add BitcoindClient type to SpecterWalletConstructorArgs


Why the comment with a question? Either do it or don't or bring it up for discussion in PR review... not in code

How should I add the BitcoindClient type to the SpecterWalletConstructorArgs? @bodymindarts @samerbuna @dolcalmi

bodymindarts · 2021-08-05T07:59:54Z

src/services/bitcoind/subscribers.ts

+export enum BITCOIND_EVENTS {
+  RAW_BLOCK = "rawblock",
+  RAW_TX = "rawtx",
+  // HASH_BLOCK = "hashblock",


Why keep commented code here?

For this one I am not really sure why we have to remove them. It is there for completeness of the events that bitcoind emits. And they are commented out because we don't currently use them, but we could in the future.

- estimateSmartFee added to BitcoindClient - listTransactions added to BitcoindWalletClient - getTransaction moved to BitcoindWalletClient - getNewAddress now accepts optional address_type param - sendToAddress now accepts optional fee_rate param

bodymindarts reviewed Jul 29, 2021

View reviewed changes

jotapea force-pushed the zeromq-bitcoind-sub branch from 47c1a75 to 74ceac5 Compare August 2, 2021 17:06

bodymindarts reviewed Aug 2, 2021

View reviewed changes

jotapea added 12 commits August 3, 2021 10:23

Move getTransaction method to BitcoindClient

77c367a

Add DecodeRawTransactionResult type

b4a7649

Add bitcoindSubscribers and zeromq dependencies

0c332a2

bitcoindSubscribers module includes logging messages test functions receiveRawBlockSubscriber and receiveRawTxSubscriber

Add zeromq detection and subscription tests

7a08eb2

Make default include_watchonly true like reference

cf60907

Make bitcoindSubscribers topics an events enum

0266fea

Improve variable names and explicitness of test

4853650

Rebase adjustments

885d2df

Bitcoind receiveRawTxSubscriber client as input

241ef30

Rename to bitcoindWalletClient instance variable

1fbcbbf

BitcoindClient instance specter wallet constructor

e6f58b3

TODO: not using SpecterWalletConstructorArgs as I am not sure the best way to add BitcoindClient type

Fully remove bitcoindDefaultClient

e0da9d9

- getBalancesDetail and getBalance bitcoind index utility functions receive BitcoindClient as parameter - replaced with OnChainMixin instance variable TODO?

jotapea force-pushed the zeromq-bitcoind-sub branch from d25dac1 to e0da9d9 Compare August 3, 2021 14:23

bodymindarts requested changes Aug 5, 2021

View reviewed changes

jotapea added 2 commits August 5, 2021 11:05

Remove question from code

1076905

Multiple bitcoind updates

3b17551

- estimateSmartFee added to BitcoindClient - listTransactions added to BitcoindWalletClient - getTransaction moved to BitcoindWalletClient - getNewAddress now accepts optional address_type param - sendToAddress now accepts optional fee_rate param

samerbuna marked this pull request as draft September 13, 2021 20:23

nicolasburtey closed this Dec 20, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Zeromq bitcoind subscriptions #390

Zeromq bitcoind subscriptions #390

jotapea commented Jul 29, 2021

bodymindarts Jul 29, 2021

jotapea Aug 2, 2021

bodymindarts Jul 29, 2021 •

edited

Loading

jotapea Jul 29, 2021

bodymindarts Jul 29, 2021

jotapea Jul 29, 2021

bodymindarts Jul 29, 2021

jotapea commented Jul 29, 2021

jotapea commented Jul 30, 2021

bodymindarts left a comment

bodymindarts Aug 2, 2021

bodymindarts Aug 2, 2021

jotapea commented Aug 3, 2021

bodymindarts left a comment

bodymindarts Aug 5, 2021

bodymindarts Aug 5, 2021

jotapea Aug 5, 2021

bodymindarts Aug 5, 2021

jotapea Aug 5, 2021

bodymindarts Aug 5, 2021

jotapea Aug 5, 2021 •

edited

Loading

Zeromq bitcoind subscriptions #390

Zeromq bitcoind subscriptions #390

Conversation

jotapea commented Jul 29, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bodymindarts Jul 29, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jotapea commented Jul 29, 2021

jotapea commented Jul 30, 2021

bodymindarts left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jotapea commented Aug 3, 2021

bodymindarts left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jotapea Aug 5, 2021 • edited Loading

Choose a reason for hiding this comment

bodymindarts Jul 29, 2021 •

edited

Loading

jotapea Aug 5, 2021 •

edited

Loading