Wallet v2

[ekzyis]

Description

close #1495 fix #2234

DX for supporting a new wallet: 926c706

TODOs / Overview of Changes

• ✅ refactor vault (Migrate vault entries to new schema #2092)
  • ℹ️ new Vault table no longer contains foreign keys to wallets or users
  • ℹ️ wallets now point to the vault within their config, not separated from it
• ✅ support multiple receive and send protocols per wallet on the backend (Wallet schema v2 #2146)
  • ℹ️ new wallet schema consists of the following tables:
    • WalletTemplate
    • Wallet
    • WalletProtocol
    • table for each send and recv protocol (WalletSendNWC, WalletRecvNWC, ...)
• ✅ store send and receive config in separate tables for easier integrity checks (Wallet schema v2 #2146)
• ✅ JSDoc for protocols
  • ✅ wallets/lib (shared by client and server)
  • ✅ wallets/client
  • ✅ wallets/server
• ✅ separate wallets from protocols in frontend for better UX
  • ℹ️ we now show cards of wallets which have protocols inside of them
  • ℹ️ users can easier find a wallet to attach by just looking at their names/logos, instead of having to know if they have a wallet that supports a protocol that we also support
• ✅ create a list of wallets with the protocols they support
  • ✅ insert wallets with protocol support into static WalletTemplate table
  • ✅ double-check protocol support per wallet and if I missed any
  • ❌ add placeholder wallets like Phoenix?
  • ✅ proper input labels, placeholder, help etc. (compare with master)
• ✅ restructure wallets/ folder
  • ℹ️ client/server wallet imports are now split on the highest level: client only imports stuff from wallets/client/ and server only from wallets/server/
  • ℹ️ everything (exclusively) wallet related, including validation, was moved to the wallets/ folder
• ✅ wallet I/O
  • ✅ load plaintext wallets
  • ✅ load encrypted wallets
  • ✅ save plaintext wallets
  • ✅ save encrypted wallets
  • ✅ client-side form validation
  • ✅ server-side form validation
  • ✅ wallet network tests
    • ✅ payments (Use test payments with HODL invoices for wallet validation #1287)
    • ✅ invoices
  • ✅ save LNC (requires generating other fields)
  • ✅ create new wallets from template
  • ✅ fix LND gRPC macaroon transform to invalid value 🤔
• ✅ wallet API
  • ✅ payments
  • ✅ invoices
• ✅ update existing code to use wallet v2 API
  • ✅ send p2p zap
  • ✅ receive p2p zap
  • ✅ lightning address
  • ✅ autowithdrawals
  • ✅ manual withdrawals
• ✅ let users search for their wallet
  • ℹ️ (fuzzy) search should have autocomplete for wallet names
• ❓ include wizard / multi-step forms probably not in this PR
• ✅ wallet encryption
  • ✅ remove the requirement to have device sync enabled for a gun
    • ℹ️ wallets are now always encrypted and saved on server
  • ✅ generate key by default
  • ✅ show locked wallets if wrong key and prompt to enter passphrase
  • ✅ encrypt wallets with new key on passphrase export
  • ✅ reset option if passphrase lost
• ✅ wallet migration
  • ✅ migration of old CryptoKey
  • ✅ migrate local wallets (device sync was not enabled)
  • ✅ merge local wallets with existing wallets during migration
    • ℹ️ local wallets currently create a wallet separate from existing receiving credentials
• ✅ wallet status
  • ℹ️ set wallet send/recv status via trigger instead of deriving from logs
• ✅ wallet logs
  • ✅ always store all wallet logs on the server
    • ℹ️ no code to manage different sources
    • ℹ️ consistent logs across all devices
  • ❌ show status of wallet network tests on attach in dedicated UI instead of relying on wallet logs
  • ❌ don't show "logs", show wallet events like a wallet
    • ℹ️ this means that there won't be multiple log messages for the same payment. we will just show the current status of a payment, just like a wallet would do.
  • ✅ wallet logs for templates
  • ✅ pagination
  • ✅ log message context
• ✅ wallet settings
  • ✅ detach
  • ✅ enabled
  • ✅ priority via DnD
  • ✅ priority via DnD on mobile
  • ✅ (global) limits (dust, fee, balance)
• ✅ create another wallet from a template when there's already one
• ✅ make sure state is updated when a wallet is updated (WALLETS vs WALLET query)
  • ✅ refetch WALLET on save or detach
  • ✅ refetch WALLETS if wallets changed
• ✅ handle device-dependant wallets like WebLN
• ✅ append and validate domain in lightning address input
• ✅ make sure this fixes duplicate lost hat/horse when wallet is disabled #2234
• ✅ update lightning address form in wallet receive prompt
• ✅ update code to pay QR code with WebLN
• ✅ split CUSTOM wallet into NWC and LN_ADDR wallet
• ✅ handle IndexedDB not available
• ✅ Q&A:
  • ❓ can key hash and wallet encryption get out of sync?
    • ℹ️ in that case, the passphrase prompt would ask for the wrong key
    • ✅ update: we now always compare key hashes on the server
  • ✅ payments
  • ✅ migration

Test instructions

• a wallet seed (docker/db/wallet-seed.sql) is loaded into the database on init
• wallet seed is also used to test migration
• wallet seed contains multiple wallets for user test_wallet_v2 (id 21001)
• login as user: sndev login test_wallet_v2
• passphrase to unlock wallets:

media fit youth secret combine live cupboard response enable loyal kitchen angle

Screenshots

we now show wallets instead of protocols:

each wallet can now support multiple protocols for send or receive:

Checklist

Are your changes backwards compatible? Please answer below:

no

On a scale of 1-10 how well and how have you QA'd this change and any features it might affect? Please answer below:

8

• migration was tested with wallet seed and prod db dump, manually inspecting the database to make sure the migrated data makes sense and logging in as various users (even though I can't unlock their wallets since they are encrypted)
• everything else was tested manually via the frontend (payments, wallet I/O, withdrawals, encryption & decryption, passphrase reset etc.)

For frontend changes: Tested on mobile, light and dark mode? Please answer below:

yes

Did you introduce any new environment variables? If so, call them out explicitly here:

no

[gitguardian]

⚠️ GitGuardian has uncovered 1 secret following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secret in your pull request

GitGuardian id	GitGuardian status	Secret	Commit	Filename
16991000	Triggered	Generic High Entropy Secret	c2c2f3a	docker/db/wallet-seed.sql	View secret

🛠 Guidelines to remediate hardcoded secrets

1. Understand the implications of revoking this secret by investigating where it is used in your code.
2. Replace and store your secret safely. Learn here the best practices.
3. Revoke and rotate this secret.
4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider

• following these best practices for managing and storing secrets including API keys and other credentials
• install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.

---

^{🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.}

[ekzyis]

I rebased this PR on 17aada6 to fix conflicts but I will stop rebasing now because @huumn is reviewing

[huumn]

Feel free to merge updates, but rebasing or forced pushes will make me grumpy. :D

[cursor]

Bug: Hook Array Access Without Validation

The useSendRecvParam and useWalletProtocolParam hooks perform unsafe array access on params.slug (at indices [1] and [2] respectively) without checking if params.slug exists or has sufficient length. This can lead to a TypeError if the URL structure is unexpected or params.slug is undefined.

wallets/client/components/forms.js#L292-L302

stacker.news/wallets/client/components/forms.js

Lines 292 to 302 in b69bada

	const params = useParams()
	// returns only :send in /wallets/:name/:send
	return ['send', 'receive'].includes(params.slug[1]) ? params.slug[1] : null
	}
	function useWalletProtocolParam () {
	const params = useParams()
	const name = params.slug[2]
	// returns only :protocol in /wallets/:name/:send/:protocol
	return name ? unurlify(name) : null
	}

Fix in Cursor • Fix in Web

---

Bug: Unhandled Null in Wallet Protocol Mutations

The getWalletProtocolMutation function can return null for unsupported protocol combinations (e.g., LN_ADDR send, CLN_REST send). This null value is passed directly to useMutation() within useWalletProtocolUpsert and to client.mutate() within useWalletMigrationMutation without validation, which will cause errors or unexpected behavior.

wallets/client/hooks/query.js#L120-L123

stacker.news/wallets/client/hooks/query.js

Lines 120 to 123 in b69bada

	export function useWalletProtocolUpsert (wallet, protocol) {
	const mutation = getWalletProtocolMutation(protocol)
	const [mutate] = useMutation(mutation)

wallets/client/hooks/query.js#L433-L437

stacker.news/wallets/client/hooks/query.js

Lines 433 to 437 in b69bada

	})
	}, [client, encryptConfig])
	return useMemo(() => ({ migrate, ready: ready && !loading }), [migrate, ready, loading])
	}

Fix in Cursor • Fix in Web

---

Was this report helpful? Give feedback by reacting with 👍 or 👎

[huumn]

So far this looks/works great - strict upgrade over v1. I've only done a very cursory QA/code review so far. I'll continue with my surface level exploration today, then go deep tomorrow on the important stuff:

1. db models
2. migrations
3. general organization of code/calls/etc

Questions so far

1. have you written a doc on how to add new wallets/protocols? it'll help me understand the code better and it'll be nice to have for anyone that wants to extend the code.
2. do we need templateId if each template has a unique name? It seems to make wallets.json a bit wonky having to know the insertion id (we have templateId, name, displayName) ... and if it's not an auto-incremented int, it seems like we should just use the unique template name. Perhaps it'd make sense to use an enum for template names too, and make that the primary key?
   • "names as primary keys" are discouraged because names beg to be changed and sometimes that's inconvenient but, in this case, I don't expect these names to change and we're referencing the templateId as if it has a known mapping to wallet name already ... also, if we do UPDATE ON CASCADE, these names should be pretty convenient to change (in the rare case where we need to) ... else we can deprecate a template and migrate to one with the new name
   • idk maybe I'm missing something though. do you have examples where using templateName instead of templateId will cause problems? I can only think of BLINK changing its name to SQUINT or something. What does that break when we change the name? Do we have to change the template name in that case?