[#730] feat: cache rework #742

chedieck · 2023-11-21T20:26:38Z

Related to #730

Description

Add cache for the balance of an address.
Better organize cache logic. We currently support 3 different caches (dashboard data, payment list and the address balance), this is now reflect into three files inside the redis/ directory plus a types.ts file with all cache interfaces. Furthermore, there is now two interfaces CacheSet and CacheGet on redis/index.ts which will be used all across the codebase to actually get or set the cache values. The CacheSet interfaces export functions that are named after the unique context they will be used: for example: CacheSet.paybuttonUpdate is what will run when a paybutton is updated (and never in any other situation).
Uses isSyncing new address column to show on the wallet card that one of its addresses is syncing
Fix out-of-date cache issues when new txs arrive.

Test plan

Experiment with having a big address added (I use ecash:qrcuqadqrzp2uztjl9wn5sthepkg22majylrprqkuk) and checking if the dashboard values, the balance shown in the wallet and the payment list all are updated live, as the transactions arrive.

Also, after the sync is done, check that these values are not changing anymore and load very fast. This should include the logic for creating a Paybutton, since it gets the data for wallets and this in master was bottlenecking paybutton creation when a wallet would have a lot of transactions.

The whole codebase should work well even with large addresses added to the account, except for the payments page which should still await for #741 to have a smooth behavior.

Klakurka

This is much better than what we have in master.

Here's the issues I've been able to find:

If you add an address with a boatload of txs, the Dashboard doesn't seem to get most of them:

Would be good to be able to differentiate between when there are 'No transactions.' and when it just needs a moment to grab them the first time. We have a separate task for this I'm pretty sure.

'Payments' page no longer has a scrollbar for some reason (although scrolling with the mouse or keyboard still works).

chedieck · 2023-11-23T12:35:12Z

If you add an address with a boatload of txs, the Dashboard doesn't seem to get most of them:

This seems to be true to the IFP_ADDRESSES, I created a commit that should fix it.

Would be good to be able to differentiate between when there are 'No transactions.' and when it just needs a moment to grab them the first time. We have a separate task for this I'm pretty sure.

Indeed. I found this issue #316 which is not too specific but we can make that clearer there.

'Payments' page no longer has a scrollbar for some reason (although scrolling with the mouse or keyboard still works).

This shall be fixed when I address the Payments page in detail in the next PR,

Klakurka

When loading up the IFP address, the dashboard shows the revenue changes (doubles) on Nov 18, but this actually happened on Nov 15... any ideas?

It's obviously not a time zone issues as it's 3 days apart.

Klakurka

You know what -- it actually looks totally fine. Due to the difficulty adjustment that took place until Nov 17, there were fewer than expected blocks then.

We can double check the numbers later but looks good at first glance.

balanceCache.ts

constants/index.ts

redis/balanceCache.ts

redis/dashboardCache.ts

redis/paymentCache.ts

services/paybuttonService.ts

Klakurka · 2023-11-23T18:10:38Z

services/paybuttonService.ts

      },
      include: includeAddresses
    })
    // Update cache for existent addreses


Better comment would be explaining when this happens.

I think this comment was more useful before the creation of CacheSet; removed.

I don't understand what would be the when you refer to, since this function will always update the cache

Nevermind -- but comments should be useful if they're going to be there so I'll leave it up to you what you want to do with it.

prisma/seed.ts

Klakurka

Probably just the one renaming and we're good.

chedieck · 2023-11-24T13:11:40Z

Probably just the one renaming and we're good.

Addressed.

chedieck added 30 commits November 16, 2023 14:36

chore: remove obsolete functions

74b837d

temp

49d7250

feat: clear cache for broadcasted week

7fd309a

fix: cache clearance, broadcast log

0993021

refactor: dashboardCache -> paymentCache

d6cb26a

feat: use address string, not id, on cache

f7963eb

feat: caching dashboard data

1701fb5

fix: mocked redis del

120ed55

feat: different api responses to payments & dashboard

723e61b

feat: clear dashboard cache on broadcast

1a7817b

feat: clear dashboard cache on paybutton update

58e9265

feat: fix up cache after button deletion

140ce2f

feat: removed debugging console.logs

50b5385

feat: add cache for address balance

c0dd56d

refactor: rename summarize -> cache

cf868ea

refactor: own file to balanceCache

cb0976b

feat: wallet balance can try to fetch cached addresses

6d1aaba

feat: cache address balance upon calculation

b09d965

feat: using cache in Wallet API call

de838bb

feat: syncing txs wallets ca

c4dbdfb

refactor: better variable name

5813b02

feat: reorganized cache functions and logic

04becac

feat: dedicated folder to cache types

f345aa8

feat: rework to use only CacheSet and CacheGet

94bf02e

fix: import of cache functions

66bdaa1

fix: update balance cache on tx insertions

2fd0d18

fix: remove unecessarsy parameter

991714a

test: fix tests for new cache behavior

5e7bba7

test: fix mocked redit get to get nothing

1435b30

feat: separate payments & dashboard APIs

2020210

Klakurka self-requested a review November 21, 2023 20:26

test: mock userProfile.findMany

480096f

Klakurka requested changes Nov 21, 2023

View reviewed changes

feat: comment out production address seeding logic

546d37a

chedieck force-pushed the feat/cache-rework branch from c5b7718 to 546d37a Compare November 23, 2023 12:36

Klakurka requested changes Nov 23, 2023

View reviewed changes

Klakurka approved these changes Nov 23, 2023

View reviewed changes

Klakurka requested changes Nov 23, 2023

View reviewed changes

chedieck added 4 commits November 23, 2023 15:18

feat: remove IFP addresses commented

965e0a0

chore: remove accidentally commited files

5e1c4c7

chore: remove obsolete comment

ea888a1

chore: remove commented seed code

63ee696

chedieck requested a review from Klakurka November 23, 2023 18:52

Klakurka requested changes Nov 23, 2023

View reviewed changes

refactor: IFP_ADDRESSES -> BLOCKED_ADDRESSES

cc44582

Klakurka approved these changes Nov 24, 2023

View reviewed changes

Klakurka merged commit 27a2104 into master Nov 24, 2023

[#730] feat: cache rework #742

[#730] feat: cache rework #742

Uh oh!

Conversation

chedieck commented Nov 21, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Test plan

Uh oh!

Klakurka left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chedieck commented Nov 23, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Klakurka left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Klakurka left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Klakurka Nov 23, 2023

Choose a reason for hiding this comment

Uh oh!

chedieck Nov 23, 2023

Choose a reason for hiding this comment

Uh oh!

Klakurka Nov 23, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Klakurka left a comment

Choose a reason for hiding this comment

Uh oh!

chedieck commented Nov 24, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

chedieck commented Nov 21, 2023 •

edited

Loading

Klakurka left a comment •

edited

Loading

chedieck commented Nov 23, 2023 •

edited

Loading

Klakurka left a comment •

edited

Loading