Sub 1s loading UI #169

Keyrxng · 2024-02-20T16:50:07Z

Resolves #168

The issue with things as far as I could see was that the UI rendering was being held back due to the laggy RPC calls and you'd see this:

hold_my_brewery.mp4

Although using the static data available through the permit and hardcoded token info we can see this instead:

hold_my_beer.mp4

It has involved moving around a few parts but the new interval for the loading ui rendering is sub 1s

With a forked foundry instance, I was trying to replicate the stalling offline to see if I could squeeze a tx through while the stalling rpc calls takes its time but I'm not able to replicate it and work with real permits as local anvil hits instantly everytime.

But thinking about it, you should be able to as mm will be using a different rpc/provider instance so long as the it's not locking down the UI and a user can click they should be able to make a claim with or without the treasuryInfo

…om-db

…rls-from-db # Conflicts: # build/esbuild-config.ts # package.json # static/scripts/audit-report/audit.ts # static/scripts/audit-report/helpers.ts # yarn.lock

0x4007 · 2024-02-20T16:54:45Z

Your code is outdated. Pull from head

Keyrxng · 2024-02-20T17:18:32Z

lmao I'll get there sooner or later with Git 😂

feat: call db to retrieve permits instead of parsing github comments

0x4007 · 2024-02-21T13:36:29Z

lib/chainlist

Are you sure you're up to date? If you are then you might need to rebase or something because its unlikely that you went out of your way to remove this module.

ubiquibot-continuous-deploys · 2024-02-21T16:47:44Z

29a1da8

6f3bfdf

16a534e

Keyrxng · 2024-02-21T16:48:15Z

Optimistic rendering time till claimable:

Non-optimistic rendering time till claimable:

Keyrxng · 2024-02-21T16:51:25Z

had a nightmare trying to get that submodule added, thats what caused my initial issues it was crashing my gh desktop app and would not pull/update or replace because it existed in the index no matter how many time I removed it from cache and the git file, nightmare but got there in the end

Keyrxng · 2024-02-21T17:12:01Z

The rpc speed mapping is cool but I think it's a bit overkill pinging that many rpcs just to fetch a balance and allowance lmao especially if it's a mainnet tx, hitting 49 RPCs is nuts, 15 for gnosis is still pretty wild

Looking at the privacy statements within extraRpcs.js some of them look pretty scary with all of the data that are collecting esp when you consider the maxi mindset

A task could be to read through them and exclude specific rpcs and lean towards those that collect no data or only onchain data and use the extraRPCs in other areas with direct on-chain writes as opposed to just in this case two reads

rndquu · 2024-02-21T19:48:14Z

local anvil hits instantly everytime

As far as I remember when you use a forked anvil instance and send a transaction then it is not mined instantly but waits for a new block from a forked network. Anyway there is also this option since anvil supports hardhat's RPC requiest.

Keyrxng · 2024-02-21T19:56:10Z

It's not determined by a block being mined as it involves no onchain writes, it's hitting 127.0.0.1:8545 with a read op to getChainId, followed by two reads for balance and allowance. Afaik you can't stop anvil/hardhat from responding to reads but you could obv cause a write op to 'stall' if you disable auto mine

rndquu · 2024-02-21T20:00:27Z

It's not determined by a block being mined as it involves no onchain writes, it's hitting 127.0.0.1:8545 with a read op to getChainId, followed by two reads for balance and allowance. Afaik you can't stop anvil/hardhat from responding to reads but you could obv cause a write op to 'stall' if you disable auto mine

I see, there is also the --no-storage-caching option which will add some delay to read operations but obviously it would be better to have a separate delay option for onchain read operations

Keyrxng · 2024-02-21T20:08:35Z

It's not determined by a block being mined as it involves no onchain writes, it's hitting 127.0.0.1:8545 with a read op to getChainId, followed by two reads for balance and allowance. Afaik you can't stop anvil/hardhat from responding to reads but you could obv cause a write op to 'stall' if you disable auto mine

I see, there is also the --no-storage-caching option which will add some delay to read operations but obviously it would be better to have a separate delay option for onchain read operations

I just started looking over some of the flags, often updating and changing plus I've forgotten most I think lmao but that might help test the edge case but either way the UI currently loads quicker than the metamask popup so trying to squeeze a tx through while bal and allowance and fetching is very unlikely and if already connected via mm before page load it still shouldn't interfere with the fetching as its two distinct providers for reads and writes

0x4007 · 2024-02-21T20:17:31Z

I got it under one second on my branch. Just fixing some details around refactoring the code

Keyrxng · 2024-02-21T20:21:50Z

As did I without the additional 15 rpc calls however it's a good fallback function to have if writes are timing out or whatever but for two reads it's heavy artillery 😂

0x4007 · 2024-02-22T09:55:29Z

As did I without the additional 15 rpc calls however it's a good fallback function to have if writes are timing out or whatever but for two reads it's heavy artillery 😂

My plan is to combine both. Unfortunately I did a large refactor here (again) breaking apart large functions into seperate files. I anticipate some more merge conflicts. Let me get mine merged in first and then we can merge yours in?

0x4007 · 2024-02-22T22:32:36Z

I was going to merge yours in but I realized that its not stable either due to a combination of my bad commit and the refactoring being incompatible with some pulls getting merged at the same time. Can you get this stable so we can use it for production? I need more time to fix my branch stability.

Keyrxng · 2024-02-23T03:42:07Z

I'm a little confused as to what you mean by it's not currently stable. I'm seeing no errors and was able to successfully claim on 29a1da8 , the UI near instantly loads and there are no conflicts

I realise that I'm merging against ubiquity:main and not into ubiquity:development might that be part of the issue? lmk what the best thing to do would be whether to merge your refactor branch into here, vice-versa or whether to open a new pr against development, cheers

0x4007 · 2024-02-23T08:33:14Z

I'm a little confused as to what you mean by it's not currently stable. I'm seeing no errors and was able to successfully claim on 29a1da8 , the UI near instantly loads and there are no conflicts

It seems unstable when I refresh several times and press claim. If you set up a test wallet you can generate real permits with you can test almost the entire flow up until you confirm the transaction. You will see it only works some times. I assume there is an issue with some of the RPCs that need to be taken into account.

I realise that I'm merging against ubiquity:main and not into ubiquity:development might that be part of the issue?

I don't think so but you should merge to development. It was my mistake because I briefly set main as the default branch but I realize given the pace of progress, stability might not be 100% so I switched the default back to development. After manual testing we can merge to main which automatically deploys to pay.ubq.fi

lmk what the best thing to do would be whether to merge your refactor branch into here, vice-versa or whether to open a new pr against development, cheers

You just hit edit on the top right and change the merge target to development. Either way though the intermittent issues I mentioned I saw on your latest commit.

Keyrxng · 2024-02-23T12:14:42Z

Hitting localhost I generated 10 permits with an open window each, refreshed a few times on each and claimed to completion and all I could see was a little layout shift to the right within the claim box.

Using the networkRPCs the claim button flickers when an RPC call fails, specifically "https://xdai-archive.blockscout.com" so when that's blacklisted the claim button stops flickering but the layout shift is still there

0x4007 · 2024-02-23T22:11:41Z

Let me spend some time testing today and I'll be careful to clear cache etc to ensure that everything works as expected.

I'll see if I can post proof of the specific error I remember receiving.

Thanks for testing!

fix: continuous deploys

0x4007 · 2024-02-23T22:07:38Z

build/esbuild-build.ts

+  const officialUrls = extraRpcs[networkId].rpcs.filter((rpc) => {
+    if (typeof rpc === "string") {
+      if (blacklist.includes(rpc)) {
+        return "";


That doesn't look right. You should not return a truthy value if you want to filter it out.

0x4007 · 2024-02-23T22:08:49Z

scripts/typescript/generate-permit2-url.ts

+export async function generateMultiERC20Permits() {
+  for (let i = 0; i < 5; i++) {
+    const url = await generate(true);
+    log.ok("Testing URL:");
+    console.log(url);
+  }
 }


I think it makes sense to put this in a local test file?

0x4007 · 2024-02-25T22:31:33Z

The error that I got is from.

erc20-permit.ts:67 Error: could not detect network (event="noNetwork", code=NETWORK_ERROR, version=providers/5.7.2)

Keyrxng · 2024-02-26T08:13:43Z

This resolves the no network issue.

I test by setting the CHAIN_ID in .env to 31337 and then forking gnosis using flag --chain-id 31337 and then when testing against the actual RPCs I do the same but setting the chain ids to 100

0x4007 · 2024-02-26T11:07:04Z

build/esbuild-build.ts

@@ -14,13 +14,13 @@ export const entries = [...typescriptEntries, ...cssEntries];
 const allNetworkUrls: Record<string, string[]> = {};
 // this flattens all the rpcs into a single object, with key names that match the networkIds. The arrays are just of URLs per network ID.

-const blacklist = ["https://xdai-archive.blockscout.com"];
+const blacklist = ["https://xdai-archive.blockscout.com", "https://gnosis.api.onfinality.io/public"];


This blacklist idea doesn't seem like a robust approach. We can merge it in but it should be handled dynamically. I think I implemented this in an unstable manner on my branch, I vaguely recall.

Since the issue is that the claim button flickers on a failure then finding out why exactly that's happening because of a failed background call would probs be the best way to handle it I guess, otherwise just try{}catch() and don't log the error? seems hacky we could care less if one or two calls have failed

It's because the CSS/style logic is implemented poorly. I tore it apart in my refactor but didn't get to put it all together in a totally stable way (mostly the other RPC related functions- I believe the styling code was significantly simplified and stable if I recall correctly).

Unfortunately I'm running very low on code time because I'll need to focus on fundraising starting next week. So I'm just going to be focusing on top priority stuff.

Luckily for me I've got nothing but code time, regretfully it does eventually require that you have review time lmao but I'll keep on pushing forward with things as best I can in the meantime, cheers for the heads up

0x4007 · 2024-02-26T11:10:32Z

static/scripts/rewards/web3/wallet.ts

@@ -33,8 +33,7 @@ export async function handleNetwork(desiredNetworkId: number) {
    invalidateButton.disabled = true;
  }

-  const network = await web3provider.getNetwork();
-  const currentNetworkId = network.chainId;
+  const currentNetworkId = (await web3provider.getNetwork()).chainId;


Why did you recombine this?

My bad I thought I had replied to this, it was not intentional I can only assume it was from my merging of branches and it's been an oversight to spreading it out again

static/scripts/rewards/web3/erc20-permit.ts

0x4007 · 2024-02-26T11:11:35Z

static/scripts/rewards/web3/erc20-permit.ts

+  await insertErc20PermitTableData(permit, provider, symbol, decimals, table);
+}
+
+export async function fetchTreasury(contractAddr: string, owner: string, provider: JsonRpcProvider) {


More typesafe to define the return type up here via

export async function fetchTreasury(contractAddr: string, owner: string, provider: JsonRpcProvider): { balance: BigNumber; allowance: BigNumber } {

I'll catch this and the rest in the sync ERC pr

Something I wasn't able to produce before, spamming refresh there are a few failed calls maybe 3 or 4 and the balance and allowance render as 0, I'll see if I can get that too

…ements" This reverts commit 123f178, reversing changes made to 7d88536.

gentlementlegen and others added 11 commits February 17, 2024 12:30

feat: call db to retrieve permits instead of parsing github

35d0b99

chore: issue number now uses the value from git get request

7528f3e

Merge branch 'development' into permit-urls-from-db

2b21d66

chore: fixed merged characters

655ead5

Merge remote-tracking branch 'origin/development' into permit-urls-fr…

72b23f2

…om-db

Merge remote-tracking branch 'pay.ubiquity/development' into permit-u…

3b7c3d2

…rls-from-db # Conflicts: # build/esbuild-config.ts # package.json # static/scripts/audit-report/audit.ts # static/scripts/audit-report/helpers.ts # yarn.lock

chore: merged development branch

2f11234

chore: added Action secrets to the build.yml

0878552

feat: sub 1s load time

f5e3a80

fix: conflicts

ebd54e1

chore: fetch inside try

7fab1b4

Keyrxng requested review from 0x4007 and rndquu as code owners February 20, 2024 16:50

Keyrxng added 3 commits February 20, 2024 17:07

feat: sub 1s load time

f1ca988

chore: fetch inside try

97fb13c

fix: revert and pull from head

2adf64b

gentlementlegen and others added 5 commits February 21, 2024 10:33

chore: renamed supabase key to SUPABASE_PUBLIC_ANON_KEY

a1c6e4b

chore: renamed supabase key to SUPABASE_ANON_KEY

0ba50af

chore: allow personal repo to be searched

5a504fd

chore: renamed supabase env variables

ba87b12

Merge pull request ubiquity#164 from FernandVEYRIER/permit-urls-from-db

53ab0ec

feat: call db to retrieve permits instead of parsing github comments

0x4007 reviewed Feb 21, 2024

View reviewed changes

0x4007 and others added 3 commits February 21, 2024 22:47

Merge branch 'fix/perf' into development

b592128

Merge branch 'ubiquity:development' into speed-improvements

c0249df

fix: use allnetworkurls

517dd48

fix: rpc blacklist

1fe025d

0x4007 and others added 2 commits February 25, 2024 16:58

fix: continuous deploys

fd9a8e5

Merge pull request ubiquity#176 from pavlovcik/fix/cd

24284bd

fix: continuous deploys

0x4007 reviewed Feb 25, 2024

View reviewed changes

Keyrxng added 2 commits February 26, 2024 07:22

Merge branch 'ubiquity:development' into speed-improvements

6f3bfdf

fix: no network resolved

16a534e

0x4007 reviewed Feb 26, 2024

View reviewed changes

0x4007 merged commit 123f178 into ubiquity:main Feb 26, 2024
1 check passed

0x4007 reviewed Feb 26, 2024

View reviewed changes

static/scripts/rewards/web3/erc20-permit.ts Show resolved Hide resolved

0x4007 reviewed Feb 26, 2024

View reviewed changes

ubiquibot bot mentioned this pull request Feb 26, 2024

Reduce loading time #168

Closed

This was referenced Feb 26, 2024

Serialization of HTML Comments ubiquibot/comment-incentives#17

Open

Implement Dynamic RPC pick handler ubiquity/rpc-handler#3

Closed

0x4007 added a commit to 0x4007/pay.ubq.fi that referenced this pull request Mar 9, 2024

Revert "Merge pull request ubiquity#169 from ubq-testing/speed-improv…

f672dc0

…ements" This reverts commit 123f178, reversing changes made to 7d88536.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sub 1s loading UI #169

Sub 1s loading UI #169

Keyrxng commented Feb 20, 2024

0x4007 commented Feb 20, 2024

Keyrxng commented Feb 20, 2024

0x4007 Feb 21, 2024

ubiquibot-continuous-deploys bot commented Feb 21, 2024 •

edited

Keyrxng commented Feb 21, 2024

Keyrxng commented Feb 21, 2024

Keyrxng commented Feb 21, 2024 •

edited

rndquu commented Feb 21, 2024

Keyrxng commented Feb 21, 2024

rndquu commented Feb 21, 2024 •

edited

Keyrxng commented Feb 21, 2024

0x4007 commented Feb 21, 2024 •

edited

Keyrxng commented Feb 21, 2024

0x4007 commented Feb 22, 2024

0x4007 commented Feb 22, 2024

Keyrxng commented Feb 23, 2024 •

edited

0x4007 commented Feb 23, 2024

Keyrxng commented Feb 23, 2024

0x4007 commented Feb 23, 2024

0x4007 Feb 23, 2024

0x4007 Feb 23, 2024

0x4007 commented Feb 25, 2024 •

edited

Keyrxng commented Feb 26, 2024

0x4007 Feb 26, 2024

Keyrxng Feb 26, 2024 •

edited

0x4007 Feb 26, 2024 •

edited

Keyrxng Feb 26, 2024

0x4007 Feb 26, 2024

0x4007 Feb 28, 2024

Keyrxng Feb 28, 2024

0x4007 Feb 26, 2024

Keyrxng Feb 26, 2024

Keyrxng Feb 26, 2024

Sub 1s loading UI #169

Sub 1s loading UI #169

Conversation

Keyrxng commented Feb 20, 2024

0x4007 commented Feb 20, 2024

Keyrxng commented Feb 20, 2024

Choose a reason for hiding this comment

ubiquibot-continuous-deploys bot commented Feb 21, 2024 • edited

Keyrxng commented Feb 21, 2024

Keyrxng commented Feb 21, 2024

Keyrxng commented Feb 21, 2024 • edited

rndquu commented Feb 21, 2024

Keyrxng commented Feb 21, 2024

rndquu commented Feb 21, 2024 • edited

Keyrxng commented Feb 21, 2024

0x4007 commented Feb 21, 2024 • edited

Keyrxng commented Feb 21, 2024

0x4007 commented Feb 22, 2024

0x4007 commented Feb 22, 2024

Keyrxng commented Feb 23, 2024 • edited

0x4007 commented Feb 23, 2024

Keyrxng commented Feb 23, 2024

0x4007 commented Feb 23, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

0x4007 commented Feb 25, 2024 • edited

Keyrxng commented Feb 26, 2024

Choose a reason for hiding this comment

Keyrxng Feb 26, 2024 • edited

Choose a reason for hiding this comment

0x4007 Feb 26, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ubiquibot-continuous-deploys bot commented Feb 21, 2024 •

edited

Keyrxng commented Feb 21, 2024 •

edited

rndquu commented Feb 21, 2024 •

edited

0x4007 commented Feb 21, 2024 •

edited

Keyrxng commented Feb 23, 2024 •

edited

0x4007 commented Feb 25, 2024 •

edited

Keyrxng Feb 26, 2024 •

edited

0x4007 Feb 26, 2024 •

edited