Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BLOCK-40: Add indexer to the SDK #3741

Closed
wants to merge 50 commits into from
Closed

BLOCK-40: Add indexer to the SDK #3741

wants to merge 50 commits into from

Conversation

AmineAfia
Copy link
Contributor

@AmineAfia AmineAfia commented Jul 18, 2024

Problem solved

Short description of the bug fixed or feature added

Changes made

  • Add indexer functionality to the SDK

How to test

  • invoke all new functions in a the dashboard

Contributor NFT

Paste in your wallet address below and we will airdrop you a special NFT when your pull request is merged.

Address:


PR-Codex overview

This PR adds pagination functionality to Chainsaw API requests, introduces new hooks for fetching NFTs and transactions, and updates Chainsaw endpoints and URLs.

Detailed summary

  • Added pagination for Chainsaw API requests
  • Introduced useGetOwnedNFTs and useGetTransactions hooks
  • Updated Chainsaw endpoints and URLs
  • Added default Chainsaw URL configuration
  • Implemented functions to fetch block data and latest block number

The following files were skipped due to too many changes: packages/thirdweb/src/chainsaw/endpoints/getBlock.ts, packages/thirdweb/src/chainsaw/endpoints/getNFTsByOwner.ts, packages/thirdweb/src/chainsaw/types.ts, packages/thirdweb/src/chainsaw/endpoints/getNFTsByCollection.ts, packages/thirdweb/src/chainsaw/endpoints/getTransactions.ts, packages/thirdweb/src/chainsaw/formatter.ts, packages/thirdweb/src/chainsaw/endpoints/getEvents.ts, packages/thirdweb/src/react/web/ui/ConnectWallet/screens/ViewNFTs.tsx, apps/dashboard/src/contract-ui/tabs/transactions/components/transactions-feed.tsx

✨ Ask PR-Codex anything about this PR by commenting with /codex {your question}

Copy link

vercel bot commented Jul 18, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
docs-v2 ✅ Ready (Inspect) Visit Preview 💬 Add feedback Aug 7, 2024 11:38pm
thirdweb_playground ✅ Ready (Inspect) Visit Preview 💬 Add feedback Aug 7, 2024 11:38pm
thirdweb-www 🛑 Canceled (Inspect) Aug 7, 2024 11:38pm
wallet-ui ✅ Ready (Inspect) Visit Preview 💬 Add feedback Aug 7, 2024 11:38pm

Copy link

linear bot commented Jul 18, 2024

Copy link

changeset-bot bot commented Jul 18, 2024

⚠️ No Changeset found

Latest commit: 1871493

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Copy link

graphite-app bot commented Jul 18, 2024

Your org requires the Graphite merge queue for merging into main

Add the label “merge-queue” to the PR and Graphite will automatically add it to the merge queue when it’s ready to merge. Or use the label “hotfix” to add to the merge queue as a hot fix.

You must have a Graphite account and log in to Graphite in order to use the merge queue. Sign up using this link.

@github-actions github-actions bot added the TS SDK Involves changes to the v5 TypeScript SDK. label Jul 18, 2024
@AmineAfia AmineAfia changed the title BLOCK-40: initial indexer structure BLOCK-40: Add indexer to the SDK Jul 18, 2024
Copy link

codspeed-hq bot commented Jul 18, 2024

CodSpeed Performance Report

Merging #3741 will not alter performance

Comparing indexer-v1 (5309650) with main (9c292a4)

Summary

✅ 9 untouched benchmarks

Copy link
Member

@jnsdls jnsdls left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some drive-by comments and callouts / ideas :)

Comment on lines 1 to 6
export { getIndexerClient } from "../indexer/indexer";
export { getBlocks } from "../indexer/endpoints/getBlocks";
export { getEvents } from "../indexer/endpoints/getEvents";
export { getLatestBlock } from "../indexer/endpoints/getLatestBlock";
export { getReceipts } from "../indexer/endpoints/getReceipts";
export { getNFTs } from "../indexer/endpoints/getNFTs";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if these should be explicit exports or if we should instead use them "under the hood" in less indexer-specific places. I guess there is always a likely requirement to also explicitly be able to ask the indexer for data

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can both export and use them under the hood

packages/thirdweb/src/indexer/indexer.ts Outdated Show resolved Hide resolved
export async function getReceipts(client: any, params: any) {
const blocksResponse = await client(params);
// TODO: Calrify structured return type through the indexer
return await blocksResponse.json();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

at first thought this should (maybe?) return the same type as the RPC method? that way it can be used interchangeably? Maybe there's additional optional fields?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Converted transactions and blocks to the closest possible formats with extended data

Copy link

codecov bot commented Jul 31, 2024

Codecov Report

Attention: Patch coverage is 57.40072% with 354 lines in your changes missing coverage. Please review.

Project coverage is 59.19%. Comparing base (1f3448e) to head (661b61c).
Report is 47 commits behind head on main.

Files Patch % Lines
packages/thirdweb/src/chainsaw/formatter.ts 16.41% 112 Missing ⚠️
...kages/thirdweb/src/chainsaw/endpoints/getEvents.ts 69.53% 39 Missing ⚠️
...dweb/src/chainsaw/endpoints/getNFTsByCollection.ts 67.61% 34 Missing ⚠️
...rc/react/web/ui/ConnectWallet/screens/ViewNFTs.tsx 8.10% 34 Missing ⚠️
...thirdweb/src/chainsaw/endpoints/getTransactions.ts 69.72% 33 Missing ⚠️
.../thirdweb/src/chainsaw/endpoints/getNFTsByOwner.ts 69.47% 29 Missing ⚠️
...web/src/chainsaw/endpoints/getLatestBlockNumber.ts 67.64% 22 Missing ⚠️
...ckages/thirdweb/src/chainsaw/endpoints/getBlock.ts 69.11% 21 Missing ⚠️
packages/thirdweb/src/chainsaw/paging.ts 21.42% 11 Missing ⚠️
...eact/web/ui/ConnectWallet/hooks/useGetOwnedNFTs.ts 44.44% 10 Missing ⚠️
... and 2 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3741      +/-   ##
==========================================
- Coverage   60.92%   59.19%   -1.73%     
==========================================
  Files         962      677     -285     
  Lines       77531    67442   -10089     
  Branches     3680     2032    -1648     
==========================================
- Hits        47237    39925    -7312     
+ Misses      29613    27466    -2147     
+ Partials      681       51     -630     
Flag Coverage Δ
legacy_packages ?
packages 59.19% <57.40%> (-0.96%) ⬇️
Files Coverage Δ
packages/thirdweb/src/utils/domains.ts 81.25% <87.50%> (+0.69%) ⬆️
packages/thirdweb/src/chainsaw/urls.ts 82.97% <82.97%> (ø)
...eact/web/ui/ConnectWallet/hooks/useGetOwnedNFTs.ts 44.44% <44.44%> (ø)
packages/thirdweb/src/chainsaw/paging.ts 21.42% <21.42%> (ø)
...ckages/thirdweb/src/chainsaw/endpoints/getBlock.ts 69.11% <69.11%> (ø)
...web/src/chainsaw/endpoints/getLatestBlockNumber.ts 67.64% <67.64%> (ø)
.../thirdweb/src/chainsaw/endpoints/getNFTsByOwner.ts 69.47% <69.47%> (ø)
...thirdweb/src/chainsaw/endpoints/getTransactions.ts 69.72% <69.72%> (ø)
...dweb/src/chainsaw/endpoints/getNFTsByCollection.ts 67.61% <67.61%> (ø)
...rc/react/web/ui/ConnectWallet/screens/ViewNFTs.tsx 10.95% <8.10%> (+0.32%) ⬆️
... and 2 more

... and 333 files with indirect coverage changes

Copy link
Contributor

github-actions bot commented Jul 31, 2024

size-limit report 📦

Path Size Loading time (3g) Running time (snapdragon) Total time
thirdweb (esm) 43.97 KB (+1.97% 🔺) 880 ms (+1.97% 🔺) 891 ms (-4.86% 🔽) 1.8 s
thirdweb (cjs) 92.49 KB (+1.18% 🔺) 1.9 s (+1.18% 🔺) 2.9 s (+27.37% 🔺) 4.7 s
thirdweb (minimal + tree-shaking) 4.82 KB (+0.23% 🔺) 97 ms (+0.23% 🔺) 87 ms (-29.47% 🔽) 183 ms
thirdweb/chains (tree-shaking) 498 B (+1.22% 🔺) 10 ms (0%) 26 ms (+17.03% 🔺) 36 ms
thirdweb/react (minimal + tree-shaking) 13.63 KB (-0.3% 🔽) 273 ms (-0.3% 🔽) 112 ms (-24.48% 🔽) 385 ms

@gregfromstl gregfromstl removed the Dashboard Involves changes to the Dashboard. label Aug 5, 2024 — with Graphite App
@iuwqyir iuwqyir requested a review from jnsdls August 6, 2024 18:30
Copy link
Member

@jnsdls jnsdls left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pretty close

a couple of high level observations (in addition to inline comments)

  1. this should be split into 2 parts (stack it?): SDK changes + dashboard changes - makes reviewing easier and we can ship it incrementally
  2. note to self to add a lint rule to enforce avoiding const foo () => .... function definitions whenever possible (mostly style)
  3. do not use .d.ts files if u do not have to
  4. consider chainsaw support - any "sdk method" that is not clearly a "chainsaw" method has to work across all EVM chain (looking at useOwnedNFTs())
  5. is production deployed? If we merge this will it work?
  6. can we add some tests to formatting etc?
  7. can we find a way to use the same return types for "block" "transaction" etc that are returned from existing SDK methods?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do not use .d.ts files in src - this is completely fine to be a .ts file

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 584ae4c

groupBy?: GetNFTsGroupBy;
} & ChainsawPagingParams;

export type GetNFTsByCollectionResult = {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for live data you probably need to use cursor based pagination but I agree that in the short term this will likely be OK

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do all these formatters in end result output the same data type as if I directly called RPC using the SDK?

if not is that a technical limitation or can we get it there?

@@ -0,0 +1,14 @@
import type { ChainsawPagingParams } from "./types.js";

export const addPagingToRequest = (
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

prefer function foo over const foo () => wherever possible

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: export function addRequestPagination(...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in a9dd446 and 0533dd2

Just out of curiosity, is there a specific reason to prefer function?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nvm, saw the mostly style comment

packages/thirdweb/src/chainsaw/urls.ts Outdated Show resolved Hide resolved
packages/thirdweb/src/chainsaw/urls.ts Outdated Show resolved Hide resolved
type GetLatestBlockNumberParams,
} from "../chainsaw/endpoints/getLatestBlockNumber.js";

export type {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we unify the return types? is there a reason that return types (not input params) have to be different than existing block transaction etc types?

return useQuery<ChainsawNFTs, Error>({
queryKey: ["useGetOwnedNFTs", params],
queryFn: async () => {
const result = await getNFTsByOwner(params);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what about chains without chainsaw support?

queryFn: () =>
fetchNFTs(props.client, nft.chain, nft.address, activeAccount?.address),
})),
const { data, isLoading } = useGetOwnedNFTs({
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what about chains without chainsaw support?

@vercel vercel bot temporarily deployed to Preview – docs-v2 August 7, 2024 22:17 Inactive
@vercel vercel bot temporarily deployed to Preview – wallet-ui August 7, 2024 22:17 Inactive
@vercel vercel bot temporarily deployed to Preview – thirdweb_playground August 7, 2024 22:17 Inactive
@github-actions github-actions bot added the Dashboard Involves changes to the Dashboard. label Aug 7, 2024
iuwqyir and others added 4 commits August 8, 2024 02:03
Co-authored-by: Jonas Daniels <jonas.daniels@outlook.com>
Signed-off-by: Toomas Oosalu <toomas.oosalu@gmail.com>
Co-authored-by: Jonas Daniels <jonas.daniels@outlook.com>
Signed-off-by: Toomas Oosalu <toomas.oosalu@gmail.com>
Co-authored-by: Jonas Daniels <jonas.daniels@outlook.com>
Signed-off-by: Toomas Oosalu <toomas.oosalu@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Dashboard Involves changes to the Dashboard. TS SDK Involves changes to the v5 TypeScript SDK.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants