Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(next): add some cache primitives to data fetching #4375

Merged
merged 90 commits into from
Jun 28, 2023
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
90 commits
Select commit Hold shift + click to select a range
848623f
add some caching stuff
juliusmarminge May 17, 2023
cc73466
revalidation
juliusmarminge May 17, 2023
a3a508c
Merge branch 'main' into next-fetch
KATT May 17, 2023
7fc8c52
Merge branch 'main' into next-fetch
juliusmarminge May 18, 2023
182a5ce
fetch -> http & experimental_
juliusmarminge May 18, 2023
130ed27
more http
juliusmarminge May 18, 2023
d627cef
Merge branch 'main' into next-fetch
juliusmarminge May 19, 2023
67361e3
rev
juliusmarminge May 19, 2023
5856850
shared types are ahrd
juliusmarminge May 19, 2023
a3c3a90
Merge branch 'main' into next-fetch
juliusmarminge May 25, 2023
3a8d7f6
types
juliusmarminge May 25, 2023
79e36da
fix
juliusmarminge May 25, 2023
a5b53c1
remove revalidate on client
juliusmarminge May 25, 2023
8c06a04
some busted caching going on here
juliusmarminge May 25, 2023
d8a5bb8
add staleTime option
juliusmarminge May 25, 2023
c6c5aca
staleTIme -> revalidate
juliusmarminge May 25, 2023
66a1f60
on http too
juliusmarminge May 25, 2023
d55e678
working ish
juliusmarminge May 25, 2023
da48d0f
http revalidate kinda bork
juliusmarminge May 25, 2023
9507328
override
juliusmarminge May 25, 2023
2d5fa7f
whoops
juliusmarminge May 25, 2023
ae33ca7
Co-authored-by: Alex / KATT <alexander@n1s.se>
juliusmarminge May 26, 2023
8fd76bb
md
juliusmarminge May 26, 2023
9248d08
WTF IS THIS EVEN
KATT May 26, 2023
b54bf23
something
KATT May 26, 2023
43d8523
push
KATT May 26, 2023
5fda004
wip
KATT May 26, 2023
be0dcf3
somehow it works
KATT May 26, 2023
14637a3
cool this works
KATT May 26, 2023
5aa6160
explore
KATT May 26, 2023
8772191
cool
KATT May 26, 2023
1564806
Merge branch 'main' into next-fetch
juliusmarminge Jun 1, 2023
95f14a2
alrgiht
juliusmarminge Jun 1, 2023
669f3eb
fix
juliusmarminge Jun 1, 2023
5fe4af2
small fixes
juliusmarminge Jun 1, 2023
8116769
ts prune
juliusmarminge Jun 1, 2023
4561fd0
add playwright
juliusmarminge Jun 1, 2023
85b0c6e
prettier
juliusmarminge Jun 1, 2023
a100636
push
KATT Jun 1, 2023
30ffca2
Merge branch 'main' into next-fetch
juliusmarminge Jun 16, 2023
557f2a2
merge main
juliusmarminge Jun 16, 2023
4d3304a
revert client stuff
juliusmarminge Jun 16, 2023
bcb0c4f
rm unused
juliusmarminge Jun 16, 2023
3acbd00
eh
juliusmarminge Jun 16, 2023
3857708
Merge branch 'main' into next-fetch
juliusmarminge Jun 19, 2023
6bb38c9
Merge branch 'main' into next-fetch
kodiakhq[bot] Jun 19, 2023
6ff0580
tests
juliusmarminge Jun 22, 2023
5feca23
pushy push
juliusmarminge Jun 22, 2023
a50449e
Merge branch 'main' into next-fetch
kodiakhq[bot] Jun 22, 2023
57d6857
rm bad header
KATT Jun 22, 2023
f9018ae
Merge branch 'next-fetch' of github.com:trpc/trpc into next-fetch
KATT Jun 22, 2023
2e3de22
lgtm
KATT Jun 22, 2023
735b35d
cleanup
juliusmarminge Jun 22, 2023
eddd0b5
more fixies
juliusmarminge Jun 22, 2023
728df68
enable teest
juliusmarminge Jun 22, 2023
86372e7
rm unused
juliusmarminge Jun 22, 2023
36a2f7a
nit
juliusmarminge Jun 22, 2023
21be3b0
add a "fullstack" example
juliusmarminge Jun 22, 2023
fc0d92e
rm md
juliusmarminge Jun 22, 2023
99429e2
revert some test stuff
juliusmarminge Jun 22, 2023
ea77f66
handle serialization
juliusmarminge Jun 22, 2023
2600092
Merge branch 'main' into next-fetch
kodiakhq[bot] Jun 22, 2023
9c7d162
Merge branch 'main' into next-fetch
kodiakhq[bot] Jun 22, 2023
cb52473
I GUESS WE'RE NOT BUMPING NEXT THEN GAARRARARAAR
juliusmarminge Jun 22, 2023
94e8c8e
here is fine i guess
juliusmarminge Jun 22, 2023
b908ff5
Merge branch 'main' into next-fetch
juliusmarminge Jun 22, 2023
1793e8d
fix workspaces
juliusmarminge Jun 22, 2023
28ebada
fix lock
juliusmarminge Jun 22, 2023
1462fa8
fix lock again
juliusmarminge Jun 22, 2023
a23873d
revert example
juliusmarminge Jun 22, 2023
8e945cb
fix manypkg
juliusmarminge Jun 22, 2023
813df40
revert
juliusmarminge Jun 22, 2023
5b087a4
fix lock
juliusmarminge Jun 22, 2023
d7d1047
dont need my env
juliusmarminge Jun 22, 2023
536a649
ignore
juliusmarminge Jun 22, 2023
25aaafe
eh
juliusmarminge Jun 22, 2023
3f9dbd0
Merge branch 'main' into next-fetch
kodiakhq[bot] Jun 23, 2023
01b6d43
Merge branch 'main' into next-fetch
kodiakhq[bot] Jun 23, 2023
3f918e6
bump next
juliusmarminge Jun 26, 2023
0205e41
Merge branch 'main' into next-fetch
kodiakhq[bot] Jun 26, 2023
8acf01c
Merge branch 'main' into next-fetch
kodiakhq[bot] Jun 26, 2023
098f091
Merge branch 'main' into next-fetch
juliusmarminge Jun 27, 2023
fdac1f8
don't use experimental nextauth
juliusmarminge Jun 27, 2023
2f116c9
comment why edge is disabled
juliusmarminge Jun 27, 2023
87e2ca7
Merge branch 'main' into next-fetch
kodiakhq[bot] Jun 28, 2023
7077e1c
Merge branch 'main' into next-fetch
kodiakhq[bot] Jun 28, 2023
e2ad25d
Merge branch 'main' into next-fetch
juliusmarminge Jun 28, 2023
3a36579
Merge branch 'main' into next-fetch
kodiakhq[bot] Jun 28, 2023
bb87feb
lint fix
juliusmarminge Jun 28, 2023
db3a1f6
lint fix 2
juliusmarminge Jun 28, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion examples/.experimental/next-app-dir/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
"@types/node": "^18.7.20",
"@types/react": "^18.2.6",
"@types/react-dom": "^18.2.4",
"next": "^13.3.4",
"next": "^13.4.2",
"react": "^18.2.0",
"react-dom": "^18.2.0",
"trpc-api": "link:./src/trpc",
Expand Down
11 changes: 11 additions & 0 deletions examples/.experimental/next-app-dir/src/app/ServerInvoker.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
import { api } from '~/trpc/server-invoker';

export async function ServerInvoker() {
const data = await api.greeting.query({
text: 'i never hit an api endpoint',
});

console.log({ serverInvoker: data });

return <div>Server {data}</div>;
}
23 changes: 20 additions & 3 deletions examples/.experimental/next-app-dir/src/app/page.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import { Suspense } from 'react';
import { api } from 'trpc-api';
import { ClientGreeting } from './ClientGreeting';
import { ServerGreeting } from './ServerGreeting';
import { ServerInvoker } from './ServerInvoker';

export default async function Home() {
const promise = new Promise(async (resolve) => {
Expand All @@ -22,7 +23,7 @@ export default async function Home() {
>
<div
style={{
width: '12rem',
width: '24rem',
padding: '1rem',
background: '#e5e5e5',
borderRadius: '0.5rem',
Expand All @@ -35,17 +36,33 @@ export default async function Home() {
</div>

<div>
<Suspense fallback={<>Loading Server...</>}>
<Suspense fallback={<>Loading Server (fetched)...</>}>
{/* @ts-expect-error RSC + TS not friends yet */}
<ServerGreeting />
</Suspense>
</div>
<div>
<Suspense fallback={<>Loading stream...</>}>
<Suspense fallback={<>Loading stream (fetched)...</>}>
{/** @ts-expect-error - Async Server Component */}
<StreamedSC promise={promise} />
</Suspense>
</div>

<div>
<Suspense fallback={<>Loading Server (invoked)...</>}>
{/* @ts-expect-error RSC + TS not friends yet */}
<ServerInvoker />
</Suspense>
</div>

<form
action={async () => {
'use server';
api.greeting.revalidate();
}}
>
<button type="submit">Revalidate</button>
</form>
</div>
</main>
);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ export const appRouter = router({
)
.query(async (opts) => {
console.log('request from', opts.ctx.headers?.['x-trpc-source']);
return `hello ${opts.input.text}`;
return `hello ${opts.input.text} - ${new Date().toLocaleTimeString()}`;
}),
});

Expand Down
6 changes: 3 additions & 3 deletions examples/.experimental/next-app-dir/src/trpc/package.json
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
{
"name": "trpc-api",
"typings": "server.ts",
"typings": "server-fetcher.ts",
"exports": {
".": {
"types": "./server.ts",
"react-server": "./server.ts",
"types": "./server-fetcher.ts",
juliusmarminge marked this conversation as resolved.
Show resolved Hide resolved
"react-server": "./server-fetcher.ts",
"default": "./client.ts"
}
}
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
'use server';

import { httpBatchLink, loggerLink } from '@trpc/client';
import { loggerLink } from '@trpc/client';
import { nextFetchLink } from '@trpc/next/app-dir/links/nextFetch';
import { experimental_createTRPCNextAppDirServer } from '@trpc/next/app-dir/server';
import { headers } from 'next/headers';
import { AppRouter } from '~/server/routers/_app';
Expand All @@ -15,7 +16,8 @@ export const api = experimental_createTRPCNextAppDirServer<AppRouter>({
process.env.NODE_ENV === 'development' ||
(op.direction === 'down' && op.result instanceof Error),
}),
httpBatchLink({
nextFetchLink({
batch: false,
url: getUrl(),
headers() {
// Forward headers from the browser to the API
Expand Down
28 changes: 28 additions & 0 deletions examples/.experimental/next-app-dir/src/trpc/server-invoker.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
'use server';

import { loggerLink } from '@trpc/client';
import { nextCacheLink } from '@trpc/next/app-dir/links/nextCache';
import { experimental_createTRPCNextAppDirServer } from '@trpc/next/app-dir/server';
import { headers } from 'next/headers';
import { appRouter } from '~/server/routers/_app';

export const api = experimental_createTRPCNextAppDirServer<typeof appRouter>({
config() {
return {
links: [
loggerLink({
enabled: (op) =>
process.env.NODE_ENV === 'development' ||
(op.direction === 'down' && op.result instanceof Error),
}),
nextCacheLink({
router: appRouter,
createContext: async () => {
const h = Object.fromEntries(headers().entries());
return { headers: h };
},
}),
],
};
},
});
12 changes: 11 additions & 1 deletion packages/next/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,16 @@
"import": "./dist/app-dir/client.mjs",
"require": "./dist/app-dir/client.js",
"default": "./dist/app-dir/client.js"
},
"./app-dir/links/nextCache": {
"import": "./dist/app-dir/links/nextCache.mjs",
"require": "./dist/app-dir/links/nextCache.js",
"default": "./dist/app-dir/links/nextCache.js"
},
"./app-dir/links/nextFetch": {
"import": "./dist/app-dir/links/nextFetch.mjs",
"require": "./dist/app-dir/links/nextFetch.js",
"default": "./dist/app-dir/links/nextFetch.js"
}
},
"files": [
Expand Down Expand Up @@ -76,7 +86,7 @@
"@types/react-dom": "^18.2.4",
"eslint": "^8.40.0",
"express": "^4.17.1",
"next": "^13.3.4",
"next": "^13.4.2",
"react": "^18.2.0",
"react-dom": "^18.2.0",
"rollup": "^2.79.1",
Expand Down
2 changes: 2 additions & 0 deletions packages/next/rollup.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ export const input = [
'src/index.ts',
'src/app-dir/server.ts',
'src/app-dir/client.ts',
'src/app-dir/links/nextCache.ts',
'src/app-dir/links/nextFetch.ts',
];

export default function rollup(): RollupOptions[] {
Expand Down
56 changes: 56 additions & 0 deletions packages/next/src/app-dir/links/nextCache.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
// import "server-only";

import { TRPCClientError, TRPCLink } from '@trpc/client';
import { AnyRouter, callProcedure, inferRouterContext } from '@trpc/server';
import { observable } from '@trpc/server/observable';
import { unstable_cache } from 'next/cache';
import { generateCacheTag } from '../shared';

type NextCacheLinkOptions<TRouter extends AnyRouter> = {
router: TRouter;
createContext: () => Promise<inferRouterContext<TRouter>>;
};

export function nextCacheLink<TRouter extends AnyRouter>(
opts: NextCacheLinkOptions<TRouter>,
): TRPCLink<TRouter> {
return () =>
({ op }) =>
observable((observer) => {
const { path, input, type, context } = op;

const cacheTag = generateCacheTag(path, input);
const revalidate =
typeof context?.revalidate === 'number' ? context.revalidate : false;

const promise = opts
.createContext()
.then(async (ctx) => {
const callProc = async () =>
callProcedure({
procedures: opts.router._def.procedures,
path,
rawInput: input,
ctx: ctx,
type,
});

if (type === 'query') {
return unstable_cache(callProc, path.split('.'), {
revalidate,
tags: [cacheTag],
})();
}

return callProc();
})
.catch((cause) => observer.error(TRPCClientError.from(cause)));

promise
.then((data) => {
observer.next({ result: { data } });
observer.complete();
})
.catch((cause) => observer.error(TRPCClientError.from(cause)));
});
}
38 changes: 38 additions & 0 deletions packages/next/src/app-dir/links/nextFetch.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
import {
HTTPLinkOptions,
HttpBatchLinkOptions,
TRPCLink,
httpBatchLink,
httpLink,
} from '@trpc/client';
import { AnyRouter } from '@trpc/server';
import { generateCacheTag } from '../shared';

type NextFetchLinkOptions<TBatch extends boolean> = {
batch?: TBatch;
} & (TBatch extends true ? HttpBatchLinkOptions : HTTPLinkOptions);

export function nextFetchLink<
TRouter extends AnyRouter,
TBatch extends boolean,
>(opts: NextFetchLinkOptions<TBatch>): TRPCLink<TRouter> {
return (runtime) => {
return (ctx) => {
const { path, input } = ctx.op;
const cacheTag = generateCacheTag(path, input);

const linkFactory = opts.batch ? httpBatchLink : httpLink;
const link = linkFactory({
url: opts.url,
fetch: (url, fetchOpts) => {
return fetch(url, {
...fetchOpts,
next: { tags: [cacheTag] },
});
},
})(runtime);

return link(ctx);
};
};
}
57 changes: 48 additions & 9 deletions packages/next/src/app-dir/server.ts
Original file line number Diff line number Diff line change
@@ -1,14 +1,48 @@
/// <reference types="next" />

import {
CreateTRPCProxyClient,
Resolver,
clientCallTypeToProcedureType,
createTRPCUntypedClient,
} from '@trpc/client';
import { AnyRouter } from '@trpc/server';
import {
AnyMutationProcedure,
AnyProcedure,
AnyQueryProcedure,
AnyRouter,
AnySubscriptionProcedure,
ProcedureRouterRecord,
} from '@trpc/server';
import { createRecursiveProxy } from '@trpc/server/shared';
import { revalidateTag } from 'next/cache';
import { cache } from 'react';
import { CreateTRPCNextAppRouterOptions } from './shared';
import { CreateTRPCNextAppRouterOptions, generateCacheTag } from './shared';

export type DecorateProcedureServer<TProcedure extends AnyProcedure> =
TProcedure extends AnyQueryProcedure
? {
query: Resolver<TProcedure>;
revalidate: () => void;
}
: TProcedure extends AnyMutationProcedure
? {
mutate: Resolver<TProcedure>;
}
: TProcedure extends AnySubscriptionProcedure
? {
subscribe: Resolver<TProcedure>;
}
: never;

export type ServerDecoratedProcedureRecord<
TProcedures extends ProcedureRouterRecord,
> = {
[TKey in keyof TProcedures]: TProcedures[TKey] extends AnyRouter
? ServerDecoratedProcedureRecord<TProcedures[TKey]['_def']['record']>
: TProcedures[TKey] extends AnyProcedure
? DecorateProcedureServer<TProcedures[TKey]>
: never;
};

// ts-prune-ignore-next
export function experimental_createTRPCNextAppDirServer<
Expand All @@ -24,11 +58,16 @@ export function experimental_createTRPCNextAppDirServer<
const client = getClient();

const pathCopy = [...callOpts.path];
const procedureType = clientCallTypeToProcedureType(
pathCopy.pop() as string,
);
const fullPath = pathCopy.join('.');
const action = pathCopy.pop() as string;

const procedurePath = pathCopy.join('.');
const procedureType = clientCallTypeToProcedureType(action);
const cacheTag = generateCacheTag(procedurePath, callOpts.args[0]);

if (action === 'revalidate') {
return revalidateTag(cacheTag);
}

return (client[procedureType] as any)(fullPath, ...callOpts.args);
}) as CreateTRPCProxyClient<TRouter>;
return (client[procedureType] as any)(procedurePath, ...callOpts.args);
}) as ServerDecoratedProcedureRecord<TRouter['_def']['record']>;
}
9 changes: 9 additions & 0 deletions packages/next/src/app-dir/shared.ts
Original file line number Diff line number Diff line change
Expand Up @@ -59,3 +59,12 @@ export type CreateTRPCNextAppRouter<TRouter extends AnyRouter> =
export interface CreateTRPCNextAppRouterOptions<TRouter extends AnyRouter> {
config: () => CreateTRPCClientOptions<TRouter>;
}

/**
* @internal
*/
export function generateCacheTag(procedurePath: string, input: any) {
return input
? `${procedurePath}?input=${JSON.stringify(input)}`
: procedurePath;
}
Loading
Loading