Control your data cache on the front-end.
Data fetching and caching using a robust middleware system. Quickly build data loading within your redux application and reduce boilerplate.
This library is undergoing active development. Consider this in a beta state.
- Examples
- Control your data cache
- Typescript
- Manipulating the request
- Auto caching
- Dispatching many actions
- Dependent queries
- Dynamic endpoints
- Error handling
- Loading state
- React
- React Native
- Cache timer
- Take leading
- Polling
- Optimistic UI
- Undo
- Performance monitor
- redux-toolkit
- Write middleware to handle fetching, synchronizing, and caching API requests on the front-end
- A middleware system that node.js developers are familiar with (e.g. express.js, koa.js)
- Automatically track loading states for data fetching
- Automatically cache data in redux
- Simple recipes to handle complex use-cases like cancellation, polling, optimistic updates, loading states, undo
- React hooks to make it even easier to use in react applications
- Full control over the data fetching and caching layers in your application
- Fine tune selectors for your specific needs
// api.ts
import { createApi, requestMonitor, fetcher } from 'saga-query';
const api = createApi();
api.use(requestMonitor());
api.use(api.routes());
api.use(fetcher({ baseUrl: 'https://api.github.com' }));
export const fetchRepo = api.get(`/repos/redux-saga/saga-query`, api.cache());
// app.tsx
import React, { useEffect } from 'react';
import { useCache } from 'saga-query/react';
import { fetchRepo } from './api';
interface Repo {
name: string;
stargazers_count: number;
}
const App = () => {
const { data, isInitialLoading, isError, message } = useCache<Repo>(fetchRepo());
if (isInitialLoading) return <div>Loading ...</div>;
if (isError) return <div>{message}</div>;
return (
<div>
<div>{data.name}</div>
<div>{data.stargazers_count}</div>
</div>
);
};
Libraries like react-query, rtk-query, and apollo-client are making it easier than ever to fetch and cache data from an API server. All of them have their unique attributes and I encourage everyone to check them out.
There's no better async flow control system than redux-saga
. Treating
side-effects as data makes testing dead simple and provides a powerful effect
handling system to accommodate any use-case. Features like polling, data loading
states, cancellation, racing, parallelization, optimistic updates, and undo are
at your disposal when using redux-saga
. Other libraries and paradigms can also
accomplish the same tasks, but I think nothing rivals the readability and
maintainability of redux/redux-saga.
All three libraries above are reinventing async flow control and hiding them from the end-developer. For the happy path, this works beautifully. Why learn how to cache API data when a library can do it for you? However:
- What happens when
useMemo
isn't good enough? - What happens when the data syncing library lacks the caching granularity you need?
- What happens when the data syncing library doesn't cache things in an optimized way for your needs?
- What happens when you want to reuse your business logic for another platform
(e.g. a cli) and can't use
react
?
This library is built to support both small and large scale, complex flow control applications that need full control over the data cache layer while setting good standards for using redux and a flexible middleware to handle all business logic.
- The end-developer should have full control over fetching/caching/querying their server data
- Fetching and caching data should be separate from the view layer
- We should treat side-effects as data
- Sagas are the central processing unit for IO/business logic
- A minimal API that encourages end-developers to write code instead of configuring objects
- A DSL wrapped around data fetching and caching logic
- A one-line solution to fetch and cache server data automatically
Underneath saga-query
is a simple middleware system that will feel familiar to
those using express
or koa
. Our thought was: why not use it for the FE? So
we took that middleware system and integrated it with redux-saga
to create a
powerful system for managing business logic in your FE apps.
The middleware that is loaded into the system via .use(...)
which gets added
to an array. Each action that is dispatched using this system moves through this
array pipeline in order. When yield next()
is called inside the middleware, it
calls the next middleware in the stack until it finishes. Everything after
yield next()
gets called after all the middleware ahead of the current
function finishes its execution.
An example of the order of execution can be found in our tests.
The actions created from saga-query
are JSON serializable. We are not
passing middleware functions through our actions. This is a design decision to
support things like
inter-process communication.
At its core, we have createPipe
which just builds the middleware system and
allows you to dispatch actions that traverse through the middleware pipeline.
import { createPipe, put, delay } from 'saga-query';
const thunks = createPipe();
thunks.use(thunks.routes());
thunks.use(function* (ctx, next) {
console.log('start');
yield next();
console.log('all done!');
});
const increment = thunks.create('increment', function* (ctx, next) {
yield next();
console.log('waiting 1s');
yield delay(1000);
console.log('incrementing!');
yield* put({ type: 'INCREMENT' });
});
store.dispatch(increment());
// start
// waiting 1s
// incrementing!
// all done!
Even if you don't want to use saga-query
for fetching/caching functionality,
it is still exceptionally useful with just createPipe
.
createApi
will build a set of actions and sagas for each create
or http
method used (e.g. get
, post
, put
). Let's call them endpoints. Each
endpoint gets their own action and linked saga. When you call api.saga()
it
loops through all the endpoints and creates a root saga that is fault tolerant
(one saga won't crash all the other sagas). The default for each endpoint is to
use takeEvery
from redux-saga
but as you'll see in other recipes, this can
be easily overridden.
import {
createApi,
requestMonitor,
// ApiCtx is an interface that's built around using window.fetch
// You don't have to use it if you don't want to.
ApiCtx,
put,
call,
} from 'saga-query';
import { createSlice, configureStore } from '@reduxjs/toolkit';
interface MapEntity<E> {
[key: string]: E | undefined;
}
// create a reducer that acts like a SQL database table
// the keys are the id and the value is the record
const users = createSlice({
name: 'users',
initialState: {},
reducers: {
add: (state, action) => {
action.payload.forEach((user) => {
state[user.id] = user.id;
});
},
},
});
// something awesome happens in here
// The default generic value here is `ApiCtx` which includes a `payload`,
// `request`, and `response`.
// The type passed to `createApi` must extend `ApiCtx` to be accepted.
const api = createApi<ApiCtx>();
// This middleware monitors the lifecycle of the request. It needs to be
// loaded before `.routes()` because it needs to be around after everything
// else finishes.
// [queryCtx] sets up the ctx object with `ctx.request` and `ctx.response`
// required for `createApi` to function properly.
// [dispatchActions] This middleware leverages `redux-batched-actions` to
// dispatch all the actions stored within `ctx.actions` which get added by
// other middleware during the lifecycle of the request.
// [loadingMonitor] This middleware will monitor the lifecycle of a request and
// attach the appropriate loading states to the loader associated with the
// endpoint.
// [urlParser] is a middleware that will take the name of `api.create(name)` and
// replace it with the values passed into the action.
// [simpleCache] is a middleware that will automatically store the response of
// endpoints if the endpoint has `ctx.cache = true`
api.use(requestMonitor());
// This is where all the endpoints (e.g. `.get()`, `.put()`, etc.) you created
// get added to the middleware stack. It is recommended to put this as close to
// the beginning of the stack so everything after `yield next()`
// happens at the end of the effect.
api.use(api.routes());
// Under the hood this is a middleware that handles fetching
// an endpoint using window.fetch. It also automatically
// processes JSON and stores it in `ctx.json`.
api.use(fetcher({ baseUrl: 'https://...' }));
// This is how you create a function that will fetch an API endpoint. The
// first parameter is the name of the action type. When using `urlParser` it
// will also be the URL inside `ctx.request.url` of which you can do what you
// want with it.
const fetchUsers = api.get(
`/users`,
// Since this middleware is first it has the unique benefit of being in full
// control of when the other middleware get activated.
// The type inside of `ApiCtx` is the response object
function* (ctx: ApiCtx<{ users: User[] }>, next) {
// anything before this call can mutate the `ctx` object before it gets
// sent to the other middleware
yield next();
// anything after the above line happens *after* the middleware gets called and
// and a fetch has been made.
// using ApiCtx `ctx.json` is a discriminated union based on the
// boolean `ctx.response.ok`.
if (!ctx.json.ok) return;
// data = { users: User[] };
const { data } = ctx.json;
const curUsers = data.users.reduce<MapEntity<User>>((acc, u) => {
acc[u.id] = u;
return acc;
}, {});
// save the data to our redux slice called `users`
yield* put(users.actions.add(curUsers));
},
);
// This is a helper function that does a bunch of stuff to prepare redux for
// saga-query. In particular, it will:
// - Setup redux-saga
// - Setup redux-batched-actions
// - Setup a couple of reducers that saga-query will use: loaders and data
const prepared = prepareStore({
reducers: { users: users.reducer },
sagas: { api: api.saga() },
});
const store = configureStore({
reducer: prepared.reducer,
middleware: prepared.middleware,
});
// This runs the sagas
prepared.run();
store.dispatch(fetchUsers());
This library was built with typescript in mind. One advantage this library has
over redux-saga
is that we leverage typed-redux-saga.
By leveraging yield delegates yield*
we can now get fully typed results from
our yield statements! So if you're coming from redux-saga
and don't know
what that means, just add yield*
to your call effects (e.g. call, put,
select, etc.) and enjoy the better types.
import { createApi, ApiCtx } from 'saga-query';
interface Ctx<P = any, S = any, E = any> extends ApiCtx<P, S, E> {
something: boolean;
};
interface Props { id: string }
interface Success { result: string }
interface Err { message: string }
// set the context object here to have it apply to
// all endpoints by default
const api = createApi<Ctx>();
const fetchUser = api.get<Props, Success, Err>(
'/users/:id',
function*(ctx, next) {
// access `Ctx` properties
ctx.something;
// will be set to `Props`
ctx.payload;
yield next();
if (ctx.json.ok) {
// will be set to `Success`
ctx.json.data;
} else {
// will be set to `Err`
ctx.json.data;
}
},
);
Under the hood, ctx.request
goes directly into fetch
. ctx.response
is the
Response
object from fetch
. The fetcher
middleware assumes you are dealing
with JSON so it will automatically set the Content-Type
and also try to
convert the Response
to JSON.
We built a helper function that is baked into the ctx
object called
ctx.req()
.
The entire purpose of this function is to help make it easier to update the
request object that will be sent directly into fetch
. It does a smart merge
with the current ctx.request
object and whatever you pass into it.
We recommend not updating properies on the ctx.request
object directly and
instead use ctx.req
to assign the value of ctx.request
.
const createUser = api.post<{ id: string; email: string }>(
`/users`,
function* onCreateUser(ctx: ApiCtx<User>, next) {
// here we manipulate the request before it gets sent to our middleware
ctx.request = ctx.req({
body: JSON.stringify({ email: ctx.payload.email }),
});
yield next();
if (!ctx.json.ok) return;
const curUser = ctx.json.data;
const curUsers = { [curUser.id]: curUser };
yield* put(users.actions.add(curUsers));
},
);
store.dispatch(createUser({ id: '1' }));
Have some request
data that you want to set when creating the endpoint?
const fetchUsers = api.get('/users', api.request({ credentials: 'include' }));
api.request()
accepts the request for the ctx
that the end-developer
provides.
If you want to have a cache that doesn't enforce strict types and is more of a
dumb cache that fetches and stores data for you, then simpleCache
will provide
that functionality for you.
The following code will mimic what a library like react-query
is doing
behind-the-scenes. I want to make it clear that react-query
is doing a lot
more than this so I don't want to understate what it does. However, you can see
that not only can we get a core chunk of the functionality react-query
provides with a little over 100 lines of code but we also have full control over
fetching, querying, and caching data with the ability to customize it using
middleware.
// api.ts
import {
createApi,
requestMonitor,
fetcher,
timer,
} from 'saga-query';
const api = createApi();
api.use(requestMonitor());
api.use(api.routes());
api.use(fetcher());
// this will only activate the endpoint at most once every 5 minutes.
const cacheTimer = timer(5 * 60 * 1000);
export const fetchUsers = api.get(
'/users',
{ saga: cacheTimer },
// set `ctx.cache=true` to have simpleCache middleware cache response data
// automatically
api.cache(),
);
// app.tsx
import React from 'react';
import { useCache } from 'saga-query/react';
import { fetchUsers } from './api';
interface User {
id: string;
name: string;
}
const useUsers = () => useCache<{ users: User[] }>(fetchUsers());
export const App = () => {
const { data = [], isInitialLoading, isError, message } = useUsers();
if (isInitialLoading) return <div>Loading ...</div>;
if (isError) return <div>Error: {message}</div>;
return (
<div>
{data.map((user) => (
<div key={user.id}>{user.name}</div>
))}
</div>
);
};
Sometimes we need to dispatch a bunch of actions for an endpoint. From loading
states to making multiple requests in a single saga, there can be a lot of
actions being dispatched. When using prepareStore
we automatically setup
redux-batched-actions
so you don't have to. Anything that gets added to
ctx.actions
will be automatically dispatched by the dispatchActions
middleware.
Sometimes it's necessary to compose multiple endpoints together. For example we
might want to fetch a mailbox and its associated messages. Every endpoint also
returns a property on the action creator .run
which returns the saga that runs
when the action is dispatched.
This allows us to yield
to that saga inside another endpoint.
const fetchMailboxes = api.get('/mailboxes');
const fetchMessages = api.get('/mailboxes/:id/messages', function* (ctx, next) {
// The return value of a `.run` is the entire `ctx` object.
const mailCtx = yield* call(fetchMailboxes.run, fetchMailboxes());
if (!mailCtx.json.ok) {
yield next();
return;
}
// grab first mailbox from list
const url = `/mailboxes/${mailCtx.json[0].id}/messages`;
ctx.request = ctx.req({ url });
yield next();
});
*NOTE: This will bypass the saga
that is listenting for the action to be
dispatched. This means that if there's any special saga logic like our
timer()
saga it will not be run.
Sometimes a URL needs to be generated from other data. When creating an
endpoint, it must be created before api.saga()
is called. Because of
this, there's a limitation to what we can permit inside the name
of the
endpoint. The name
is the first parameter passed to the HTTP methods
api.get(name)
or api.post(name)
. If you need to generate the URL based on
dynamic content, like a state derived value, then the recommended solution is to
do the following:
api.post<{ message: string }>(
'create-message',
function* onCreateMsg(ctx, next) {
// made up selector that grabs a mailbox
const mailbox = yield* select(selectMailbox);
const message = ctx.payload.message;
ctx.request = ctx.req({
url: `/mailboxes/${mailbox.id}/messages`,
body: JSON.stringify({ message }),
});
yield next();
},
);
As you can see, we can put whatever want for the name
parameter passed into
api.get(name)
. The key thing to realize here is that the name must be
unique across all endpoints since the name is what we use for the action type.
Error handling can be accomplished in a bunch of places in the middleware pipeline.
Catch all middleware before itself:
const api = createApi();
api.use(function* upstream(ctx, next) {
try {
yield next();
} catch (err) {
console.log('error!');
}
});
api.use(api.routes());
api.use(function* fail() {
throw new Error('some error');
});
const action = api.create(`/error`);
const store = setupStore(api.saga());
store.dispatch(action());
Catch middleware inside the action handler:
const api = createApi();
api.use(api.routes());
api.use(function* fail() {
throw new Error('some error');
});
const action = api.create(`/error`, function* (ctx, next) {
try {
yield next();
} catch (err) {
console.log('error!');
}
});
const store = setupStore(api.saga());
store.dispatch(action());
Global error handler:
const api = createApi({
onError: (err: Error) => {
console.log('error!');
},
});
api.use(api.routes());
api.use(function* upstream(ctx, next) {
throw new Error('failure');
});
const action = api.create(`/error`);
const store = setupStore(api.saga());
store.dispatch(action());
When using prepareStore
in conjunction with requestMonitor
so the loading
state will automatically be added to all of your endpoints. We also export
QueryState
which is the interface that contains all the state types that
saga-query
provides.
// app.tsx
import React, { useEffect } from 'react';
import { useSelector, useDispatch } from 'react-redux';
import type { QueryState } from 'saga-query';
import { useLoader } from 'saga-query/react';
interface MapEntity<E> {
[key: string]: E | undefined;
}
import { fetchUsers, selectUsersAsList } from './api';
interface AppState extends QueryState {
users: MapEntity<User>;
}
const App = () => {
const dispatch = useDispatch();
const users = useSelector(selectUsersAsList);
const loader = useLoader(fetchUsers);
useEffect(() => {
dispatch(fetchUsers());
}, []);
if (loader.isInitialLoading) return <div>Loading ...</div>;
if (loader.isError) return <div>Error: {loader.message}</div>;
return (
<div>
{users.map((user) => (
<div key={user.id}>{user.email}</div>
))}
</div>
);
};
We built a couple of simple hooks useQuery
, useCache
, useLoader
, and
useLoaderSuccess
to make interacting with saga-query
easier. Having said
that, it would be trivial to build your own custom hooks to do exactly what you
want.
This section is a WIP, for now you can read the source
In order to use imports from saga-query/react
in React Native, a custom resolve request must be defined. This is due to Metro not currently supporting package exports. See here for details. Experimental support for package exports looks to be coming in 0.72. Below is an example of the workaround:
// metro.config.js
module.exports = {
// ...
resolver: {
// ...
resolveRequest: (context, moduleName, platform) => {
if (moduleName === "saga-query/react") {
return {
filePath: path.resolve(
__dirname,
"node_modules/saga-query/dist/react.js",
),
type: "sourceFile",
};
}
return context.resolveRequest(context, moduleName, platform);
}
}
}
Only call the endpoint at most on an interval. We can call the endpoint as many times as we want but it will only get activated once every X milliseconds. This effectively updates the cache on an interval.
import { timer } from 'saga-query';
const SECONDS = 1000;
const MINUTES = 60 * SECONDS;
const fetchUsers = api.get('/users', { saga: timer(5 * MINUTES) });
If two requests are made:
- (A) request; then
- (B) request
While (A) request is still in flight, (B) request would be canceled.
import { takeLeading } from 'saga-query';
// this is for demonstration purposes, you can import it using
// import { leading } from 'saga-query';
function* leading(action: string, saga: any, ...args: any[]) {
yield takeLeading(`${action}`, saga, ...args);
}
const fetchUsers = api.get(
`/users`,
{ saga: leading },
function* processUsers(ctx, next) {
yield next();
// ...
},
);
We built this saga helper for you which will either accept the timer as a number or if the payload contains a timer prop:
import { poll } from 'saga-query';
const pollUsers = api.get(`/users`, { saga: poll(5 * 1000) });
action.payload.timer
takes precedence.
import React, { useState, useEffect } from 'react';
import { useDispatch } from 'react-redux';
import { pollUsers } from './api';
const App = () => {
const dispatch = useDispatch();
const [polling, setPolling] = useState('init');
const onClick = () => {
if (polling === 'on') setPolling('off');
else setPolling('on');
};
useEffect(() => {
if (polling === 'init') return;
dispatch(pollRepo());
}, [polling]);
return (
<div>
<div>Polling: {polling}</div>
<button onClick={onClick}>Toggle Polling</button>
</div>
);
};
Here is the manual, one-off way to handle optimistic ui:
import { put, select } from 'saga-query';
interface UpdateUser { id: string; email: string };
const updateUser = api.patch<UpdateUser>(
`/users/:id`,
function* onUpdateUser(ctx: ApiCtx<User, UpdateUser>, next) {
// the payload gets typed to UpdateUser
const { id, email } = ctx.payload;
ctx.request = ctx.req({
body: JSON.stringify(email),
});
// save the current user record in a variable
const prevUser = yield* select(selectUserById, { id }));
// optimistically update user
yield* put(users.actions.patch({ [user.id]: { email } }));
// activate PATCH request
yield next();
// oops something went wrong, revert!
if (!ctx.json.ok) {
yield* put(users.actions.add({ [prevUser.id]: prevUser });
return;
}
// even though we know what was updated, it's still a good habit to
// update our local cache with what the server sent us
const nextUser = ctx.json.data;
yield* put(users.actions.add({ [nextUser.id]: nextUser }));
},
)
Not too bad, but we built an optimistic middleware for you:
import { OptimisticCtx, optimistic } from 'saga-query';
interface MapEntity<E> {
[key: string]: E | undefined;
}
interface PatchEntity<T> {
[key: string]: Partial<T[keyof T]>;
}
const api = createApi();
api.use(api.routes());
api.use(optimistic);
api.patch(
function* (ctx: OptimisticCtx<PatchEntity<User>, MapEntity<User>>, next) {
const { id, email } = ctx.payload;
const prevUser = yield* select(selectUserById, { id }));
ctx.optimistic = {
apply: users.actions.patch({ [id]: { email } }),
revert: users.actions.add({ [id]: prevUser }),
};
ctx.request = ctx.req({
method: 'PATCH',
body: JSON.stringify({ email }),
});
yield next();
}
);
We build a simple undo middleware that waits for one of two actions to be dispatched:
- doIt() which will call the endpoint
- undo() which will cancel the endpoint
The middleware accepts three properties:
doItType
(default:${doIt}
) => action typeundoType
(default:${undo}
) => action typetimeout
(default: 30 * 1000) => time in milliseconds before the endpoint get canceled automatically
import {
createApi,
requestMonitor,
undoer,
undo,
doIt,
UndoCtx,
delay,
put,
race,
} from 'saga-query';
interface Message {
id: string;
archived: boolean;
}
const messages = createTable<Message>({ name: 'messages' });
const api = createApi<UndoCtx>();
api.use(requestMonitor());
api.use(api.routes());
api.use(undoer());
const archiveMessage = api.patch<{ id: string }>(
`message/:id`,
function* onArchive(ctx, next) {
ctx.undoable = true;
// prepare the request
ctx.request = ctx.req({
body: JSON.stringify({ archived: true }),
});
// make the API request
yield next();
},
);
const reducers = createReducerMap(messages);
const store = setupStore(api.saga(), reducers);
store.dispatch(archiveMessage({ id: '1' }));
// wait 2 seconds to cancel endpoint
store.dispatch(undo());
// -or- to activate the endpoint
store.dispatch(doIt());
This is not the only way to implement an undo mechanism, it's just the one we provide out-of-the-box to work with a UI that fully controls the undo mechanism.
For example, if you want the endpoint to be called automatically after some timer, you could build a middleware to do that for you:
import { race, delay } from 'saga-query';
const undo = () => ({ type: 'UNDO' });
function* undoer<Ctx extends UndoCtx = UndoCtx>() {
if (!ctx.undoable) {
yield next();
return;
}
const winner = yield* race({
timer: delay(3 * 1000),
undo: take(`${undo}`),
});
if (winner.undo) return;
yield next();
}
import {
performanceMonitor,
createPipe,
wrap,
PerfCtx,
delay,
} from 'saga-query';
const thunks = createPipe<PerfCtx>();
thunks.use(function* (ctx, next) {
yield next();
console.log(`calling ${ctx.name} took ${ctx.performance} ms`);
});
thunks.use(performanceMonitor);
thunks.use(thunks.routes());
function* slowSaga() {
yield delay(10 * 1000);
}
const slow = thunks.create('something-slow', wrap(slowSaga));
store.dispatch(slow());
// calling something-slow took 10000 ms
redux-toolkit
is a very popular redux library officially supported by the
redux
team. When using it with saga-query
the main thing it is responsible
for is setting up the redux slice where we want to cache the API endpoint
response data.
import { prepareStore, createApi, requestMonitor, fetcher } from 'saga-query';
import { createSlice, configureStore } from '@reduxjs/toolkit';
const users = createSlice({
name: 'users',
initialState: {},
reducers: {
add: (state, action) => {
action.payload.forEach((user) => {
state[user.id] = user.id;
});
},
},
});
const api = createApi();
api.use(requestMonitor());
api.use(api.routes());
api.use(fetcher());
const fetchUsers = api.get<{ users: User[] }>('/users', function* (ctx, next) {
yield next();
if (!ctx.json.ok) return;
const { data } = ctx.json.data;
yield* put(users.actions.add(data.users));
});
const prepared = prepareStore({
reducers: { users: users.reducer },
sagas: { api: api.saga() },
});
const store = configureStore({
reducer: prepared.reducer,
middleware: prepared.middleware,
});
prepared.run();
store.dispatch(fetchUsers());