Skip to content

Commit dae832c

Browse files
r1tsuuDanRibbens
r1tsuu
and
DanRibbens
authored
feat: select fields (#8550)
Adds `select` which is used to specify the field projection for local and rest API calls. This is available as an optimization to reduce the payload's of requests and make the database queries more efficient. Includes: - [x] generate types for the `select` property - [x] infer the return type by `select` with 2 modes - include (`field: true`) and exclude (`field: false`) - [x] lots of integration tests, including deep fields / localization etc - [x] implement the property in db adapters - [x] implement the property in the local api for most operations - [x] implement the property in the rest api - [x] docs --------- Co-authored-by: Dan Ribbens <dan.ribbens@gmail.com>
1 parent 6cdf141 commit dae832c

File tree

116 files changed

+5479
-359
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

116 files changed

+5479
-359
lines changed

‎docs/local-api/overview.mdx

Lines changed: 16 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -77,21 +77,22 @@ Both options function in exactly the same way outside of one having HMR support
7777

7878
You can specify more options within the Local API vs. REST or GraphQL due to the server-only context that they are executed in.
7979

80-
| Local Option | Description |
81-
|-----------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
82-
| `collection` | Required for Collection operations. Specifies the Collection slug to operate against. |
83-
| `data` | The data to use within the operation. Required for `create`, `update`. |
84-
| `depth` | [Control auto-population](../queries/depth) of nested relationship and upload fields. |
85-
| `locale` | Specify [locale](/docs/configuration/localization) for any returned documents. |
86-
| `fallbackLocale` | Specify a [fallback locale](/docs/configuration/localization) to use for any returned documents. |
87-
| `overrideAccess` | Skip access control. By default, this property is set to true within all Local API operations. |
88-
| `overrideLock` | By default, document locks are ignored (`true`). Set to `false` to enforce locks and prevent operations when a document is locked by another user. [More details](../admin/locked-documents). |
89-
| `user` | If you set `overrideAccess` to `false`, you can pass a user to use against the access control checks. |
90-
| `showHiddenFields` | Opt-in to receiving hidden fields. By default, they are hidden from returned documents in accordance to your config. |
91-
| `pagination` | Set to false to return all documents and avoid querying for document counts. |
92-
| `context` | [Context](/docs/hooks/context), which will then be passed to `context` and `req.context`, which can be read by hooks. Useful if you want to pass additional information to the hooks which shouldn't be necessarily part of the document, for example a `triggerBeforeChange` option which can be read by the BeforeChange hook to determine if it should run or not. |
93-
| `disableErrors` | When set to `true`, errors will not be thrown. Instead, the `findByID` operation will return `null`, and the `find` operation will return an empty documents array. |
94-
| `disableTransaction` | When set to `true`, a [database transactions](../database/transactions) will not be initialized. |
80+
| Local Option | Description |
81+
| -------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
82+
| `collection` | Required for Collection operations. Specifies the Collection slug to operate against. |
83+
| `data` | The data to use within the operation. Required for `create`, `update`. |
84+
| `depth` | [Control auto-population](../queries/depth) of nested relationship and upload fields. |
85+
| `locale` | Specify [locale](/docs/configuration/localization) for any returned documents. |
86+
| `select` | Specify [select](../queries/select) to control which fields to include to the result. |
87+
| `fallbackLocale` | Specify a [fallback locale](/docs/configuration/localization) to use for any returned documents. |
88+
| `overrideAccess` | Skip access control. By default, this property is set to true within all Local API operations. |
89+
| `overrideLock` | By default, document locks are ignored (`true`). Set to `false` to enforce locks and prevent operations when a document is locked by another user. [More details](../admin/locked-documents). |
90+
| `user` | If you set `overrideAccess` to `false`, you can pass a user to use against the access control checks. |
91+
| `showHiddenFields` | Opt-in to receiving hidden fields. By default, they are hidden from returned documents in accordance to your config. |
92+
| `pagination` | Set to false to return all documents and avoid querying for document counts. |
93+
| `context` | [Context](/docs/hooks/context), which will then be passed to `context` and `req.context`, which can be read by hooks. Useful if you want to pass additional information to the hooks which shouldn't be necessarily part of the document, for example a `triggerBeforeChange` option which can be read by the BeforeChange hook to determine if it should run or not. |
94+
| `disableErrors` | When set to `true`, errors will not be thrown. Instead, the `findByID` operation will return `null`, and the `find` operation will return an empty documents array. |
95+
| `disableTransaction` | When set to `true`, a [database transactions](../database/transactions) will not be initialized. |
9596

9697
_There are more options available on an operation by operation basis outlined below._
9798

‎docs/queries/select.mdx

Lines changed: 99 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,99 @@
1+
---
2+
title: Select
3+
label: Select
4+
order: 30
5+
desc: Payload select determines which fields are selected to the result.
6+
keywords: query, documents, pagination, documentation, Content Management System, cms, headless, javascript, node, react, nextjs
7+
---
8+
9+
You may not need the full data from your Local API / REST queries, but only some specific fields. The select fields API can help you to optimize those cases.
10+
11+
## Local API
12+
13+
To specify select in the [Local API](../local-api/overview), you can use the `select` option in your query:
14+
15+
```ts
16+
// Include mode
17+
const getPosts = async () => {
18+
const posts = await payload.find({
19+
collection: 'posts',
20+
select: {
21+
text: true,
22+
// select a specific field from group
23+
group: {
24+
number: true
25+
},
26+
// select all fields from array
27+
array: true,
28+
}, // highlight-line
29+
})
30+
31+
return posts
32+
}
33+
34+
// Exclude mode
35+
const getPosts = async () => {
36+
const posts = await payload.find({
37+
collection: 'posts',
38+
// Select everything except for array and group.number
39+
select: {
40+
array: false,
41+
group: {
42+
number: false
43+
}
44+
}, // highlight-line
45+
})
46+
47+
return posts
48+
}
49+
```
50+
51+
52+
<Banner type="warning">
53+
<strong>Important:</strong>
54+
To perform querying with `select` efficiently, it works on the database level. Because of that, your `beforeRead` and `afterRead` hooks may not receive the full `doc`.
55+
</Banner>
56+
57+
58+
## REST API
59+
60+
To specify select in the [REST API](../rest-api/overview), you can use the `select` parameter in your query:
61+
62+
```ts
63+
fetch('https://localhost:3000/api/posts?select[color]=true&select[group][number]=true') // highlight-line
64+
.then((res) => res.json())
65+
.then((data) => console.log(data))
66+
```
67+
68+
To understand the syntax, you need to understand that complex URL search strings are parsed into a JSON object. This one isn't too bad, but more complex queries get unavoidably more difficult to write.
69+
70+
For this reason, we recommend to use the extremely helpful and ubiquitous [`qs`](https://www.npmjs.com/package/qs) package to parse your JSON / object-formatted queries into query strings:
71+
72+
```ts
73+
import { stringify } from 'qs-esm'
74+
75+
const select = {
76+
text: true,
77+
group: {
78+
number: true
79+
}
80+
// This query could be much more complex
81+
// and QS would handle it beautifully
82+
}
83+
84+
const getPosts = async () => {
85+
const stringifiedQuery = stringify(
86+
{
87+
select, // ensure that `qs` adds the `select` property, too!
88+
},
89+
{ addQueryPrefix: true },
90+
)
91+
92+
const response = await fetch(`http://localhost:3000/api/posts${stringifiedQuery}`)
93+
// Continue to handle the response below...
94+
}
95+
96+
<Banner type="info">
97+
<strong>Reminder:</strong>
98+
This is the same for [Globals](../configuration/globals) using the `/api/globals` endpoint.
99+
</Banner>

‎docs/rest-api/overview.mdx

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@ All Payload API routes are mounted and prefixed to your config's `routes.api` UR
1818
- [depth](../queries/depth) - automatically populates relationships and uploads
1919
- [locale](/docs/configuration/localization#retrieving-localized-docs) - retrieves document(s) in a specific locale
2020
- [fallback-locale](/docs/configuration/localization#retrieving-localized-docs) - specifies a fallback locale if no locale value exists
21+
- [select](../queries/select) - speicifes which fields to include to the result
2122

2223
## Collections
2324

‎packages/db-mongodb/src/deleteOne.ts

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,12 +2,13 @@ import type { DeleteOne, Document, PayloadRequest } from 'payload'
22

33
import type { MongooseAdapter } from './index.js'
44

5+
import { buildProjectionFromSelect } from './utilities/buildProjectionFromSelect.js'
56
import { sanitizeInternalFields } from './utilities/sanitizeInternalFields.js'
67
import { withSession } from './withSession.js'
78

89
export const deleteOne: DeleteOne = async function deleteOne(
910
this: MongooseAdapter,
10-
{ collection, req = {} as PayloadRequest, where },
11+
{ collection, req = {} as PayloadRequest, select, where },
1112
) {
1213
const Model = this.collections[collection]
1314
const options = await withSession(this, req)
@@ -17,7 +18,14 @@ export const deleteOne: DeleteOne = async function deleteOne(
1718
where,
1819
})
1920

20-
const doc = await Model.findOneAndDelete(query, options).lean()
21+
const doc = await Model.findOneAndDelete(query, {
22+
...options,
23+
projection: buildProjectionFromSelect({
24+
adapter: this,
25+
fields: this.payload.collections[collection].config.fields,
26+
select,
27+
}),
28+
}).lean()
2129

2230
let result: Document = JSON.parse(JSON.stringify(doc))
2331

‎packages/db-mongodb/src/find.ts

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ import type { MongooseAdapter } from './index.js'
77

88
import { buildSortParam } from './queries/buildSortParam.js'
99
import { buildJoinAggregation } from './utilities/buildJoinAggregation.js'
10+
import { buildProjectionFromSelect } from './utilities/buildProjectionFromSelect.js'
1011
import { sanitizeInternalFields } from './utilities/sanitizeInternalFields.js'
1112
import { withSession } from './withSession.js'
1213

@@ -21,6 +22,7 @@ export const find: Find = async function find(
2122
pagination,
2223
projection,
2324
req = {} as PayloadRequest,
25+
select,
2426
sort: sortArg,
2527
where,
2628
},
@@ -67,6 +69,14 @@ export const find: Find = async function find(
6769
useEstimatedCount,
6870
}
6971

72+
if (select) {
73+
paginationOptions.projection = buildProjectionFromSelect({
74+
adapter: this,
75+
fields: collectionConfig.fields,
76+
select,
77+
})
78+
}
79+
7080
if (this.collation) {
7181
const defaultLocale = 'en'
7282
paginationOptions.collation = {

‎packages/db-mongodb/src/findGlobal.ts

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,17 +4,23 @@ import { combineQueries } from 'payload'
44

55
import type { MongooseAdapter } from './index.js'
66

7+
import { buildProjectionFromSelect } from './utilities/buildProjectionFromSelect.js'
78
import { sanitizeInternalFields } from './utilities/sanitizeInternalFields.js'
89
import { withSession } from './withSession.js'
910

1011
export const findGlobal: FindGlobal = async function findGlobal(
1112
this: MongooseAdapter,
12-
{ slug, locale, req = {} as PayloadRequest, where },
13+
{ slug, locale, req = {} as PayloadRequest, select, where },
1314
) {
1415
const Model = this.globals
1516
const options = {
1617
...(await withSession(this, req)),
1718
lean: true,
19+
select: buildProjectionFromSelect({
20+
adapter: this,
21+
fields: this.payload.globals.config.find((each) => each.slug === slug).fields,
22+
select,
23+
}),
1824
}
1925

2026
const query = await Model.buildQuery({

‎packages/db-mongodb/src/findGlobalVersions.ts

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ import { buildVersionGlobalFields, flattenWhereToOperators } from 'payload'
66
import type { MongooseAdapter } from './index.js'
77

88
import { buildSortParam } from './queries/buildSortParam.js'
9+
import { buildProjectionFromSelect } from './utilities/buildProjectionFromSelect.js'
910
import { sanitizeInternalFields } from './utilities/sanitizeInternalFields.js'
1011
import { withSession } from './withSession.js'
1112

@@ -18,6 +19,7 @@ export const findGlobalVersions: FindGlobalVersions = async function findGlobalV
1819
page,
1920
pagination,
2021
req = {} as PayloadRequest,
22+
select,
2123
skip,
2224
sort: sortArg,
2325
where,
@@ -69,6 +71,7 @@ export const findGlobalVersions: FindGlobalVersions = async function findGlobalV
6971
options,
7072
page,
7173
pagination,
74+
projection: buildProjectionFromSelect({ adapter: this, fields: versionFields, select }),
7275
sort,
7376
useEstimatedCount,
7477
}

0 commit comments

Comments
 (0)