Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: route-level entry generators #9571

Merged
merged 23 commits into from May 4, 2023
Merged
Show file tree
Hide file tree
Changes from 21 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/fuzzy-insects-shake.md
@@ -0,0 +1,5 @@
---
'@sveltejs/kit': minor
---

feat: route-level entry generators via `export const entries`
2 changes: 1 addition & 1 deletion documentation/docs/20-core-concepts/40-page-options.md
Expand Up @@ -33,7 +33,7 @@ export const prerender = 'auto';

> If your entire app is suitable for prerendering, you can use [`adapter-static`](https://github.com/sveltejs/kit/tree/master/packages/adapter-static), which will output files suitable for use with any static webserver.

The prerenderer will start at the root of your app and generate files for any prerenderable pages or `+server.js` routes it finds. Each page is scanned for `<a>` elements that point to other pages that are candidates for prerendering — because of this, you generally don't need to specify which pages should be accessed. If you _do_ need to specify which pages should be accessed by the prerenderer, you can do so with the `entries` option in the [prerender configuration](configuration#prerender).
The prerenderer will start at the root of your app and generate files for any prerenderable pages or `+server.js` routes it finds. Each page is scanned for `<a>` elements that point to other pages that are candidates for prerendering — because of this, you generally don't need to specify which pages should be accessed. If you _do_ need to specify which pages should be accessed by the prerenderer, you can do so with the `entries` option in the [prerender configuration](configuration#prerender), or by exporting an entry generator from your dynamic route.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that's too little to add to the docs for this feature. We need some place to explain it properly.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree -- I called this out in the PR description. I wasn't sure where we'd want to put this, though.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably in page options at the very least, and maybe also reference it in some of the prerender-related docs.


While prerendering, the value of `building` imported from [`$app/environment`](modules#$app-environment) will be `true`.

Expand Down
14 changes: 14 additions & 0 deletions packages/kit/src/core/config/options.js
Expand Up @@ -229,6 +229,20 @@ const options = object(
}
),

handleEntryGeneratorMismatch: validate(
(/** @type {any} */ { message }) => {
throw new Error(
message +
`\nTo suppress or handle this error, implement \`handleEntryGeneratorMismatch\` in https://kit.svelte.dev/docs/configuration#prerender`
);
},
(input, keypath) => {
if (typeof input === 'function') return input;
if (['fail', 'warn', 'ignore'].includes(input)) return input;
throw new Error(`${keypath} should be "fail", "warn", "ignore" or a custom function`);
}
),

origin: validate('http://sveltekit-prerender', (input, keypath) => {
assert_string(input, keypath);

Expand Down
19 changes: 14 additions & 5 deletions packages/kit/src/core/postbuild/analyse.js
Expand Up @@ -2,14 +2,17 @@ import { join } from 'node:path';
import { pathToFileURL } from 'node:url';
import { get_option } from '../../utils/options.js';
import {
validate_common_exports,
validate_layout_exports,
validate_layout_server_exports,
validate_page_exports,
validate_page_server_exports,
validate_server_exports
} from '../../utils/exports.js';
import { load_config } from '../config/index.js';
import { forked } from '../../utils/fork.js';
import { should_polyfill } from '../../utils/platform.js';
import { installPolyfills } from '../../exports/node/polyfills.js';
import { resolve_entry } from '../../utils/routing.js';

export default forked(import.meta.url, analyse);

Expand Down Expand Up @@ -72,6 +75,8 @@ async function analyse({ manifest_path, env }) {
let prerender = undefined;
/** @type {any} */
let config = undefined;
/** @type {import('types').PrerenderEntryGenerator | undefined} */
let entries = undefined;

if (route.endpoint) {
const mod = await route.endpoint();
Expand All @@ -95,6 +100,7 @@ async function analyse({ manifest_path, env }) {
if (mod.OPTIONS) api_methods.push('OPTIONS');

config = mod.config;
entries = mod.entries;
}

if (route.page) {
Expand All @@ -109,8 +115,8 @@ async function analyse({ manifest_path, env }) {

for (const layout of layouts) {
if (layout) {
validate_common_exports(layout.server, layout.server_id);
validate_common_exports(layout.universal, layout.universal_id);
validate_layout_server_exports(layout.server, layout.server_id);
validate_layout_exports(layout.universal, layout.universal_id);
}
}

Expand All @@ -119,12 +125,13 @@ async function analyse({ manifest_path, env }) {
if (page.server?.actions) page_methods.push('POST');

validate_page_server_exports(page.server, page.server_id);
validate_common_exports(page.universal, page.universal_id);
validate_page_exports(page.universal, page.universal_id);
}

prerender = get_option(nodes, 'prerender') ?? false;

config = get_config(nodes);
entries ??= get_option(nodes, 'entries');
}

metadata.routes.set(route.id, {
Expand All @@ -136,7 +143,9 @@ async function analyse({ manifest_path, env }) {
api: {
methods: api_methods
},
prerender
prerender,
entries:
entries && (await entries()).map((entry_object) => resolve_entry(route.id, entry_object))
});
}

Expand Down
45 changes: 42 additions & 3 deletions packages/kit/src/core/postbuild/prerender.js
Expand Up @@ -127,6 +127,14 @@ async function prerender({ out, manifest_path, metadata, verbose, env }) {
}
);

const handle_entry_generator_mismatch = normalise_error_handler(
log,
config.prerender.handleEntryGeneratorMismatch,
({ generatedFromId, entry, matchedId }) => {
return `The entries export from ${generatedFromId} generated entry ${entry}, which was matched by ${matchedId} - see the \`handleEntryGeneratorMismatch\` option in https://kit.svelte.dev/docs/configuration#prerender for more info.`;
}
);

const q = queue(config.prerender.concurrency);

/**
Expand Down Expand Up @@ -164,23 +172,25 @@ async function prerender({ out, manifest_path, metadata, verbose, env }) {
* @param {string | null} referrer
* @param {string} decoded
* @param {string} [encoded]
* @param {string} [generated_from_id]
*/
function enqueue(referrer, decoded, encoded) {
function enqueue(referrer, decoded, encoded, generated_from_id) {
if (seen.has(decoded)) return;
seen.add(decoded);

const file = decoded.slice(config.paths.base.length + 1);
if (files.has(file)) return;

return q.add(() => visit(decoded, encoded || encodeURI(decoded), referrer));
return q.add(() => visit(decoded, encoded || encodeURI(decoded), referrer, generated_from_id));
}

/**
* @param {string} decoded
* @param {string} encoded
* @param {string?} referrer
* @param {string} [generated_from_id]
*/
async function visit(decoded, encoded, referrer) {
async function visit(decoded, encoded, referrer, generated_from_id) {
if (!decoded.startsWith(config.paths.base)) {
handle_http_error({ status: 404, path: decoded, referrer, referenceType: 'linked' });
return;
Expand All @@ -206,6 +216,20 @@ async function prerender({ out, manifest_path, metadata, verbose, env }) {
}
});

const encoded_id = response.headers.get('x-sveltekit-routeid');
const decoded_id = encoded_id && decode_uri(encoded_id);
if (
decoded_id !== null &&
generated_from_id !== undefined &&
decoded_id !== generated_from_id
) {
handle_entry_generator_mismatch({
generatedFromId: generated_from_id,
entry: decoded,
matchedId: decoded_id
});
}

const body = Buffer.from(await response.arrayBuffer());

save('pages', response, body, decoded, encoded, referrer, 'linked');
Expand Down Expand Up @@ -363,9 +387,18 @@ async function prerender({ out, manifest_path, metadata, verbose, env }) {
saved.set(file, dest);
}

/** @type {Array<{ id: string, entries: Array<string>}>} */
const route_level_entries = [];
for (const [id, { entries }] of metadata.routes.entries()) {
if (entries) {
route_level_entries.push({ id, entries });
}
}

if (
config.prerender.entries.length > 1 ||
config.prerender.entries[0] !== '*' ||
route_level_entries.length > 0 ||
prerender_map.size > 0
) {
// Only log if we're actually going to do something to not confuse users
Expand All @@ -386,6 +419,12 @@ async function prerender({ out, manifest_path, metadata, verbose, env }) {
}
}

for (const { id, entries } of route_level_entries) {
for (const entry of entries) {
enqueue(null, config.paths.base + entry, undefined, id);
}
}

await q.done();

// handle invalid fragment links
Expand Down
6 changes: 6 additions & 0 deletions packages/kit/src/core/sync/write_types/index.js
Expand Up @@ -194,6 +194,12 @@ function update_types(config, routes, route, to_delete = new Set()) {
.join('; ')} }`
);

if (route.params.length > 0) {
exports.push(
`export type EntryGenerator = () => Promise<Array<RouteParams>> | Array<RouteParams>;`
);
}

declarations.push(`type RouteId = '${route.id}';`);

// These could also be placed in our public types, but it would bloat them unnecessarily and we may want to change these in the future
Expand Down
4 changes: 2 additions & 2 deletions packages/kit/src/runtime/client/client.js
Expand Up @@ -31,7 +31,7 @@ import { stores } from './singletons.js';
import { unwrap_promises } from '../../utils/promises.js';
import * as devalue from 'devalue';
import { INDEX_KEY, PRELOAD_PRIORITIES, SCROLL_KEY, SNAPSHOT_KEY } from './constants.js';
import { validate_common_exports } from '../../utils/exports.js';
import { validate_page_exports } from '../../utils/exports.js';
import { compact } from '../../utils/array.js';
import { validate_depends } from '../shared.js';

Expand Down Expand Up @@ -594,7 +594,7 @@ export function create_client(app, target) {
const node = await loader();

if (DEV) {
validate_common_exports(node.universal);
validate_page_exports(node.universal);
}

if (node.universal?.load) {
Expand Down
13 changes: 9 additions & 4 deletions packages/kit/src/runtime/server/respond.js
Expand Up @@ -20,7 +20,9 @@ import { add_cookies_to_headers, get_cookies } from './cookie.js';
import { create_fetch } from './fetch.js';
import { Redirect } from '../control.js';
import {
validate_common_exports,
validate_layout_exports,
validate_layout_server_exports,
validate_page_exports,
validate_page_server_exports,
validate_server_exports
} from '../../utils/exports.js';
Expand Down Expand Up @@ -193,8 +195,11 @@ export async function respond(request, options, manifest, state) {

for (const layout of layouts) {
if (layout) {
validate_common_exports(layout.server, /** @type {string} */ (layout.server_id));
validate_common_exports(
validate_layout_server_exports(
layout.server,
/** @type {string} */ (layout.server_id)
);
validate_layout_exports(
layout.universal,
/** @type {string} */ (layout.universal_id)
);
Expand All @@ -203,7 +208,7 @@ export async function respond(request, options, manifest, state) {

if (page) {
validate_page_server_exports(page.server, /** @type {string} */ (page.server_id));
validate_common_exports(page.universal, /** @type {string} */ (page.universal_id));
validate_page_exports(page.universal, /** @type {string} */ (page.universal_id));
}
}

Expand Down
48 changes: 31 additions & 17 deletions packages/kit/src/utils/exports.js
@@ -1,9 +1,7 @@
/**
* @param {string[]} expected
* @param {Set<string>} expected
*/
function validator(expected) {
const set = new Set(expected);

/**
* @param {any} module
* @param {string} [file]
Expand All @@ -12,11 +10,13 @@ function validator(expected) {
if (!module) return;

for (const key in module) {
if (key[0] === '_' || set.has(key)) continue; // key is valid in this module
if (key[0] === '_' || expected.has(key)) continue; // key is valid in this module

const values = [...expected.values()];

const hint =
hint_for_supported_files(key, file?.slice(file.lastIndexOf('.'))) ??
`valid exports are ${expected.join(', ')}, or anything with a '_' prefix`;
`valid exports are ${values.join(', ')}, or anything with a '_' prefix`;

throw new Error(`Invalid export '${key}'${file ? ` in ${file}` : ''} (${hint})`);
}
Expand All @@ -33,34 +33,45 @@ function validator(expected) {
function hint_for_supported_files(key, ext = '.js') {
let supported_files = [];

if (valid_common_exports.includes(key)) {
if (valid_layout_exports.has(key)) {
supported_files.push(`+layout${ext}`);
}

if (valid_page_exports.has(key)) {
supported_files.push(`+page${ext}`);
}

if (valid_page_server_exports.includes(key)) {
if (valid_layout_server_exports.has(key)) {
supported_files.push(`+layout.server${ext}`);
}

if (valid_page_server_exports.has(key)) {
supported_files.push(`+page.server${ext}`);
}

if (valid_server_exports.includes(key)) {
if (valid_server_exports.has(key)) {
supported_files.push(`+server${ext}`);
}

if (supported_files.length > 0) {
return `'${key}' is a valid export in ${supported_files.join(` or `)}`;
return `'${key}' is a valid export in ${supported_files.slice(0, -1).join(`, `)}${
supported_files.length > 1 ? ' or ' : ''
}${supported_files.at(-1)}`;
}
}

const valid_common_exports = ['load', 'prerender', 'csr', 'ssr', 'trailingSlash', 'config'];
const valid_page_server_exports = [
const valid_layout_exports = new Set([
'load',
'prerender',
'csr',
'ssr',
'actions',
'trailingSlash',
'config'
];
const valid_server_exports = [
]);
const valid_page_exports = new Set([...valid_layout_exports, 'entries']);
const valid_layout_server_exports = new Set([...valid_layout_exports, 'actions']);
const valid_page_server_exports = new Set([...valid_layout_server_exports, 'entries']);
const valid_server_exports = new Set([
'GET',
'POST',
'PATCH',
Expand All @@ -69,9 +80,12 @@ const valid_server_exports = [
'OPTIONS',
'prerender',
'trailingSlash',
'config'
];
'config',
'entries'
]);

export const validate_common_exports = validator(valid_common_exports);
export const validate_layout_exports = validator(valid_layout_exports);
export const validate_page_exports = validator(valid_page_exports);
export const validate_layout_server_exports = validator(valid_layout_server_exports);
export const validate_page_server_exports = validator(valid_page_server_exports);
export const validate_server_exports = validator(valid_server_exports);