Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add auto static/dynamic #7293

Merged
merged 39 commits into from
May 22, 2019
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
ffe825c
Add automatic exporting of pages with no getInitialProps
ijjk May 9, 2019
89a3ad3
Add support for exporting serverless to static
ijjk May 9, 2019
4ae98f8
Fix missing runtimeEnv when requiring page, re-add warning
ijjk May 9, 2019
f53b31c
Update flying-shuttle test
ijjk May 9, 2019
9a325a2
Merge remote-tracking branch 'upstream/canary' into add/auto-static
ijjk May 10, 2019
20c3e87
revert un-used pagesManifest change
ijjk May 10, 2019
6a96aff
remove query.amp RegExp test
ijjk May 10, 2019
4757537
Fix windows backslashes not being replaced
ijjk May 10, 2019
89b2da3
Merge remote-tracking branch 'upstream/canary' into add/auto-static
ijjk May 10, 2019
ceba225
Merge remote-tracking branch 'upstream/canary' into add/auto-static
ijjk May 13, 2019
9973273
Re-enable serverless support for next start
ijjk May 13, 2019
2df95ab
Merge branch 'canary' into add/auto-static
ijjk May 13, 2019
68927e3
bump
ijjk May 13, 2019
6ccd760
Merge remote-tracking branch 'upstream/canary' into add/auto-static
ijjk May 14, 2019
4e2e958
Fix getInitialProps check
ijjk May 14, 2019
7d33924
Fix incorrect error check
ijjk May 14, 2019
70cfd06
Re-add check for reserved pages
ijjk May 14, 2019
a547bec
Fix static check
ijjk May 14, 2019
333430a
Merge remote-tracking branch 'upstream/canary' into add/auto-static
ijjk May 15, 2019
a30f548
Update to ignore /api pages and clean up some tests
ijjk May 15, 2019
e5aada9
Re-add needed next.config for test and correct behavior
ijjk May 15, 2019
6f85821
Merge remote-tracking branch 'upstream/canary' into add/auto-static
ijjk May 16, 2019
7517a99
Update RegExp for ignored pages for auto-static
ijjk May 16, 2019
5ae7cf0
Add checking for custom getInitialProps in pages/_app
ijjk May 17, 2019
9672558
Merge branch 'canary' into add/auto-static
ijjk May 17, 2019
ad999e9
Update isPageStatic logic to only use default export
ijjk May 17, 2019
3a971e3
Re-add retrying to CircleCi
ijjk May 17, 2019
56b873a
Update query during dev to only have values
ijjk May 17, 2019
4a66457
Fix test
ijjk May 17, 2019
07e2605
Merge branch 'canary' into add/auto-static
ijjk May 17, 2019
c7710b0
Merge remote-tracking branch 'upstream/canary' into add/auto-static
ijjk May 18, 2019
b6414ab
Merge branch 'add/auto-static' of github.com:ijjk/next.js into add/au…
ijjk May 18, 2019
b188864
Add warning when page without default export is
ijjk May 18, 2019
e377e3f
Merge remote-tracking branch 'upstream/canary' into add/auto-static
ijjk May 18, 2019
99ca34e
Fix backslashes not being replaced
ijjk May 18, 2019
e29b946
Merge remote-tracking branch 'upstream/canary' into add/auto-static
ijjk May 21, 2019
e80a694
Merge remote-tracking branch 'upstream/canary' into add/auto-static
ijjk May 22, 2019
7be354c
Integrate auto-static with flying-shuttle
ijjk May 22, 2019
f2748ce
Add autoExport for opting in
ijjk May 22, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions packages/next-server/lib/constants.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ export const BUILD_MANIFEST = 'build-manifest.json'
export const REACT_LOADABLE_MANIFEST = 'react-loadable-manifest.json'
export const CHUNK_GRAPH_MANIFEST = 'compilation-modules.json'
export const SERVER_DIRECTORY = 'server'
export const SERVERLESS_DIRECTORY = 'serverless'
export const CONFIG_FILE = 'next.config.js'
export const BUILD_ID_FILE = 'BUILD_ID'
export const BLOCKED_PAGES = [
Expand Down
8 changes: 8 additions & 0 deletions packages/next-server/server/next-server.ts
Original file line number Diff line number Diff line change
Expand Up @@ -319,7 +319,15 @@ export default class Server {
query: ParsedUrlQuery = {},
opts: any,
) {
// try serving a static AMP version first
if (query.amp) {
try {
const result = await loadComponents(this.distDir, this.buildId, (pathname === '/' ? '/index' : pathname) + '.amp')
if (typeof result.Component === 'string') return result.Component
} catch (_) {}
ijjk marked this conversation as resolved.
Show resolved Hide resolved
}
const result = await loadComponents(this.distDir, this.buildId, pathname)
if (typeof result.Component === 'string') return result.Component
return renderToHTML(req, res, pathname, query, { ...result, ...opts })
}

Expand Down
2 changes: 1 addition & 1 deletion packages/next-server/server/render.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -304,7 +304,7 @@ export async function renderToHTML(

const ampMode = {
enabled: false,
hasQuery: Boolean(query.amp && /^(y|yes|true|1)/i.test(query.amp.toString())),
hasQuery: Boolean(query.amp),
}

if (ampBindInitData) {
Expand Down
7 changes: 7 additions & 0 deletions packages/next-server/server/require.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,11 @@
import fs from 'fs'
import {join} from 'path'
import {promisify} from 'util'
import {PAGES_MANIFEST, SERVER_DIRECTORY} from '../lib/constants'
import { normalizePagePath } from './normalize-page-path'

const readFile = promisify(fs.readFile)

export function pageNotFoundError(page: string): Error {
const err: any = new Error(`Cannot find module for page: ${page}`)
err.code = 'ENOENT'
Expand Down Expand Up @@ -34,5 +38,8 @@ export function getPagePath(page: string, distDir: string): string {

export function requirePage(page: string, distDir: string): any {
const pagePath = getPagePath(page, distDir)
if (pagePath.endsWith('.html')) {
return readFile(pagePath, 'utf8')
}
return require(pagePath)
}
88 changes: 78 additions & 10 deletions packages/next/build/index.ts
Original file line number Diff line number Diff line change
@@ -1,12 +1,14 @@
import chalk from 'chalk'
import {
PAGES_MANIFEST,
CHUNK_GRAPH_MANIFEST,
PHASE_PRODUCTION_BUILD,
} from 'next-server/constants'
import loadConfig from 'next-server/next-config'
import nanoid from 'next/dist/compiled/nanoid/index.js'
import path from 'path'

import fs from 'fs'
import { promisify } from 'util'
import formatWebpackMessages from '../client/dev-error-overlay/format-webpack-messages'
import { recursiveDelete } from '../lib/recursive-delete'
import { verifyTypeScriptSetup } from '../lib/verifyTypeScriptSetup'
Expand All @@ -29,6 +31,15 @@ import {
getPageChunks,
} from './webpack/plugins/chunk-graph-plugin'
import { writeBuildId } from './write-build-id'
import { recursiveReadDir } from '../lib/recursive-readdir';
import mkdirpOrig from 'mkdirp'

const fsUnlink = promisify(fs.unlink)
const fsRmdir = promisify(fs.rmdir)
const fsMove = promisify(fs.rename)
const fsReadFile = promisify(fs.readFile)
const fsWriteFile = promisify(fs.writeFile)
const mkdirp = promisify(mkdirpOrig)

export default async function build(dir: string, conf = null): Promise<void> {
if (!(await isWriteable(dir))) {
Expand Down Expand Up @@ -238,7 +249,8 @@ export default async function build(dir: string, conf = null): Promise<void> {
console.log(chalk.green('Compiled successfully.\n'))
}

const pageInfos = new Map()
const pageInfos: Map<string, any> = new Map()
const staticPages: Set<string> = new Set()
const distPath = path.join(dir, config.distDir)
let pageKeys = Object.keys(mappedPages)

Expand All @@ -250,19 +262,80 @@ export default async function build(dir: string, conf = null): Promise<void> {
distPath,
buildId,
false,
config.target === 'serverless'
config.target === 'serverless',
config
)

pageInfos.set(page, {
...(info || {}),
chunks,
})

if (!(typeof info.serverSize === 'number')) {
if (info.static) staticPages.add(page)

if (!(typeof info.clientSize === 'number')) {
pageKeys = pageKeys.filter(pg => pg !== page)
}
}

if (flyingShuttle) {
await flyingShuttle.save()
}
await writeBuildId(distDir, buildId, selectivePageBuilding)

if (staticPages.size > 0) {
const exportApp = require('../export').default
const exportOptions = {
silent: true,
buildExport: true,
pages: Array.from(staticPages),
outdir: path.join(distDir, 'export'),
}
const exportConfig = {
...config,
exportPathMap: (defaultMap: any) => defaultMap,
experimental: {
...config.experimental,
exportTrailingSlash: false,
}
}
await exportApp(dir, exportOptions, exportConfig)
const toMove = await recursiveReadDir(exportOptions.outdir, /.*\.html$/)

let serverDir: string = ''
// remove server bundles that were exported
for (const page of staticPages) {
const { serverBundle } = pageInfos.get(page)
if (!serverDir) serverDir = path.dirname(serverBundle)
await fsUnlink(serverBundle)
}
let pagesManifest: any = {}
const manifestPath = path.join(distDir, 'server/', PAGES_MANIFEST)

if (target !== 'serverless') {
pagesManifest = JSON.parse(await fsReadFile(manifestPath, 'utf8'))
}

for (const file of toMove) {
const orig = path.join(exportOptions.outdir, file)
const dest = path.join(serverDir, file)
const relativeDest = path.join('static', buildId, 'pages', file).replace(/\\/g, '/')
let page = file.split('.html')[0].replace(/\\/g, '/')
pagesManifest[page] = relativeDest
page = page === '/index' ? '/' : page
pagesManifest[page] = relativeDest
await mkdirp(path.dirname(dest))
await fsMove(orig, dest)
}
// remove temporary export folder
await recursiveDelete(exportOptions.outdir)
await fsRmdir(exportOptions.outdir)

if (target !== 'serverless') {
await fsWriteFile(manifestPath, JSON.stringify(pagesManifest), 'utf8')
}
}

if (Array.isArray(configs[0].plugins)) {
configs[0].plugins.some((plugin: any) => {
if (plugin.ampPages) {
Expand All @@ -271,6 +344,7 @@ export default async function build(dir: string, conf = null): Promise<void> {
if (info) {
info.ampOnly = true
pageInfos.set(pg, info)
if (pageKeys.indexOf(pg) < 0) pageKeys.push(pg)
}
})
}
Expand All @@ -279,10 +353,4 @@ export default async function build(dir: string, conf = null): Promise<void> {
}

printTreeView(pageKeys, pageInfos)

if (flyingShuttle) {
await flyingShuttle.save()
}

await writeBuildId(distDir, buildId, selectivePageBuilding)
}
26 changes: 20 additions & 6 deletions packages/next/build/utils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ import { recursiveReadDir } from '../lib/recursive-readdir'
const fsStat = promisify(fs.stat)
const fsExists = promisify(fs.exists)
const fsReadFile = promisify(fs.readFile)
const nextEnvConfig = require('next-server/config')

export function collectPages(
directory: string,
Expand Down Expand Up @@ -89,7 +90,7 @@ export function printTreeView(list: string[], pageInfos: Map<string, PageInfo>)
` Client size: ${prettyBytes(clientSize)}`
}

if (sizes) console.log(sizes)
if (sizes !== ' \x1b[90m|') console.log(sizes)
console.log(` \x1b[90m${i === list.length - 1 ? '└' : '|'}`);
})

Expand Down Expand Up @@ -215,6 +216,7 @@ export async function getPageInfo(
buildId: string,
dev: boolean,
serverless?: boolean,
nextConfig?: any
) {
const info: any = {}
const staticPath = dev ? 'development' : buildId
Expand All @@ -226,18 +228,30 @@ export async function getPageInfo(
: path.join(distPath, 'server/static', staticPath, 'pages')

const serverBundle = path.join(serverPath, `${page}.js`)
info.serverBundle = serverBundle
info.clientBundle = clientBundle

if (!dev) {
try {
info.serverSize = (await fsStat(serverBundle)).size
} catch (_) {}
if (!page.match(/(_app|_error|_document)/)) {
nextEnvConfig.setConfig({
publicRuntimeConfig: nextConfig.publicRuntimeConfig,
serverRuntimeConfig: nextConfig.serverRuntimeConfig,
})
// require server bundle to check if it has `getInitialProps`
const mod = require(serverBundle)
const Component = mod.default || mod
info.static = typeof Component.getInitialProps !== 'function'
}

if (!info.static) {
try {
info.serverSize = (await fsStat(serverBundle)).size
} catch (_) {}
}
try {
info.clientSize = (await fsStat(clientBundle)).size
} catch (_) {}
}

if (page.match(/(_app|_error|_document)/)) return info

return info
}
20 changes: 12 additions & 8 deletions packages/next/build/webpack/loaders/next-serverless-loader.ts
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,8 @@ const nextServerlessLoader: loader.Loader = function () {
import Error from '${absoluteErrorPath}';
import App from '${absoluteAppPath}';
import Component from '${absolutePagePath}';
async function renderReqToHTML(req, res) {
export default Component
export async function renderReqToHTML(req, res, fromExport) {
const options = {
App,
Document,
Expand All @@ -53,15 +54,18 @@ const nextServerlessLoader: loader.Loader = function () {
ampBindInitData: ${ampBindInitData === true || ampBindInitData === 'true'}
}
const parsedUrl = parse(req.url, true)
const renderOpts = Object.assign(
{
Component,
dataOnly: req.headers && (req.headers.accept || '').indexOf('application/amp.bind+json') !== -1,
},
options,
)
try {
${page === '/_error' ? `res.statusCode = 404` : ''}
const result = await renderToHTML(req, res, "${page}", parsedUrl.query, Object.assign(
{
Component,
dataOnly: req.headers && (req.headers.accept || '').indexOf('application/amp.bind+json') !== -1,
},
options,
))
const result = await renderToHTML(req, res, "${page}", parsedUrl.query, renderOpts)

if (fromExport) return { html: result, renderOpts }
return result
} catch (err) {
if (err.code === 'ENOENT') {
Expand Down
13 changes: 9 additions & 4 deletions packages/next/export/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ export default async function (dir, options, configuration) {
const distDir = join(dir, nextConfig.distDir)
const subFolders = nextConfig.experimental.exportTrailingSlash

if (nextConfig.target !== 'server') throw new Error('Cannot export when target is not server. https://err.sh/zeit/next.js/next-export-serverless')
if (!options.buildExport && nextConfig.target !== 'server') throw new Error('Cannot export when target is not server. https://err.sh/zeit/next.js/next-export-serverless')

log(`> using build directory: ${distDir}`)

Expand All @@ -35,9 +35,9 @@ export default async function (dir, options, configuration) {
}

const buildId = readFileSync(join(distDir, BUILD_ID_FILE), 'utf8')
const pagesManifest = require(join(distDir, SERVER_DIRECTORY, PAGES_MANIFEST))
const pagesManifest = !options.pages && require(join(distDir, SERVER_DIRECTORY, PAGES_MANIFEST))

const pages = Object.keys(pagesManifest)
const pages = options.pages || Object.keys(pagesManifest)
const defaultPathMap = {}

for (const page of pages) {
Expand Down Expand Up @@ -141,6 +141,7 @@ export default async function (dir, options, configuration) {
}
)
}
const workers = new Set()

await Promise.all(
chunks.map(
Expand All @@ -149,6 +150,7 @@ export default async function (dir, options, configuration) {
const worker = fork(require.resolve('./worker'), [], {
env: process.env
})
workers.add(worker)
worker.send({
distDir,
buildId,
Expand All @@ -158,7 +160,8 @@ export default async function (dir, options, configuration) {
renderOpts,
serverRuntimeConfig,
concurrency,
subFolders
subFolders,
serverless: nextConfig.target === 'serverless'
})
worker.on('message', ({ type, payload }) => {
if (type === 'progress' && progress) {
Expand All @@ -176,6 +179,8 @@ export default async function (dir, options, configuration) {
)
)

workers.forEach(worker => worker.kill())

if (Object.keys(ampValidations).length) {
console.log(formatAmpMessages(ampValidations))
}
Expand Down
Loading