Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[template-renderer] Make it possible to not automatically render used async scripts #9847

Open
maoberlehner opened this issue Apr 8, 2019 · 26 comments

Comments

@maoberlehner
Copy link
Contributor

What problem does this feature solve?

In oder to make https://github.com/maoberlehner/vue-lazy-hydration more useful it would be great if we could prevent the template renderer from automatically injecting async scripts. The way vue-lazy-hydration works is, that it behaves differently on the server than it does on the client. On the server the script (of an async component) is loaded immediately so the template renderer correctly detects that it is used. But on the client the script might not be needed at all but because the template renderer has already injected it it is immediately loaded on page load.

There is currently kind of a backlash against loading huge amounts of JavaScript. vue-lazy-hydration can help with removing a lot of unnecessary JavaScript on server side rendered, mostly static sites like blogs and documentation. But currently it can't completely prevent loading all of the unnecessary JavaScript because of the way how template renderer works.

Here is the relevant line in the code: https://github.com/vuejs/vue/blob/dev/src/server/template-renderer/index.js#L226

What does the proposed API look like?

I propose to make this configurable:

const renderer = createBundleRenderer(serverBundle, {
  template,
  renderUsedAsyncScripts: false,
});
@maoberlehner maoberlehner changed the title [template-renderer] Make it possible to not automatically render used async files [template-renderer] Make it possible to not automatically render used async scripts Apr 8, 2019
maoberlehner added a commit to maoberlehner/vue-lazy-hydration that referenced this issue May 23, 2019
@matthewp
Copy link

I'm working on an app where this would be a huge improvement to lighthouse scores. Would a PR be accepted to include this option?

jeanphilippeds pushed a commit to jeanphilippeds/vue that referenced this issue Nov 4, 2019
Provide a new option (shouldRenderAsyncScripts) to render or not async scripts in TemplateRenderer

fix vuejs#9847
jeanphilippeds pushed a commit to jeanphilippeds/vue that referenced this issue Nov 4, 2019
@simllll
Copy link

simllll commented Dec 30, 2019

this is the related PR #10794
anything we can do to get this into the core?

@simplenotezy
Copy link

That would be neat, Vue & Nuxt applications would see a huge performance boost, especially on LightHouse scores.

jeanphilippeds pushed a commit to jeanphilippeds/vue that referenced this issue Mar 20, 2020
Provide a new option (shouldRenderAsyncScripts) to render or not async scripts in TemplateRenderer

fix vuejs#9847
jeanphilippeds pushed a commit to jeanphilippeds/vue that referenced this issue Mar 20, 2020
@mnooblet
Copy link

mnooblet commented May 9, 2020

this would be super awesome to have, I'm currently struggling with lighthouse performance and this would make things a lot easier... ( want that nearly perfect score :) )

@simllll
Copy link

simllll commented May 9, 2020

If you use nuxt, you can actually work around either by using modern mode (due to a bug) or by using a regex to remove the async deferred chunks from html ouput (render:route, see https://nuxtjs.org/api/internals-renderer#hooks)

example plugin:

import consola from 'consola';

const logger = consola.withScope('js-optimization:module');

const bodyRegex = /<body[^>]*>(.*)<\/body>/s;
// list of all JS includes
const scriptRegex = /<script[\w"= ]*src="(.*?)".*?><\/script>/g;
// essenitials are all with "pages" or ending with "app.js"
const validScriptRegex = /\/(legacy-)?.*?-(pages.*?|.*app).js/;

module.exports = async function JSOptimizer(moduleOptions) {
	if (!moduleOptions.setOutputFilenames) {
		logger.error(
			'JS optimization works only when you explicitly opt in for overwriting output filenames in nuxt!'
		);
		return;
	}

	if (!this.options.build) this.options.build = {};
	this.options.build.filenames = {
		...this.options.build.filenames,
		app: ({ isModern, isDev }) =>
			`${!isModern ? 'legacy-' : ''}${!isDev ? '[contenthash]' : '[name]'}-app.js`,
		chunk: ({ isModern, isDev }) =>
			`${!isModern ? 'legacy-' : ''}${!isDev ? '[contenthash]-' : ''}[name].js`
	};

	this.nuxt.hook('render:route', async (url, page, { req, res }) => {
		if (!page.html || (res.statusCode && res.statusCode !== 200) || page.redirected || page.error) {
			if (moduleOptions.debug) {
				logger.info(
					'skipping optimize JS render:route',
					JSON.stringify({
						url,
						isAmp: req.isAMP,
						matchedRoute: req.matchedRoute,
						page: page.html.length,
						statusCode: res.statusCode,
						error: page.error,
						redirected: page.redirected
					})
				);
			}
			return;
		}

		if (moduleOptions.debug) {
			logger.info(
				'optimize JS render:route',
				JSON.stringify({ url, isAmp: req.isAMP, matchedRoute: req.matchedRoute })
			);
		}

		if (!req.isAMP) {
			// remove all non-essential JS files

			let { html } = page;

			const bodyString = bodyRegex.exec(html);

			if (!bodyString || !bodyString[0]) {
				logger.warn('no body tag found', html);
				return;
			}
			const body = bodyString[0];

			const links = body.matchAll(scriptRegex);

			for (const match of links) {
				if (!validScriptRegex.test(match[1])) {
					// remove non essential JS
					html = html
						.replace(match[0], '') // script tag
						.replace(`<link rel="modulepreload" href="${match[1]}" as="script">`, '') // module preload
						.replace(`<link rel="preload" href="${match[1]}" as="script">`, ''); // preload

					if (moduleOptions.debug) {
						logger.info('removed js tags for', match[1]);
					}
				}
			}

			page.html = html; // set new response
		}
	});

	logger.success('JS optimization module initialised');
};

save it in modules/js-optimization.js and add ['modules/js-optimizer.js', { setOutputFilenames: true }] to the modules of nuxt.config.js

@Moncef12
Copy link

@simllll
Tried this it works, one thing I had to block loading the preload links. awesome thank you!

@simllll
Copy link

simllll commented May 14, 2020

@simllll
Tried this it works, one thing I had to block loading the preload links. awesome thank you!

You are welcome, I just updated the example plugin to also get rid of preload tags. How did you remove them?

@Moncef12
Copy link

I used another replace with the matched scripts:

head = head.replace(`<link rel="preload" href="${match[1]}" as="script">`, '');

Will update to use fully your module 👍

@asennoussi
Copy link

@simllll I've got a question here.
Will this manipulation work with a universal app? i.e remove all the unnecessary JS, but still have the SPA behavior after the SSR rendering?
Thank you

@simllll
Copy link

simllll commented May 15, 2020

@simllll I've got a question here.
Will this manipulation work with a universal app? i.e remove all the unnecessary JS, but still have the SPA behavior after the SSR rendering?
Thank you

Yes, that's exactly how we use it. It only makes sense to use with lazy hydration though, otherwise the JS will be needed for the inital page load and removing the preload tags would just slow down the page rendering.

@asennoussi
Copy link

Gotcha! It makes perfect sense.
I'll get back to you with the results tonight.

@asennoussi
Copy link

Unfortunately, I couldn't see any change in the scripts loaded nor in the performance.
So If I understand correctly your design, it's something like this:
Use Lazy hydration on the page components:

 <LazyHydrate ssr-only>
         <ArticleContent :content="article.content"/>
    </LazyHydrate>

Then

import LazyHydrate from 'vue-lazy-hydration';
 
export default {
  components: {
    LazyHydrate,
    ArticleContent: () => import('./ArticleContent.vue'),
  },
  // ...
};

Then build: nuxt build --universal
I could see a change in the scripts names (adding legacy-) But not much for the performance.
Your help is appreciated

@simllll
Copy link

simllll commented May 17, 2020

Your example looks right, you can easily add some output in "mounted" hook to see if everything works as expected. Lazy hydration with ssr-only mode, shouldn't call the mounted hook at all. So e.g if you place a console log in there, the browser should not log it.
For "simple" components with basically no logic in it and components that only occur once in your page, the performance benefit is small. But if you apply it to more components, you will see two things:

  • initial js loading & parsing is faster (lighthouse "first cpu idle")
  • page "reacts" faster (Estimated Input Latency)

It's very likely that you will only notice this on mobile phones though. Try reducing cpu and network performance in chrom dev tools to debug and analyze this on your desktop pc. Check out the performance tab, and watch the JS profiler.

@asennoussi
Copy link

Yes, did that and the JS files for that specific page were indeed deleted. However, the performance didn't improve at all. I guess there is a core issue with vuetify performance.
I mean, if you start a fresh vue project and add vuetify, and run the Lighthouse score, you will get a pretty high First CPU idle and TTI.
There is an open issue with vuetify to fix this and honestly if it doesn't get any love soon, we are going to move out of Vuetify:
vuetifyjs/vuetify#7265

@simplenotezy
Copy link

simplenotezy commented Jun 4, 2020

@simllll Hey Simon, I did exactly what you told, but I am getting:

│ ✖ Nuxt Fatal Error │
│ │
│ Error: Module modules/js-optimizer.js not found. │
│ │

Update:

Had to call it ['~/modules/js-optimization.js', { setOutputFilenames: true }],

@simplenotezy
Copy link

@simllll I am getting this error:

 FATAL  body.matchAll is not a function                                                    01:59:47

  at JSOptimizer.nuxt.hook (modules/js-optimization.js:68:23)
  at o (node_modules/hable/dist/hable.js:1:1052)
  at o.then.o (node_modules/hable/dist/hable.js:1:270)
  at process._tickCallback (internal/process/next_tick.js:68:7)

@simplenotezy
Copy link

simplenotezy commented Jun 5, 2020

@simllll Had to install the matchall polyfill (https://www.npmjs.com/package/string.prototype.matchall) (aparently only es2020). My file ended up looking like so:

import consola from 'consola';

import shim from 'string.prototype.matchall/shim'
shim()

const logger = consola.withScope('js-optimization:module');

const bodyRegex = /<body[^>]*>(.*)<\/body>/s;
// list of all JS includes
const scriptRegex = /<script[\w"= ]*src="(.*?)".*?><\/script>/g;
// essenitials are all with "pages" or ending with "app.js"
const validScriptRegex = /\/(legacy-)?.*?-(pages.*?|.*app).js/;

// eslint-disable-next-line require-await
module.exports = async function JSOptimizer (moduleOptions) {
	if (!moduleOptions.setOutputFilenames) {
		logger.error(
			'JS optimization works only when you explicitly opt in for overwriting output filenames in nuxt!'
		);
		return;
	}

	if (!this.options.build) { this.options.build = {}; }
	this.options.build.filenames = {
		...this.options.build.filenames,
		app: ({ isModern, isDev }) =>
			`${!isModern ? 'legacy-' : ''}${!isDev ? '[contenthash]' : '[name]'}-app.js`,
		chunk: ({ isModern, isDev }) =>
			`${!isModern ? 'legacy-' : ''}${!isDev ? '[contenthash]-' : ''}[name].js`
	};

	this.nuxt.hook('render:route', async (url, page, { req, res }) => {
		if (!page.html || (res.statusCode && res.statusCode !== 200) || page.redirected || page.error) {
			if (moduleOptions.debug) {
				logger.info(
					'skipping optimize JS render:route',
					JSON.stringify({
						url,
						isAmp: req.isAMP,
						matchedRoute: req.matchedRoute,
						page: page.html.length,
						statusCode: res.statusCode,
						error: page.error,
						redirected: page.redirected
					})
				);
			}
			return;
		}

		if (moduleOptions.debug) {
			logger.info(
				'optimize JS render:route',
				JSON.stringify({ url, isAmp: req.isAMP, matchedRoute: req.matchedRoute })
			);
		}

		if (!req.isAMP) {
			// remove all non-essential JS files

			let { html } = page;

			const bodyString = bodyRegex.exec(html);

			if (!bodyString || !bodyString[0]) {
				logger.warn('no body tag found', html);
				return;
			}
			const body = bodyString[0].toString();

			const links = body.matchAll(scriptRegex);

			for (const match of links) {
				if (!validScriptRegex.test(match[1])) {
					// remove non essential JS
					html = html
						.replace(match[0], '') // script tag
						.replace(`<link rel="modulepreload" href="${match[1]}" as="script">`, '') // module preload
						.replace(`<link rel="preload" href="${match[1]}" as="script">`, ''); // preload

					if (moduleOptions.debug) {
						logger.info('removed js tags for', match[1]);
					}
				}
			}

			page.html = html; // set new response
		}
	});

	logger.success('JS optimization module initialised');
}

However, it didn't work. No scripts would be loaded to page at all.

When visiting any page, having yarn dev running, this is the only log output I get:

ℹ optimize JS render:route {"url":"/product/lace-ring-pave-2"}      js-optimization:module 02:31:51
ℹ removed js tags for /_nuxt/legacy-lang-da-DK.js                   js-optimization:module 02:31:51

@lautr
Copy link

lautr commented Jul 14, 2020

seems this workaround breaks with

nuxt v2.13.2
when http2.push is enabled

nuxt v2.13.3
always

does somebody has an updated solution?

@simplenotezy
Copy link

@maoberlehner are you aware of a workaround to this solution posted by @simllll which unfortunately doesn't seem to work anymore?

@simplenotezy
Copy link

Actually, I just tried again to use @simllll's solution on latest Nuxt version, and this time I didn't need the polyfill, and it worked out of the box. Will run a pagespeed test soon to see how/if things have changed.

@arkhamvm
Copy link

arkhamvm commented Nov 3, 2020

@simplenotezy Can you share any results, please?

@arkhamvm
Copy link

arkhamvm commented Nov 3, 2020

@simllll Thanks for js-optimizer.js script! But looks like it removes layout and common styles too, which should be kept.
For my example, it removes all scripts, except /_nuxt/runtime-app.js. So vue-awesome-swiper, v-popover and client-only wont work.

@Velikolay
Copy link

Managed to adapt @simllll workaround to work with nuxt full static mode.
You should add ['~/modules/js-optimizer.js', { setOutputFilenames: true }] to your buildModules and utilise the generate:page hook to edit the html. Also make sure component auto import is not messing with the chunks you want to lazy load - e.g. you'll have to explicitly ignore them from in the nuxt.config.js and import manually.

import consola from 'consola'

const logger = consola.withScope('js-optimization:module')

const bodyRegex = /<body[^>]*>(.*)<\/body>/s
// list of all JS includes
const scriptRegex = /<script[\w"= ]*src="(.*?)".*?><\/script>/g
// essenitials are all with "pages" or ending with "app.js"
const validScriptRegex = /(\/(legacy-)?.*?-(pages.*?|.*app|.*index).js|\/static\/.*(state).js)/

// eslint-disable-next-line require-await
module.exports = async function JSOptimizer(moduleOptions) {
  if (!moduleOptions.setOutputFilenames) {
    logger.error(
      'JS optimization works only when you explicitly opt in for overwriting output filenames in nuxt!'
    )
    return
  }

  if (!this.options.build) this.options.build = {}
  this.options.build.filenames = {
    ...this.options.build.filenames,
    app: ({ isModern, isDev }) =>
      `${!isModern ? 'legacy-' : ''}${
        !isDev ? '[contenthash]' : '[name]'
      }-app.js`,
    chunk: ({ isModern, isDev }) =>
      `${!isModern ? 'legacy-' : ''}${!isDev ? '[contenthash]-' : ''}[name].js`,
  }

  // eslint-disable-next-line require-await
  this.nuxt.hook('generate:page', async (page) => {
    let { html } = page

    const bodyString = bodyRegex.exec(html)

    if (!bodyString || !bodyString[0]) {
      logger.warn('no body tag found', html)
      return
    }

    const body = bodyString[0]
    const links = body.matchAll(scriptRegex)

    for (const match of links) {
      if (!validScriptRegex.test(match[1])) {
        // remove non essential JS
        html = html
          .replace(match[0], '') // script tag
          .replace(
            `<link rel="modulepreload" href="${match[1]}" as="script">`,
            ''
          ) // module preload
          .replace(`<link rel="preload" href="${match[1]}" as="script">`, '') // preload

        if (moduleOptions.debug) {
          logger.info('removed js tags for', match[1])
        }
      }
    }
    page.html = html // set new html
  })

  logger.success('JS optimization module initialised')
}

@Aliaaaam
Copy link

Managed to adapt @simllll workaround to work with nuxt full static mode.
You should add ['~/modules/js-optimizer.js', { setOutputFilenames: true }] to your buildModules and utilise the generate:page hook to edit the html. Also make sure component auto import is not messing with the chunks you want to lazy load - e.g. you'll have to explicitly ignore them from in the nuxt.config.js and import manually.

import consola from 'consola'

const logger = consola.withScope('js-optimization:module')

const bodyRegex = /<body[^>]*>(.*)<\/body>/s
// list of all JS includes
const scriptRegex = /<script[\w"= ]*src="(.*?)".*?><\/script>/g
// essenitials are all with "pages" or ending with "app.js"
const validScriptRegex = /(\/(legacy-)?.*?-(pages.*?|.*app|.*index).js|\/static\/.*(state).js)/

// eslint-disable-next-line require-await
module.exports = async function JSOptimizer(moduleOptions) {
  if (!moduleOptions.setOutputFilenames) {
    logger.error(
      'JS optimization works only when you explicitly opt in for overwriting output filenames in nuxt!'
    )
    return
  }

  if (!this.options.build) this.options.build = {}
  this.options.build.filenames = {
    ...this.options.build.filenames,
    app: ({ isModern, isDev }) =>
      `${!isModern ? 'legacy-' : ''}${
        !isDev ? '[contenthash]' : '[name]'
      }-app.js`,
    chunk: ({ isModern, isDev }) =>
      `${!isModern ? 'legacy-' : ''}${!isDev ? '[contenthash]-' : ''}[name].js`,
  }

  // eslint-disable-next-line require-await
  this.nuxt.hook('generate:page', async (page) => {
    let { html } = page

    const bodyString = bodyRegex.exec(html)

    if (!bodyString || !bodyString[0]) {
      logger.warn('no body tag found', html)
      return
    }

    const body = bodyString[0]
    const links = body.matchAll(scriptRegex)

    for (const match of links) {
      if (!validScriptRegex.test(match[1])) {
        // remove non essential JS
        html = html
          .replace(match[0], '') // script tag
          .replace(
            `<link rel="modulepreload" href="${match[1]}" as="script">`,
            ''
          ) // module preload
          .replace(`<link rel="preload" href="${match[1]}" as="script">`, '') // preload

        if (moduleOptions.debug) {
          logger.info('removed js tags for', match[1])
        }
      }
    }
    page.html = html // set new html
  })

  logger.success('JS optimization module initialised')
}

This will stop all other components js files, no mater on lazy-hydrate or not! how to exclude them?

@nezzard
Copy link

nezzard commented Sep 21, 2021

If you use nuxt, you can actually work around either by using modern mode (due to a bug) or by using a regex to remove the async deferred chunks from html ouput (render:route, see https://nuxtjs.org/api/internals-renderer#hooks)

example plugin:

import consola from 'consola';

const logger = consola.withScope('js-optimization:module');

const bodyRegex = /<body[^>]*>(.*)<\/body>/s;
// list of all JS includes
const scriptRegex = /<script[\w"= ]*src="(.*?)".*?><\/script>/g;
// essenitials are all with "pages" or ending with "app.js"
const validScriptRegex = /\/(legacy-)?.*?-(pages.*?|.*app).js/;

module.exports = async function JSOptimizer(moduleOptions) {
	if (!moduleOptions.setOutputFilenames) {
		logger.error(
			'JS optimization works only when you explicitly opt in for overwriting output filenames in nuxt!'
		);
		return;
	}

	if (!this.options.build) this.options.build = {};
	this.options.build.filenames = {
		...this.options.build.filenames,
		app: ({ isModern, isDev }) =>
			`${!isModern ? 'legacy-' : ''}${!isDev ? '[contenthash]' : '[name]'}-app.js`,
		chunk: ({ isModern, isDev }) =>
			`${!isModern ? 'legacy-' : ''}${!isDev ? '[contenthash]-' : ''}[name].js`
	};

	this.nuxt.hook('render:route', async (url, page, { req, res }) => {
		if (!page.html || (res.statusCode && res.statusCode !== 200) || page.redirected || page.error) {
			if (moduleOptions.debug) {
				logger.info(
					'skipping optimize JS render:route',
					JSON.stringify({
						url,
						isAmp: req.isAMP,
						matchedRoute: req.matchedRoute,
						page: page.html.length,
						statusCode: res.statusCode,
						error: page.error,
						redirected: page.redirected
					})
				);
			}
			return;
		}

		if (moduleOptions.debug) {
			logger.info(
				'optimize JS render:route',
				JSON.stringify({ url, isAmp: req.isAMP, matchedRoute: req.matchedRoute })
			);
		}

		if (!req.isAMP) {
			// remove all non-essential JS files

			let { html } = page;

			const bodyString = bodyRegex.exec(html);

			if (!bodyString || !bodyString[0]) {
				logger.warn('no body tag found', html);
				return;
			}
			const body = bodyString[0];

			const links = body.matchAll(scriptRegex);

			for (const match of links) {
				if (!validScriptRegex.test(match[1])) {
					// remove non essential JS
					html = html
						.replace(match[0], '') // script tag
						.replace(`<link rel="modulepreload" href="${match[1]}" as="script">`, '') // module preload
						.replace(`<link rel="preload" href="${match[1]}" as="script">`, ''); // preload

					if (moduleOptions.debug) {
						logger.info('removed js tags for', match[1]);
					}
				}
			}

			page.html = html; // set new response
		}
	});

	logger.success('JS optimization module initialised');
};

save it in modules/js-optimization.js and add ['modules/js-optimizer.js', { setOutputFilenames: true }] to the modules of nuxt.config.js

i'm getting
Cannot read property 'startsWith' of undefined

@RokeAlvo
Copy link

i'm getting Cannot read property 'startsWith' of undefined

check node version

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests