Full static generated mode

[manniL]

Current state

Nuxt's Static Site Generator (nuxt generate) is growing! I love the static mode when it comes to portfolio pages (or generally, pages that don't include a lot of dynamic data).

Problem

Usually, you use statically generated pages together with a Headless CMS or another external API.

Currently, you can generate the HTML (with static + universal mode) but asyncData calls are still made during client-side navigation, which means that an external API will likely be called on such a static page.

While it's worth here and there to make these calls even after static generation, it's absolutely not needed (from my POV) for the majority of the pages. Instead, the author/developer could simply issue another build (eg. through Netlify) to update the content.

Also you might encounter different content as asyncData is not called on the entry route of you static app. (Going from /b to /a can lead to different content than directly accessing /a)

Proposal

As announced by @Atinux on Vue Toronto (see his talk at 26:18), I want to propose the full static generation option for nuxt generate.

Instead of relying on the API calls, we should inline the response of the asyncData method(s) inside a .json file (per page) when the generate.fullStatic flag is enabled in the nuxt.config.js (the flag name is debatable). Then we can use the .json file as data source without issues.

Related issues

nuxt/nuxt#4607

3rd party modules that apply this approach

https://github.com/DreaMinder/nuxt-payload-extractor
https://github.com/stursby/nuxt-static/

[YamenSharaf]

As someone who's working to create a fully static site, this has been an issue for me as it looks like there's no way currently to have a fully static site with a headless CMS.
Nuxt has a lot of tools that help with that, but not quite in a way that ties them together to create something as fast as a static site should be.

These are the options I've stumbled across so far:

Relying on asyncData in page components

Pros

• This is the natural /encouraged approach by Nuxt docs
• Each page pulls in its own static content. Refreshing the page yeilds static content without extra AJAX requests.

Cons

• When navigating through nuxt-link to pages with async-data function defined, an AJAX call is fired before each page navigation to, again, load the data that is already statically on the page in a needless fashion. I'm no expert, but I guess that entails also shipping the vue engine to bind data coming from the server (also needlessly).

Relying on payload

Pros

• Nothing I can think of really, since the earlier approach seemed to work fine for pulling data for individual pages.

Cons

• More complex to implement.
• When navigating through nuxt-link, payload property is always undefined, which means I would have to have an asyncData method that would fire anyway on each page navigation.

Utilizing the vuex store and nuxtServerInit to get needed data

Pros

• No AJAX call is triggered when pulling data from the store (seems fair), even when navigating with nuxt-link.

Cons

• Store is basically global and not confined, which means that pulling data upfront. eg, posts, will cause them to load on each page even if not needed, but you do get faster navigation afterward.

Conclusion

I would have had rather be a straightforward way to pull data from an API without the overhead of extra AJAX requests or loading unneeded data.

[pi0]

Nice writeup @manniL.

For more specific terminology let's call this behavior/mode static and the offline data source files cache data.

Adding some more important notes:

• We should consider we have fetch, asyncData, nuxtServerInit and SSR plugins that all should be addressed also can cause side effects like triggering store dispatches that they can also fetch/process data.
• Cache items have no one-to-one relation with pages or route entries. Wildcard routes and query parameters require different responses. (edited: this is a rare case as we generate one HTML per each)

Regarding implementation and future opportunities I can think of 3 ways:

1. Memorize asyncData

This way, during generating we can spy result of asyncData during SSR generate and store it somewhere like .nuxt/dist/cache. Then by modification of asyncData implementation for client-side instead of the original method we import corresponding cache item. This also addresses the need for duplicate HTTP requests.

Pros

Most straight-forward approach.

Cons

We can't guarantee everything is cached. fetch and store may have their own logic.

2. Cache window.__NUXT__

This way, during generate we cache nuxt states into .json files and like method (1), change the behavior of client-side for fetch and asyncData

Pros

Straight-forward and covers store state too.

Cons

• The possibility of breaking change for exciting projects.
• How to handle non pojos?

3. Cache network requests

This way was the one that me and @Atinux talking about it almost one year ago for static mode. Instead of internal nuxt hacks, we can find a way to spy on network requests and statically cache them somewhere like .nuxt/dist/cache/[hash].json and instead of doing actual requests we can import json chunks.

Cons

• Spy part is delegated to official nuxt modules. (However, if changes required to the core, we make them)

Pros

• Updating API cache is independent of nuxt generate. We can update it without rebuilds
• We have full control over each ajax/http call to be cached. Not caching a black-box
• No internal hacks and breaking changes
• Transparent caching means making it optional without the need to changes of fetch/asyncData and store actions for offline support
• More service-worker spec friendly for caching and PWA like approach.
• Works with other modes (@manniL)

[pi0]

After talking with @Atinux, both (2) and (3) options seem good approaches. Most of the static generated websites do not need complex logic and simply storing NUXT object will be enough but also network level caching has it's own benefits too.

The suggested approach is introducing a new $cache API to the nuxt core which is responsible for storing and loading cache entries. This API may be used directly by users or module like axios. The cache layer may leverage a cache adapter too.

I do believe we can include a really basic default implementation to the core. Adapters should be probably different for server and client (cache.client.js and cache.server.js. Like nuxt/nuxt#4574 which made by Sebastien for nuxt-link)

For page level caching during generating full static version, we can add a cache property to the pages or global config (Like transition) that can enable/customize page level caching and probably globally enabled when using nuxt generate --full-static. We can then store ssrContext.nuxt.data to a JSON file that will be loaded instead of calling asyncData and fetch on the client side. This should be also clearly documented that those functions are no longer being called on full static mode.

For changing client-side behavior we can use lodash template to replace default asyncData/fetch calls with $cache.get($route) calls (vue-app/template/client.js)
For client-side support we can add logics to read cache option from page or global and use $cache.get($route) instead of calling functions. (vue-app/template/client.js)

[Atinux]

I strongly validate @pi0 comment.

This caching system could be used by other modules and inside services too in the future.

Nuxt is build on modules, we simply need to have @nuxt/cache now.

The thing I want to work on is to give more possibilities to modules to customize the Nuxt logic of client-side. This could be done by extending defaults components, or having some hooks system. So Nuxt plugins could also interact more the client & server logic of the default Nuxt logic (middleware, async data, transitions, store, etc)

[manniL]

One aspect we shouldn't forget: Making a page "full static" by actually removing the API calls from the code also means that the underlying API is harder to attack (via DDoS, exploits, ...) as a possible attacker doesn't know where the target API lives.

Of course, Security through Obscurity is no sophisticated security concept at all.

[pimlie]

Could we take this rfc maybe a step further and also include an option to build an all inclusive, single (static) html file? There are times this would be useful, e.g. when you create a demo of something so you can just email a single html file to a client or upload multiple demos to your website without the need for subfolders etc. But also an Electron application could benefit from this when all html/js/cs is loaded at application start-up, or a better example if you want to put a single html file on a usb-stick to hand-out as a promotion.

E.g. in the past I've used a grunt script to achieve this (albeit for a non-Nuxt project). As combining all the css/js/font files resulted in a single 1.3MB html file, I embedded all files zipped & then base64 encoded which reduced the file size to 550KB. (Of course the unzip javascript code and the js for the loading dialog was not zipped).

[pi0]

@pimlie Nice idea. We may use MHTML format to encapsulate all resources of each page. And it is easier to compress.

[pimlie]

Although mhtml would in theory be better, it lacks the full support which plain html file has (eg Firefox doesnt support it by default). But if all the hooks are in place, we could choose to create modules to generate both mhtml as html.

[Atinux]

Indeed, this could be made with the help of a module @pimlie, this module could:

• Overwrite all webpack file/url loader to force include as data64
• Find any url that point to a static file and include it's base64
• Remove code-splitting to have only one huge JS file?

Also, with 2.4, modules can create sub commands 🚀

[DreaMinder]

I've made a module for this purpose, didn't know that a simillar one already exists.
Its very simple, but requires custom code in asyncData like payload mechanism does.
Repo: https://github.com/DreaMinder/nuxt-payload-extractor
Working demo: https://dreaminder.pro/ru/blog/nuxt-vs-vue-spa-battle (just a sketch of my blog)
Maybe you'll find usefull some ideas I used.

[manniL]

Another problem that comes to my mind is queryString. But something like /articles?page=2 won't work properly when you have no server behind it.

We should still think about that

[DirkStevens]

Wonderful thoughts!

We're using Nuxt as the front-end for data from a Headless Drupal. We'd love to be able to render a static site.

These are some of the challenges we face with static rendering:

We've got >40,000 pages managed and maintained by a team of 25+ writers

• Most pages are updated regularly several times a year (think: 25 * 40,000 = 1,000,000 updates/year)
• Pages are added on a daily basis
• Some pages are removed

How and when are re-renders performed of added/changed pages? How do you know about new pages? Removed pages?

How can we let Nuxt know what data changed?

Routes are managed in the CMS by the writers

• This is very different from common router scenarios where the routes are known upfront and can be defined beforehand.
• Pages can be moved from one route to another
• This requires a re-generate of several pages
• Routes/slugs can be renamed - this touches all pages that link to that page also
• Our routes can not be defined with patterns at all...

How can we let Nuxt know a route changed?

Search pages

• We've got search pages with fuzzy and facet bases search that require server side calls
• We've experimented with browser bases search on full-data fetches and have excellent results in the UX but problem is to get the data to the client and keep the data current

Some pages combine data from multiple sources

• Page A uses DataA
• Page B uses DataB
• Page C uses parts of DataA and DataB

Ideally the generate takes re-use of data into account.
If DataA changes, Page A and C need to be regenerated. If DataB changes Page B & C need to be regenerated.

We're not familiar with the deep insides of Nuxt but would love to help.

Cheers!

[DreaMinder]

@DirkStevens Wow, sounds like worst case scenario for static-generated project =)
I'm not nuxt contributor, but I think complicated invalidation logic varies with every project and can't be included in framework core.
If you want speed up your project, I'd suggest to go for nginx-cache or varnish. There are ways to communicate with between nuxt and proxy-cache to invalidate updated content, it would be much better reliable solution.

Just imagine that you changed navigation item in your layout and you now have to generate 40k pages, because every page contains navigation.

[pimlie]

@DirkStevens You'd probably should read this question: nuxt/nuxt#2370 and the nuxt generate docs in general.

[DirkStevens]

@DreaMinder @pimlie Thanks for your thoughts.

True, invalidation logic should happen in the project and not in Nuxt. For large, high-impact production static sites it would be good if Nuxt exposes events that let us trigger partial re-generate of items that we mark as invalid.

Right now we use Akamai CDN. @DreaMinder Could you point me to information about proxy-cache invalidation with Nuxt?

[DreaMinder]

@DirkStevens sorry, no. Maybe I'll make an article about it, it is complicated. Shortly, nginx-proxy-cache (or cloudflare) respond with cache by default but checks in background if content is updated with limited interval. I forgot how this strategy is called...

[sebtiz13]

I'm just starting on nuxt, but for my blog i have create an modules to download the list of my articles and I do my treatment in hook "build:before" for save that in json file.

After in pages I import this json file and return the part of data which interest me in asyncData.
Currently I have only 5 articles on my blog so I do not know the impact on the performance of generation and its most a proof of concept for the introduction to the static site for me.

But i think its not bad because the file is transformed in js file with hash by nuxt, so the browser can keep it in cache.

I know this solution its not adapted for big site, but I think we can create chunked file with the specific data for each pages.

Finally Its true that it would be nice if nuxt managed the creation of these files through asyncData and this function its only called on server side.

[yann-yinn]

My two cents

Nuxt is actually an awesome Static Site Generator and i built several sites it as a static generator.

My guess is that a solution like Gridsome will probably soon get most of the hype concerning JAMStack with Vue because :

• It is focused on JAMStack only, so it will probably move much faster in this direction
• It will offer, out of the box, some plugins to pull data to generate pages (yaml, wordpress, contentful etc), so it will probably work faster for beginners
• There is a GraphQL layer to get the data, people seems to love the idea and tha's part of what made Gatsby so successfull (mixing the good words together : JAMStack + React + GraphQL, bingo ^^').

I would say the most important things for the users is to have plugins to fetch data from sources in a easy way, out of the box.

For example : today, how do you fetch data from markdown files to generate html files ? Which is the best way to create a "Hugo Like" or "Jekyll Like" static site ? This is not something that is easy to find. I began to code my own package for that, nuxtent was a solution but seems not to be maintained anymore.

The asyncData / JSON files issue is more a technical issue than a way to vastly improve the Developper experience when generating static pages with Nuxt.

[manniL]

@yann-yinn Thanks for the feedback ☺️

A few words on your post:

It [Gridsome] is focused on JAMStack only, so it will probably move much faster in this direction

Could be possible, but doesn't have to be. We are also concerned with JAMstack as you can see 😋

It will offer, out of the box, some plugins to pull data to generate pages (yaml, wordpress, contentful etc), so it will probably work faster for beginners

True, that's something we can start with / focus more on. Easier integration with Headless CMSs. There are some nice initiative (e.g. this module or this module) out there.

Also, more tutorials on that topic would be great, I agree.

The asyncData / JSON files issue is more a technical issue than a way to vastly improve the Developer experience when generating static pages with Nuxt.

I don't agree with that. A "full static mode" would increase the performance of the page and would lead to less mismatching content (as explained in the intro post).

[yann-yinn]

I don't agree with that. A "full static mode" would increase the performance of the page and would lead to less mismatching content (as explained in the intro post).

I agree a "full static" generation without additional asyncData call on client Side would be cool and also help to have more "atomic deploys" , which is part of JAMStack philosophy. My concerns about source plugins and comparison with Gatsby-like static site generators should be in another issue than this one.

[cesasol]

I do think that the static or event the ssr mode could be improved, this article points that website made with techniques like hydrating are actually bad but could be improved with partial hydrating. In my case I often find myself looking at big page components that only render static html but takes twice the time to get to an interactive state (hydrating) for the huge code that is makes, so far I managed two options, one is to use a functional per view component that renders only static html, the second (currently named nuxtent) is to have the html or markdown in another file and use it as an api.

[gridsystem]

@manniL @Atinux I just want to say, if you have a branch you can push which the community can finish off, I think you have at least 18 people who could potentially take it the rest of the way to a PR.

I started extracting the module and I am actually using it for the documentation: https://github.com/nuxt/nuxtjs.org/blob/master/modules/static/index.js

I plan to:
…
Make a PR and hopefully land it for v2.9.0 😄

I personally am a little frustrated when this happens in an open source project. Nuxt is clearly open to input from the community, but a core developer has taken the lead on a feature without sharing their code (or progress). Other contributors could have proposed solutions and raised PRs in this time, but have not because a core dev has had no transparency with their working.

I don't think anyone here is unhappy that you're taking a while with this, or even if you've decided not to pursue it. If you say you're not going to, then we can all embrace third party modules or custom code for this feature and move on, confident that the code won't be deprecated by a suddenly released core feature. Or use Gridsome, which seems to have embraced this concept.

If you're still RFCing, I think this raises an important topic - it might be useful to look at how this could have been handled differently. Are there any processes which could have been put in place to allow for the community to finish this code? Is there a platform or a GitHub feature for maintaining transparency with feature progress which could be utilised? Is there a way for the community to vote on and prioritise features which the core team wishes to plan and implement themselves?

[Atinux]

Hi @gridsystem

I am still working on it, it took me more time than planned (very busy lately) and avoiding breaking changes is pretty tough, the PR implementing it is here: nuxt/nuxt#6159

[gridsystem]

This is fantastic. I think the solution to my final paragraph is only to link PRs to RFCs!

If I put my PM hat on, good next steps (if you have time) would be to

• Braindump your notes as a comment on the PR
• Add quick summary of progress as comment to PR
• Add quick overview of why tests are failing as a comment to the PR
• Update your PR description (add any checklist items and check off complete items)

And then throw it into the wild for the community to help get to the finish line? This is such a great feature! Thanks so much for working on it.

[Andras1000]

In case this helps anyone, I've had a Vue.js app (without Nuxt.js) and used the following solution.

• There is a Node.js script that calls the API (e.g. a headless CMS) and saves the responses to a JSON file.
• The app code is slightly modified to look for a response in this JSON as an alternative to making an API call.
• The app's CircleCI build job runs the script that generates the JSON and this file is included in the static bundle that's deployed. Our goal of getting rid of unnecessary API calls is achieved.
• The headless CMS is set to notify a webhook when changes are made and the webhook triggers a re-build and re-deploy. The static bundle remains up-to-date without manual intervention.

[bdadam]

I think this issue would still be very important to solve. I miss the possibility of "fully static" generated code. I just wanted to make sure that everyone has seen what the main competitor (Next.js) is doing now.

In Next.js from components we can now export two new async functions: getStaticProps and getStaticPaths. These functions are only called during static generation/on the server side. Please check them out, maybe it would be a viable way to go for Nuxt as well.

[dswmedia]

I'd like to see this implemented too. Fully static generated first response would help with speed

[locopati]

I am looking for a solution to this for an app that uses vue-apollo to pull data from Strapi CMS. The app is deployed as a static site. What I'm noticing is that the first request to display the page fails but the a reload works because the data is in the page.

My temporary ugly solution is to force that reload in an error handler...

error: function(error, vm) {
          let loc = vm?.$el?.ownerDocument?.location;
          if(loc?.hostname !== "localhost") {
            loc?.reload();
          }
        }

However, I also noticed that if I simply remove id="__nuxt" from the main container div, the page just renders with all the data the generate placed in the page instead of trying to use vue-apollo to setup the data, failing, and rendering no data. I suspect if I understand better how these components interact and the order of events, I could see how it would be possible to use vue-apollo locally to populate the page and use nuxt/vue-clientside to load the page in a way that ignores the dynamism and just uses the data already in the page.

Is it possible via some hook to not emit that id in the generated pages and thus populate the page while allowing it to render statically? Or some other approach I'm not seeing (that doesn't involve storing json locally)? If anyone has any ideas, I'd love to hear them - I don't know enough about nuxt/vue/apollo to understand exactly how to accomplish what I want.

[kfrederix]

Not sure if this is the right place to ask, but I was wondering. What about images (or assets in general) which are stored in a headless CMS? I think it would be interesting if we had an option to automatically dowload and store files which are referred to from the output of nuxt generate. So that not only api calls, but also targetted files would become fully static and we would really eliminate all dependencies to the CMS once the site is generated.
Any thoughts on this? Or is this already possible maybe with another module I'm not aware of?

[robertpiosik]

@kfrederix from my experience, with such ideas as yours its good to think about the problem in scale. Imagine you're doing what you've explained having hundreads or thousands of pages. You would basically end up moving a lot of data from one place to another over and over again - and that data wouldn't change from deploy to deploy what obviously makes this approach rather unfortunate.

[mornir]

I think it can make sense in some situations. There's a plugin like that for Gatsby with the headless CMS Sanity: https://www.gatsbyjs.org/packages/gatsby-source-sanity-transform-images/

[emiliobondioli]

@kfrederix I am experimenting with nuxt static generation to allow something like what you describe, a static website generated from an headless CMS without the need of API calls or remote resources from the cms.
To save the cms assets (only images right now) I've written a small build-only module which runs on the generate:page hook to scan the page's html through a regex, download and store all image assets coming from the cms in the dist folder and replace the remote paths with local ones.

I'm not sure it's the correct way to do things in nuxt and as @robertpiosik said it's not really a scalable approach but for a small website without frequent updates it seems to be working fine! To reduce moving data one could avoid downloading some images if these are already present locally.

[kfrederix]

@robertpiosik you are certainly right, but that's why I mentioned "having an option". I guess in some situations this won't make sense while in other situations it can make a lot of sense... I'm definitely not advocating to have this behaviour by default but I think this could be a very useful option for certain situations.
Anyway, this might be out of scope for the full static feature...
@emiliobondioli would you mind sharing your build-only module?

[robertpiosik]

@emiliobondioli is your work available somewhere? I would like to look into it.

@kfrederix The more I think about this idea, the more I like it! Completely independent static app build is a key point here. Very useful for pull request previews, where a CMS, the app is build upon runs locally (and is fulfilled by locally managed dummy assets). Even for proper production deployments I can see now this idea being fully reasonable.

[emiliobondioli]

Sure! since the repo where I'm using this is private, I've put it in a gist:
https://gist.github.com/emiliobondioli/5ce8ece783e7256fc7530738a2968ea9

It's still quite raw and I'd like to make it work better together with nuxt, but as a proof of concept it seems to be working :)

[Atinux]

Thank you for your comments and idea.
I do believe this is related to a component support the full static mode to export external resources as well :)

[jmheretik]

hey @robertpiosik. I also attempted to do something similar if you want to look into it. i scrape all the html of pages in my routes to look for img tags, download them, and modify the img srcs to point to the locally downloaded copies

https://github.com/jmheretik/kirby-vue-starterkit/blob/master/vue-nuxt/modules/kirby-scraper.js#L23

and here is the generated demo hosted in gh-pages to confirm it works :) https://jmheretik.github.io/kirby-vue-starterkit/

[Atinux]

Thank you @jmheretik, we are doing a system with @pi0 to make it easier to have a static assets folder. BTW the demo is beautiful!

[manniL]

Full static mode landed on nuxt-edge.

nuxt/nuxt#6159

[noname202]

Have you thought about implementing more granular data fetching simmilar to what the Next.js project recently intorduced with the getStaticProps and getServerSideProbs functions. In my opinion it would be very benificial to allow mixing static and dynamic pages. Furthermore I really like the idea of being able to only fetch data on the server side as this would allow for a whole range of new use cases such as the use of internal headless CMS systems not accessible to the public.