Pass fetched data from getStaticPaths to getStaticProps

[baukevanderlaan]

Edit by @leerob: see updated response here.

---

The SSG documentation recommends to fetch the data for the list of path params in getStaticPaths and fetch data for the individual pages in getStaticProps.

When using a GraphQL API for fetching data, this results in possibly redundant calls to the API: all the props needed for the individual pages can already be fetched in one single API call in getStaticPaths.

But as far as I know, there is no way to pass this extra data along to getStaticProps yet. If I add another key to paths, I get the error: Additional keys were returned from getStaticPaths in page "/posts/[id]". URL Parameters intended for this dynamic route must be nested under the params key

llustration

Current way (paraphrased from the docs)

export async function getStaticPaths() {
  const posts = await query(`
    query {
      posts {
        id
      }
    }
  `)
  const paths = posts.map(post => ({
    params: { id: post.id },
  }))
  return { paths, fallback: false }
}

export async function getStaticProps(context) {
  const post = await query(
    `query {
      post(id: $id) {
        title
        body
      }
    }`,
    { variables: { id: context.params.id }}
  )
  return {
    props: { post },
  }
}

Hypothetical way in which all data is fetched at once

export async function getStaticPaths() {
  const posts = await query(`
    query {
      posts {
        id
        title
        body
      }
    }
  `)
  const paths = posts.map(post => ({
    params: { id: post.id },
    // additional key for passing along data
    content: {
      ...post
    } 
  }))
  return { paths, fallback: false }
}

export async function getStaticProps(context) {
  return {
    props: { context.content },
  }
}

This would reduce boilerplate code and (potentially significant) numbers of API roundtrips when working with GraphQL, thereby reducing buildtime.

Is this already possible and if not, are there any plans of making this possible in the future?

Thank you for your time!

[ijjk]

Hi, this sounds related to #10933 and this comment specifically #10933 (comment) can help you achieve caching data needed multiple times in getStaticProps

[kunal-thinlimbs]

is it possible now

[irg1008]

How is it possible now? If you may

[chinanderm]

@kunal-thinlimbs How is it possible?

[opexchanger]

lol I think @kunal-thinlimbs was asking a question actually

[Noushad-web]

Thanks for this, but getStaticPaths should return some extra object in which we can send our other data to the getStaticProps.

[trevorwhealy]

I was hoping for the same thing. Seems that the only value I'm able to pass from getStaticPaths to getStaticProps is the id from a path's params.id. The "problem" in my case is that the framework functionally treats the ID as the URL, so it's difficult to perform an entity lookup in getStaticProps if the ID we're using is actually a readable string and not an ID that can be used for easy database querying. Ideally we could return additional values on the params object alongside the ID.

[ixartz]

@trevorwhealy describes perfectly a use case.
I am also very interested to pass data from getStaticPaths to getStaticProps.

Here is another example where someone asking the same thing: #10977

@ijjk The caching solution you show us definitively solve our issue. But sometimes, the cache is seems to me overkill. We just want to share some information between getStaticPaths and getStaticProps.

[simonzsolt89]

Hello, I am having the same issue/requirement. I want to pass additional data from getStaticPaths to getStaticProps. Nothing major, just an additional ID so that in getStaticProps I would be able fetch all of the data for the page with that ID. This would be the equivalent to Gatsby's 'context' object in their 'createPage' function.

[saaymeen]

The usecase @simonzsolt89 mentions is exactly what I am after too. Seems like this would be the ideal way for the dataflow.

[cameronk]

I'm encountering this issue as well.

I want to generate static pages with URLs that do not contain an ID. My CMS only has the ability to directly query an item by ID, not by the fields I'd like to slugify and put in the URL. The best solution would be to pass the ID from getStaticPaths to getStaticProps, as in this example for a path /[company]/[role]:

In this example, the id parameter is stripped from the params object automatically in the background. If I move id outside of params, the error @baukevanderlaan mentions occurs.

[dougwithseismic]

Shame to see this isn't a feature - It would save doubling up on API callswhere you're already able to fetch that data in getStaticPaths.

Anyone looking into it or am I volunteering?

[Nithur-M]

is this solution working?

[magicspon]

Is there a solution for this?

[zeevo]

Is this still active? I am encountering this issue as well. It would be great to be able to pass a little more context to getStaticPaths but it looks like attributes are getting stripped In between getStaticPaths and getStaticProps.

[tpperez]

👀

[tobiastornros]

Would be useful. I want to fetch more data in getstaticProps and I already know in getStaticPaths if I should do it or not so should be nice to pass that information to getstaticProps. Now I need to check the same thing two times for every page.

[dasblitz]

I'd really like to be able to 'overload' the params returned in getStaticPaths as well. In REST API's the lookup is usually done using an id like /product/123, but in the url of the client we'd use the slug: http://example.com/product/readable-seo-optimized-product-name. So in nextjs my file would be named /product/[slug].js but then I lack access to the product id in getStaticProps.

Edit:

After some reconsideration I think not having the id in the url would anyway be a problem when using fallback: true. Unless the API supports retrieving products by slug. Which would solve this problem altogether.

[kieryk123]

Same problem here... I need my URLs to be in a readable format so I use a slug, but then I don't have access to ID... Anyone could help?

[zeevo]

@kieryk123 A quick workaround for now is to write your Slug -> ID mappings to the filesystem in getStaticPaths. Then lookup the ID by Slug in getStaticProps

[b3driv3r-d3v]

Node.js fs module is enough to do this?

[zeevo]

Probably

[brendanmckenzie]

I am also facing this issue and the idea of storing the mapping on the filesystem doesn't sound very clean.

[spence-s]

Just chiming in to say was also fighting with this exact same issue and also hate the idea of using an fs cache as a work around. Just seems very hacky for such a seemingly simple problem.

[balazsorban44]

UPDATE: I was maybe getting ahead of myself, and was a bit too optimistic. Although the below project is a good starting point, it will not work. I made some very silly mistakes, to begin with, and I haven't considered some use cases. 🙈 Sorry for teasing anybody. If my free time allows, I may have a further look and actually finish it.

Hi folks, just wanted to let you know, I created a package that abstracts away the fs cache 🎉
(no real magic, it creates a .cache file in the project root. You probably want to add it to .gitignore):
https://www.npmjs.com/package/get-all-static-data

You use it like this:

import getAllStaticData from "get-all-static-data"

const pageData = getAllStaticData({
  async getData() {
    return [{ slug: "/testing/foo/bar", title: "Foo", updated: "2020-10-11" }]
  },
  pathMapper: element => element.slug,  
})

export const getStaticPaths = pageData.getStaticPaths(require("fs"))
export const getStaticProps = pageData.getStaticProps(require("fs"))

export default function Test({ data }) {
  return <pre>{JSON.stringify(data, null, 2)}</pre>
}

Please see the repo for docs 📚, and open an issue if you have any questions. Also, if you have any ideas, please let me know!
https://github.com/balazsorban44/get-all-static-data

PS: I could not require/import fs directly in my package because (if I understand it correctly) next will not cut it out of the client bundle, and will not find it, throwing a Module not found error. Suggestions for abstracting this away are also welcome!. Otherwise, the API could be simpler.

import getAllStaticData from "get-all-static-data"

export const {getStaticPaths, getStaticProps} = getAllStaticData({
  async getData() {
    return [{ slug: "/testing/foo/bar", title: "Foo", updated: "2020-10-11" }]
  },
  pathMapper: element => element.slug,  
})

export default function Test({ data }) {
  return <pre>{JSON.stringify(data, null, 2)}</pre>
}

DISCLAIMER: I put this together in an hour, haven't even tested outside of my dev environment, so 🐛 are inevitable. 🤓

[7ruth]

Thank you for putting this together, looking at your lib helped me understand a path forward. I used Notion example referenced in a comment above and slimmed it down make this work for a really narrow case of having custom url and an id passed in from getStaticPaths to getStaticProps.

Here is how I did it for anyone else looking for something like this (Get API data in the getStaticPaths, cache in fs, recover props in getStaticProps):

Lets say I have a page: ['customUrl'].js:

Process kicks off in getStaticPaths:

export async function getStaticPaths() {
    const paths = await function() {
        const targetUrl = 'https://someurl.com'
        const res = await fetch(targetUrl);
        const propsToCache = res.data.map((data) => {
            return {
                params: {
                  customUrl: res.data.customUrl,
                  id: res.data.customUrl.id,
                },
              };
        })
        // cache the props from api
        await createCustomUrlIdIndex(dataToReturn);        
        // return array of only [{params: {customUrl}}]
        return propToCache.map((paramsObj => {
            return {params: paramsObj.customUrl}
        }))
     }
  
    return {
      paths,
      fallback: false,
    };
  }

createCustomUrlIdIndex.js lives in /lib and looks like:

import { writeFile } from './fs-helpers';
import { URL_ID_CACHE } from './server-constants';

export default async function createCustomUrlIdIndex(urlIdCache) {
  const cacheFile = `${URL_ID_CACHE}`;
  writeFile(cacheFile, JSON.stringify(urlIdCache), 'utf8').catch(() => {});
}

fs-helpers looks like:

import fs from 'fs';
import { promisify } from 'util';

export const readFile = promisify(fs.readFile);
export const writeFile = promisify(fs.writeFile);

server-constants.js looks like

const path = require('path');

const URL_ID_CACHE = path.resolve('.url_id_cache');

module.exports = {
  URL_ID_CACHE,
};

This should save data in .url_id_cache at the root of the project (add it to .gitignore)

Now we need to get data back in getStaticProps:

export async function getStaticProps({ params }) {
  const pageUrl = params.customUrl;
  const customUrlIdIndex = await getCustomUrlIdIndex();

  const pageData = customUrlIdIndex[customUrlIdIndex.map((obj) => obj.params.urlAddress).indexOf(pageUrl)];
  
  // Do more stuff here ... return moreData

  return {
    props: {
      urlIdIndex: urlIdIndex ? urlIdIndex : [],
      fallback: false,
      pageData: moreData
    },
  };
}

getCustomUrlIdIndex looks like this:

import { readFile } from './fs-helpers';
import { URL_ID_CACHE } from './server-constants';

const cacheFile = `${URL_ID_CACHE}`;

export default async function getHomeListingsIndex() {
  let urlIdCache;

  try {
    urlIdCache = JSON.parse(await readFile(cacheFile, 'utf8'));
  } catch (_) {
    /* not fatal */
  }

  return urlIdCache;
}

and also added a next.config.js

const fs = require('fs');
const path = require('path');

try {
  fs.unlinkSync(path.resolve('.url_id_cache'));
} catch (_) {
  /* non fatal */
}

Hope that helps.

[cymkd]

This!
THIS!!!!!
Thank you brother. Notion example was way too convoluted for my old eyes.
This is very straight forward method, and I will abuse it for every single thing I can to lessen the amount of API calls going around.
It's really useful especially with the scenario where you have 2-3 3rd party APIs that you need to hit in order to get all the data per PAGE!!
And I have 5000+ pages here.

Thank you, beer's on me.

[xaun]

Same issue, I need to webscrape a page for header footer elements. I want this to be done at build time for pages but would like to avoid having to scrape the same elements 9000 times during build..... Any ideas?

[dougwithseismic]

Tbf since my original comment Ive come around to this pattern. Separating business logic and path generation feels right - especially when dealing with huge numbers of pages

…

On Sun, 3 Apr 2022, 10:54 Balsee, ***@***.***> wrote: Yeah, had a really similar problem today. It's crazy that it's still not a feature... — Reply to this email directly, view it on GitHub <#11272 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALSIKRDD7OE5WAXBTM3NTC3VDFMEVANCNFSM4LRGVXJA> . You are receiving this because you commented.Message ID: <vercel/next. ***@***.***>

[dopry]

I have a need for this. In our project, all the paths and pages are managed in the CMS. We have a single template on our front end with [[...path]].js in our root. The downside to our CMS is that we cannot easily query content by the path. So we would like to pass a page id along to getStaticProps as a part of the context. Ideally we could just extend the context and add a few additional properties.

[adammathieson]

I also would like to see this feature; my use case requires an additional partition key for fetching a single item but I can not currently pass the data.

[kaf-lamed-beyt]

I'm having an issue that is a little bit similar to this. I'm trying to fetch data from GraphQL API endpoint with the getStaticProps data-fetching method. Everything works fine in the function, even when I log the result from the API, I can see it in the terminal.

But, when I try to render the data on the page through the props, I get a reference error.

TypeError: Cannot read properties of undefined (reading 'map')

export default function BlogPage() {
  const result = articles;

  console.log(result);

  return (
    <React.Fragment>
      <Head>
        <title>Blog</title>
      </Head>
      <Container>
        <h3>Blog</h3>
        {articles.map((post) => {
          return <p>{post.title}</p>;
        })}
      </Container>
    </React.Fragment>
  );
}

export async function getStaticProps() {
  let articles = await getArticles();

  console.log(articles);

  return {
    props: {
      articles,
    },
  };
}

[bas080]

I'm having an issue that is a little bit similar to this. I'm trying to fetch data from GraphQL API endpoint with the getStaticProps data-fetching method. Everything works fine in the function, even when I log the result from the API, I can see it in the terminal.

That's just incorrect. You get the props as an argument in your render function. Use export default function BlogPage({props}) { ... }.

[kaf-lamed-beyt]

Oops! I guess that omission of the articles prop was a mistake.

In my code the prop is present, but I still get an error. Hold on, let me cross-check again

[kaf-lamed-beyt]

I did the same thing, I'm still getting undefined in the console. But, when I log the result to the console in the geStaticProps function the data is returned correctly.

[iamcrisb]

This usecase is so common that it has to be covered. I can't make the same request twice, it is really bad in terms of performance.

[marteloge]

Any updates on this? Or is there any plan to include more context data for static paths?

I have an API - a list of countries that will create the static paths - but the API also has information about each country needed for each static page.

I have the data when creating the static paths and would be neat to also pass extra context data instead of having to ping the same API in the static props again to fetch the same data.

[semy]

I also need this... I have to specify the type of the content: Is it a "post"? Is it a "page"? Is it a "category"? Because all my pages should be accessible under a simple slug without subdirectories. Depending on the content type I want to load a different component.

Is there any way to accomplish this?

[p0zi]

Have a look here: https://github.com/chapter-three/next-drupal/blob/main/examples/example-umami/pages/%5B...slug%5D.tsx

[AdamDiament]

Does the new data fetching paradigm in Next13 basically make the need for this obsolete?

[LeonSkrilec]

Yes, I believe so, since there are no getStaticPaths and getStaticProps methods in v13.

[dougwithseismic]

Well, there are. The new fetch paradigms are opt-in beta right now and are showing a few limitations so I wouldn't write `getStatic...` methods off just yet.

…

Message ID: ***@***.*** .com>

[timneutkens]

and are showing a few limitations

Can you provide examples? getStaticProps/getStaticPaths features are fully covered in app.

[Miosame]

Not sure why the fetch above is considered a solution, maybe I've been tracking the wrong issue all along? because my understanding is that generateStaticParams() would be the closest equivalent now in v13, which also misses the exact same props passing that e.g. gatsby allows in createPage, so it's the same issue, different dress.

[xgqfrms]

why below code not work at all ?

dynamic router path screenshot

// [title].js

export async function getStaticPaths() {
  const ids = await getPostsIds();
//   ids[0] =
//     {
//       id: '/Users/xgqfrms-mm/Documents/github/nextjs-ssr/posts/markdown.md',
//       year: '2022',
//       month: '12',
//       day: '31',
//       title: 'last'
//    }
  const paths = ids.map(({id, year, month, day, title}) => ({
    params: {
      // id can not pass `id` with params ❌
      id,
      year,
      month,
      day,
      title,
    },
    // can not pass `id` by using key out of `params` ❌❌
    id,
  }));
  return {
    paths,
    fallback: false,
  };
}


export async function getStaticProps({ params, id }) {
  console.log(`params without id =`, params);
  // params without id = {
  //   params: { year: '2022', month: '12', day: '31', title: 'last' },
  //   locales: undefined,
  //   locale: undefined,
  //   defaultLocale: undefined
  // }
  // both not work ❌
  // const data = await getPostsData(params.id);
  const data = await getPostsData(id);
  return {
    props: {
      postData: data,
    },
  };
}

hack way find id ✅

// [title].js

export async function getStaticPaths() {
  const ids = await getPostsIds();
  const paths = ids.map(({id, year, month, day, title}) => ({
    params: {
      year,
      month,
      day,
      title,
    },
  }));
  return {
    paths,
    fallback: false,
  };
}


export async function getStaticProps({ params }) {
  // hack way find id ✅ 
  const ids = await getPostsIds();
  const obj = ids.find(obj => {
    if(obj.year === params.year && obj.month === params.month && obj.day === params.day && obj.title === params.title) {
      return obj;
    }
  });
  const data = await getPostsData(obj.id);
  return {
    props: {
      postData: data,
    },
  };
}

[girdzijauskas]

Encountering the same issue here - simply need an extra freeform context key, e.g. internalFetchUrl.

No idea why there need to be limitations on the kind of data passed between functions? The feeling that I get is that Next.js is treating me a bit like a child and saying "Oh no no, you've built the object wrong, we don't allow you to be creative here. Please walk back into the safe area."

[leerob]

Hey folks, for those of you exploring the app directory (beta), you'll find that the original issue proposed here should be solved, as data fetching happens at the fetch level.

https://beta.nextjs.org/docs/data-fetching/fundamentals

[dougwithseismic]

Respect Lee but app directory and server components are too janky right now

[Miosame]

could you elaborate on how app directories would cover this example?

[leerob]

Check this out 😄 https://beta.nextjs.org/docs/data-fetching/caching

[dopry]

/app is still beta. A lot of developers won't be able to touch it...

[sloreti]

@leerob Can you please explain how the app directory solves this problem?

As I understand it, generateStaticParams takes the place of getStaticPaths in defining the paths that will be statically generated. In my case, I want to do something like:

// app/articles/[slug]/page.tsx

import Article from '../../models/article';

export async function generateStaticParams() {
  const articles = await Articles.list();

  return articles.map((article) => ({
    slug: 'human-friendly-slug-that-is-derived-from-article',
  }));
}

export default function Page({ params }) {
  const { slug } = params;
  // ...
}

Then, in the Page component, how can I access the article's id to query for that specific article? Are you suggesting we do this via caching?

[sumanchapai]

We solved this by saving the JSON that we needed from getStaticPaths in an intermediate file in the public directory. In getStaticProps, we just loaded the JSON from the same file. Worked fine when deployed to Vercel.

Imported the the node apis

import fs from 'fs'
import path from 'path'

In getStaticPaths, added the following

  const file = path.join(process.cwd(), 'public/mainSlugBuildData.json')
  fs.writeFileSync(file, JSON.stringify(allPaths), 'utf-8')

Here, allPaths stores more information than just the slug.

In getStaticProps, we added the following

  const fileToReadJSONFrom = path.join(
    process.cwd(),
    'public/mainSlugBuildData.json'
  )
  const dataFromGetStaticPaths = JSON.parse(
    fs.readFileSync(fileToReadJSONFrom, 'utf-8')
  ) as Array<{ params: { slug: Array<string>; routeId: string } }>

[leerob]

With the stable App Router in 13.4, data can be fetched, cached, and shared between your component (rendering) and generating static pages (generateStaticParams). This works either using fetch caching, or using cache().

• https://nextjs.org/docs/app/building-your-application/data-fetching
• https://nextjs.org/docs/app/building-your-application/data-fetching/fetching
• https://nextjs.org/docs/app/building-your-application/data-fetching/caching
• https://nextjs.org/docs/app/api-reference/functions/generate-static-params