handleSitemapRequests will not work with next.js ^12.2.0

[priard]

The recently released Next version 12.2.0 introduces several changes, including stable middleware. Among other things, the middleware limited possibility of returning a response body, which we need for processed sitemap.

First error:

error - Nested Middleware is not allowed, found: pages/_middleware
Please move your code to a single file at /src/middleware instead.

Second error:

Your middleware function returns a response body, which is not supported.

More info about error: Returning response body in middleware
Middleware (Stable)

We need to find a solution for this, in the meantime I suggest to update package.json in the examples to limit the maximum version to 12.1.6.

"next": "<=12.1.6",

[blakewilson]

Hey @priard,

Thanks for reporting the issue!

Yes, with Next.js middleware now out of beta, they made some unexpected (imo) changes that effect how middleware work.

The first issue with nested middleware is easy to solve from the sitemap perspective. But the lack of support for returning a response body will require some additional thought from how we want to support sitemaps going forward.

We've created a ticket in our internal backlog (MERL-357) to address this.

In the meantime, I agree we should limit the version in the example project's package.json file. I'll create a PR for this.

[roeean]

Hey, is there anything new on the subject?

[blakewilson]

Hi @roeean, we are activately working on a solution for this in our current development sprint.

[rizwan92]

is there any update ? i am actively looking into this thread

[CesarBenavides777]

@rizwan92

is there any update ? i am actively looking into this thread

Doing something like this as a quick hacky fix:
sitemap.xml.ts in the pages directory with the following:

const Sitemap = () => {};

export const getServerSideProps = async ({ res }) => {
  const baseUrl = {
    development: "http://localhost:3000",
    production: "https://yourwebsite.org",
  }[process.env.NODE_ENV];

  const pages = await fetch(
    `${process.env.NEXT_PUBLIC_WORDPRESS_URL}/graphql`,
    {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        Accept: "application/json",
      },
      body: JSON.stringify({
        query: `
            query {
                posts {
                    edges {
                        node {
                            slug
                            date
                        }
                    }
                }
                pages {
                    edges {
                        node {
                            slug
                            date
                        }
                    }
                }
                events {
                    edges {
                        node {
                            slug
                            date
                        }
                    }
                }
                resources {
                    edges {
                        node {
                            slug
                            date
                        }
                    }
                }
            }
        `,
      }),
    }
  )
    .then((res) => res.json())
    .then((res) => {
      const posts = res.data.posts.edges.map((post) => {
        return {
          url: `${baseUrl}/news/${post.node.slug}/`,
          lastmod: post.node.date,
        };
      });
      const pages = res.data.pages.edges.map((page) => {
        return {
          url: `${baseUrl}/${page.node.slug}/`,
          lastmod: page.node.date,
        };
      });
      const events = res.data.events.edges.map((event) => {
        return {
          url: `${baseUrl}/events/${event.node.slug}/`,
          lastmod: event.node.date,
        };
      });
      const resources = res.data.resources.edges.map((resource) => {
        return {
          url: `${baseUrl}/resources/${resource.node.slug}/`,
          lastmod: resource.node.date,
        };
      });
      return { posts, pages, events, resources };
    })
    .catch((err) => {
      console.log(err);
      return [];
    });

  const sitemap = `<?xml version="1.0" encoding="UTF-8"?>
    <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
        ${pages["pages"]
          .map(({ url, lastmod: date }) => {
            if (url === `${baseUrl}/testing/`) {
              return null;
            }
            return `
            <url>
                <loc>${url}</loc>
                <lastmod>${date}</lastmod>
                <changefreq>daily</changefreq>
                <priority>1.0</priority>
            </url>
            `;
          })
          .join("")}
        ${pages["posts"]
          .map(({ url, lastmod: date }) => {
            return `
                <url>
                <loc>${url}</loc>
                <lastmod>${date}</lastmod>
                <changefreq>daily</changefreq>
                <priority>1.0</priority>
                </url>
            `;
          })
          .join("")}
        ${pages["events"]
          .map(({ url, lastmod: date }) => {
            return `
                <url>
                <loc>${url}</loc>
                <lastmod>${date}</lastmod>
                <changefreq>daily</changefreq>
                <priority>1.0</priority>
                </url>
            `;
          })
          .join("")}
        ${pages["resources"]
          .map(({ url, lastmod: date }) => {
            return `
                <url>
                <loc>${url}</loc>
                <lastmod>${date}</lastmod>
                <changefreq>daily</changefreq>
                <priority>1.0</priority>
                </url>
            `;
          })
          .join("")}
    </urlset>
  `;

  res.setHeader("Content-Type", "text/xml");
  res.write(sitemap);
  res.end();

  return {
    props: {},
  };
};

export default Sitemap;

[theodesp]

Hello @priard. We've recently added a helper that overcomes this issue with next.js. Unfortunately we haven't documented this yet but its easy to use it.

Here is how to enable this:

1. Create a sitemap.xml.ts in the pages directory:

2. Create a page that handles the sitemap request:

import {getSitemapProps} from '@faustjs/next/server';
export default function Page() {
  return null;
}

export const getServerSideProps = async (ctx) => {
  return getSitemapProps(ctx, {
    frontendUrl: 'http://localhost:3000', // or your public frontend URL
    wpUrl: process.env.NEXT_PUBLIC_WORDPRESS_URL,
    sitemapIndexPath: '/wp-sitemap.xml' // path in WordPress that handles sitemaps
  });
}

Visit your page at frontendUrl/sitemap.xml and you should be able to see the sitemaps.

I hope this helps.

[roeean]

Hi,

Unfortunately, it doesn't work well with multi-site.
it adds the prefix of the sub-site ("ora") to the path variable of the sub-sitemaps, which breaks the link (404):

If I try it without the prefix, it works:

How can it be solved?

Thank you,

Roee

[roeean]

Hi, it really works great, thanks a lot!
What about robots.txt? Is there a solution to this?