Turborepo support (Currently INIT_CWD workaround required)

[0xdhrv]

When using contentlayer inside turbo repo, the config gives an error for importing contentlayer/source-files

This is the same repository for reproduction of the same: https://github.com/0xdhrv/turborepo-with-contentlayer

And, inside apps/starter

[schickling]

Hi @0xdhrv. Thanks for opening this issue and providing a repro. I've just given it a try and can confirm the problem you're running into. This is currently happening due to a limitation in Turborepo itself (see vercel/turborepo#485).

However, until this is resolved you can workaround the problem by simply adding INIT_CWD=$PWD in front of next build in your package.json like the following:

// apps/starter/package.json
  "scripts": {
     // ...
    "build": "INIT_CWD=$PWD next build",
    // ...
  },

Related:

• [Solved] NoConfigFoundError vercel/community#443

[pomber]

I'm having the same issue with lerna. The INIT_CWD=$PWD trick works, but I'd prefer to avoid it.

[jrolfs]

Yeah, same here but with Yarn workspaces... discovered the INIT_CWD thing myself before finding this issue. What's the intention behind reading from INIT_CWD when resolving the working directory?

contentlayer/packages/@contentlayer/core/src/cwd.ts

Lines 9 to 13 in 20ce905

	T.succeedWith(() => {
	// `process.env.INIT_CWD` is set by `yarn` or `npm` during installation
	const cwdValue = process.env.INIT_CWD ?? process.cwd()
	return unknownToPosixFilePath(cwdValue)
	}),

[schickling]

Some background / motivation on this subject:

Contentlayer is trying to aiming to work as smoothly "out of the box" as possible. One aspect of this is the auto-discovery functionality in regards to the Contentlayer config file and where to generate the .contentlayer folder.

Most users of Contentlayer are working in a single-project repository where the Contentlayer config and generated folder are "living" in the root of the repo. However, in more complex setups (e.g. in a mono-repo) your website project (using Contentlayer) might live in a sub directory. Depending on how you're interacting (e.g. via Turbo, Yarn, ...) with your tools (e.g. Next.js, Contentlayer etc) process.cwd() might resolve to different places.

In these more complex cases Contentlayer needs to know which subdirectory you're targeting. The most commonly used "solution"/convention for this in the JS ecosystem is for mono-repo tools to set the INIT_CWD environment variable to a given subdirectory. Contentlayer is then able to pick up the intended subdirectory based on the INIT_CWD env var.

Unfortunately it seems that some tools (e.g. Turbo, Lerna) currently don't support this pattern.

---

Alternative approach:

@jrolfs @pomber would you rather prefer explicitly providing a Contentlayer config path when you're calling the Contentlayer CLI or e.g. provide it in your next.config.js?

[jrolfs]

@schickling I think ultimately it makes sense to be available in both places, right? That said, I think I would prefer a CLI argument as I have a build step on CI that invokes contentlayer build directly.

I could probably take a stab at this, although admittedly there's a bit of a learning curve with this codebase on account of Effect-TS (and its lack of documentation 😜).

---

Here's a bunch of detail around my specific case, just so you have it when considering all the idiosyncrasies of package managers 😅

I am using Yarn 1's workspace command and it seems the issue is that it does set INIT_CWD to the root of the repository in this specific case (even though process.cwd() resolves to the package (workspace) in which the package script is being invoked.

Example

Configuration lives @ ~/my-monorepo/packages/site/contentlayer.config.ts

~/my-monorepo/package.json

{
  "name": "my-monorepo",
  "scripts": {
    "start:site": "yarn workspace @foo/site run start",
    "build:site": "yarn workspace @foo/site run build"
  }
}

~/my-monorepo/packages/site/package.json

{
  "name": "@foo/site",
  "scripts": {
    "start": "next dev",
    "build:site": "contentlayer build"
  }
}

From ~/my-monorepo

yarn start:site

and

yarn build:site

...both fail to find the configuration because INIT_CWD is set to ~/my-monorepo (not ~/my-monorepo/packages/site, which is what process.cwd()` returns).

However — if I invoke yarn workspace @foo/site run build directly from ~/my-monorepo, INIT_CWD is set correctly to ~/my-monorepo/packages/site, and therefore the configuration is resolved correctly.

I'm hoping to get onto Yarn 2+ soon in this project, and I'm curious if it has different behavior for INIT_CWD in these cases. I'm not sure what the "standard" behavior is supposed to be for INIT_CWD, though. Yarn 1 does seem to always set process.cwd() correctly when scripts are invoked on sub-packages.

---

Finally, I just want to say nice work so far on contentlayer. I really like the approach y'all are taking and I'm so glad I found it :)

[schickling]

I think I would prefer a CLI argument as I have a build step on CI that invokes contentlayer build directly.

Did you try running contentlayer build --config ./some-package/contentlayer.config.ts (see docs)

[pomber]

In my use case, the contentlayer next.js app is in an examples folder together with other example apps.

I have it there for three reasons:

1. so other people can copy the example and run it outside of the monorepo
2. to have a Stackblitz demo linked to GitHub (https://stackblitz.com/github/code-hike/codehike/tree/main/examples/contentlayer)
3. to run the build step on CI every time I change something in Code Hike

So ideally for me, the build should run the same way if you run it as part of a monorepo or standalone, without adding any extra configuration inside the examples folder.

[jrolfs]

Huh, I guess I missed that CLI flag as I was originally looking for the Next.js case... unfortunately it seems to have issues as well (and from playing around it seems like the path provided is relative to INIT_CWD which kinda brings along the same set of problems?)

Maybe I'm missing something or oversimplifying things but I'm pretty sure most tools just use process.cwd() and "just work" (lots of assumptions there as obviously I haven't looked at how everything works).

[schickling]

@jrolfs would you please mind opening a new issue for this with a reproduction repo? (Ideally also including some snippets how you would like it to work vs how it's currently working.)

(PS: In the upcoming version you'll be able to specify the config path in the Next.js plugin - see #248)

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[schickling]

This should be addressed with the 0.2.8 release. 🎉

[Aaditya-23]

I am getting this error at build step. I am using next13 and turborepo. However the build is successful when i run build from the specific app's json file. Eg. build from root does not work, but cd apps/starter build does work

Edit:

I just noticed that my contentlayer config can not be located even in dev mode.

I get this message on running the app in dev mode.
Could not find contentlayer.config.ts or contentlayer.config.js in C:/Users/ffd/OneDrive/Documents/GitHub/turbo-repo

This means it is looking for my file in the root of my repo.

And unfortunately i cannot use INIT_CWD because I get this error

INIT_CWD' is not recognized as an internal or external command

[rtorcato]

I removed this line from my next.config and my turbo repo setup now works for me.

const { withContentlayer } = require('next-contentlayer')

I am using tsup to build my package, but the error was still appearing in my turbo build.

is there any reason I would need withContentLayer in a mono repo? when I build my markdown package I am getting refreshed content on my nextjs app and everything looks ok.