Merge env and passthroughEvent from package turbo.json with root turbo.json

[weyert]

Which project is this feature idea for?

Turborepo

Describe the feature you'd like to request

I would like to have an option were the env and passthroughEnv defined in a turbo.json-file located in a workspace can be merged with the ones defined in the root turbo.json file.

Currently the root turbo.json is over 900 lines of code. Mostly consisting of defining the environment variables, the rest is the actual actions.

Describe the solution you'd like

I want to be able to define env and passthroughEnv variables on a per workspace level and combine it with a standard set of environment variables used by all projects, such as NODE_ENV, NAMESPACE etc.

Now you need to define all the environment variables in root turbo.json as anything defined in a workspace turbo.json will override the ones in root turbo.json.

This makes it difficult to maintain the environment variables for a mono repo that uses Turborepo with over in total 649 environment variables. All NEXT_PUBLIC_* used in Next.js application and feature flag environments get burdensome to main in the root turbo.json

Describe alternatives you've considered

Allow to opt-in behaviour to merge the configuration settings instead of replacing/overriden

[mehulkar]

We need to figure out a syntax for this. We've noodled on it before and the options are something like:

1. Option 1

   {
       "pipeline": {
           "taskname": {
               "some_array_config": [
                   "$inherit$",
                   "custom",
                   "keys"
               ]
           }
       }
   }

2. Option 2

   {
       "pipeline": {
           "taskname": {
               "some_array_config": {
                   "mergetype": "inherit",
                   "value": [
                       "custom",
                       "keys"
                   ]
               }
           }
       }
   }

3. Option 3

   {
       "pipeline": {
           "taskname": {
               "some_array_config:inherit": [
                   "custom",
                   "keys"
               ]
           }
       }
   }

We discussed deeply about merge behaviors when doing Workspace Configs and ultimately decided it would be confusing to do something automatically without a way to opt out, so we tabled it.

---

It's also possible that some v2 of Workspace Configs can allow merging with a template, but that's probably only going to solve the 80% case, and may not work for a repo like yours.

[weyert]

I think a flag that decides the behaviour would be sufficient? It's either merging with root config or the existing behaviour?

[mehulkar]

A flag is not sufficient because it cannot target the merge behavior of individual fields or even more granular, individual fields per task per workspace. We will likely be creating an unworkable situation if we add a global flag. We are open to an RFC that thinks through these things however: https://turbo.build/governance#roadmap-and-rfc-process

[tapico-weyert]

Why can't you merge the env and passthroughEnv from the global turbo.json for the overrides of turbo.json?

Does look like you are introducing a $TURBO_DEFAULT (see PR #7113) for the inputs so I can imagine a similar approach can use for these two config options? I think that could work. The use case for inputs is not a lot different. $TURBO_DEFAULT or what the name would be, would than reflect the values from the root turbo.json.

This solves the problem needing to re-add all the entries from the root turbo.json in the override turbo.json in the workspace.

[mehulkar]

@tapico-weyert yeah, we had resisted adding microsyntaxes for too long, but @tknickman managed to convince us of $TURBO_DEFAULT$.

$TURBO_DEFAULT [...] would than reflect the values from the root turbo.json.

It would definitely need a different token to inherit from root, since this one is about turbo's built-in behavior, not the user's config.

[weyert]

Yeah, that seems reasonable. I am not too fussed about naming :) I think earlier a $inherit$ was suggested. That works for me.