Config.toml env variable replacement fails

[alex-ketch]

Bug report

Describe the bug

When trying to run supabase start/stop using CLI versions 0.21.0 or above, an error is shown if an env variable is used in the config.toml.
This used to work with 0.18.2.

❯ source .env && supabase stop
Error: Error evaluating "env(GITHUB_CLIENT_ID)": environment variable GITHUB_CLIENT_ID is unset.
❯ echo $GITHUB_CLIENT_ID
some-client-id

Config

# other sections omitted
[auth.external.github]
enabled = true
client_id = "env(GITHUB_CLIENT_ID)"
secret = "env(GITHUB_SECRET)"

Let me know if more details are necessary.
Thanks!

To Reproduce

1. Add the above config snippet to a Supabase project
2. Create a .env file with GITHUB_CLIENT_ID=foo as contents
3. run source .env && supabase start
4. See error

Expected behavior

The env variables should be interpolated into the config.toml.
Ideally:

• misconfigurations in the TOML file shouldn't prevent stopping the local Docker containers
• env variable substitution should work throughout the config, and not just for auth providers

System information

• OS: macOS 12.3
• Version of supabase-cli: v0.22.4
• Version of Node.js: v16.13.0

[soedirgo]

Thanks for the report! Will try to reproduce this.

[soedirgo]

Does it work if you do it like GITHUB_CLIENT_ID=a GITHUB_SECRET=b supabase stop?

[alex-ketch]

Does it work if you do it like GITHUB_CLIENT_ID=a GITHUB_SECRET=b supabase stop?

Yes, that works.

[soedirgo]

I think you might need to use export GITHUB_CLIENT_ID=a etc. in your .env - source runs the script in a subshell so the envars are not passed to the supabase command.

I'm not sure how/why it used to work though.

Edit: setting the variable only makes it available in the current shell, not to subcommands - echo worked because the substitution happens in the current shell. So you'd need to export it.

[soedirgo]

As for your 2 points above:

misconfigurations in the TOML file shouldn't prevent stopping the local Docker containers

Agree, I should fix this.

env variable substitution should work throughout the config, and not just for auth providers

I'd like to keep it as is for now - the env() interpolation was really for things that shouldn't be committed to source control. I do think it's awkward that we only do it for external provider client IDs and secrets - might think up a better alternative at some point.

[alex-ketch]

I think you might need to use export GITHUB_CLIENT_ID=a etc. in your .env - source runs the script in a subshell so the envars are not passed to the supabase command.

I'm not sure how/why it used to work though.

Before I didn't even need to run source. I would just alias my root /.env file to /supabase/.env. That seemed to work as intended.
And just to confirm, the env variables aren't loaded in automatically if I omit the source command either.

Issue with having to put export ... is that this then breaks the dotenv format and breaks usage with other tools.

I liked the aliasing workflow as it meant that we didn't have to juggle two sets of configs, and less chances of errors/accidental VCS commits.

I'll see if I can find a workaround, or have to adapt our workflow.

Thanks for investigating 🙏

[soedirgo]

As a workaround: https://unix.stackexchange.com/a/79077. Still not clean, but it works

[alex-ketch]

@soedirgo, I did a bit more testing and identified 0.21.0 as the specific version where this behaviour changed.
I have zero Go knowledge and no knowledge about the reason for the change, but am assuming it's the removal of the godotenv from the main.go file that stopped reading in the .env file values.

package main

import (
-  "github.com/joho/godotenv"
  "github.com/supabase/cli/cmd"
)

func main() {
- _ = godotenv.Load("supabase/.env")
-
  cmd.Execute()
}

[soedirgo]

Thanks for the digging into this. supabase/.env was an implementation detail - it was used to store SUPABASE_REMOTE_DB_URL (and only that), and we now store it in supabase/.temp/remote-db-url.

[alex-ketch]

ah, that explains why my method of aliasing .env -> ./supabase/.env used to work then. I was just coopting the file.

I do think providing a more streamlined way for integrating with existing config conventions such as .env files would be desirable.
Please feel free to close this issue if you feel the above is out of scope, or want to track it another issue—new or reopening #177.
Either way, thanks for your time!

[soedirgo]

Yeah, I might revisit the secrets situation again. Thanks for your inputs!

[pocin]

Hey @soedirgo not sure if i am doing something wrong but if I do
DUMMY=1234 supabase start
while having this in supabase/config.toml

[auth.external.discord]
enabled = true
client_id = "env(DUMMY)"
clinet_secret = "an actual secret"

and go to http://localhost:54321/auth/v1/authorize?provider=discord
i get an error from discord that "env(DUMMY)" is not a valid snowflake, suggesting that the env var did not get injected.

Can you confirm it works for you?

[soedirgo]

@pocin I was able to reproduce this. Thanks for the catch!

[github-actions]

🎉 This issue has been resolved in version 0.32.2 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

[josuemy]

does the fix apply if I run supabase start through npx? This still doesn't work for me, unless I use the export workaround within the .env file

[iwasrobbed]

FYI for others who stumble on this:

think you might need to use export GITHUB_CLIENT_ID=abcd

export GOTRUE_EXTERNAL_GITHUB_CLIENT_ID=id_abcd
export GOTRUE_EXTERNAL_GITHUB_SECRET=key_abcd

Without the export, it'll fail even if echo $GOTRUE_EXTERNAL_GITHUB_CLIENT_ID succeeds

(Probably would be good to put this in the docs or readme under https://github.com/supabase/gotrue#external-authentication-providers)

[lightstrike]

Thanks @iwasrobbed, saved me! 🌟