Allow server port to be set via environment variable #220

f1sherman · 2021-02-12T00:02:54Z

Thank you for this amazing tool!

Heroku expects the web process to dynamically set the listen port based on the PORT variable. This, in combination with
a .profile to set PORT=BULLDOZER_PORT, will make it easier to run bulldozer in Heroku, as well as allowing other HTTP-related config to be set via environment variables.

https://devcenter.heroku.com/articles/container-registry-and-runtime#dockerfile-commands-and-runtime

This is my first time writing Go and I'm sure I didn't get everything exactly right. Please let me know what you'd like me to change here. And of course no worries if this feature cannot be accepted.

palantirtech · 2021-02-12T00:02:56Z

Thanks for your interest in palantir/bulldozer, @f1sherman! Before we can accept your pull request, you need to sign our contributor license agreement - just visit https://cla.palantir.com/ and follow the instructions. Once you sign, I'll automatically update this pull request.

f1sherman · 2021-02-12T00:50:13Z

CLA signed!

bluekeyes · 2021-02-12T02:03:02Z

Thanks for proposing this and I'm glad you find Bulldozer useful! I'm not familiar with Heroku - do applications have to use the plain PORT environment variable or is it possible to map the port to a different variable when you configure things?

I ask because the library we use for the server, go-baseapp, supports setting configuration from environment variables, but we haven't wired it up in Bulldozer yet. When we did this for our companion app, policy-bot, we added a prefix to all the variables. Since Bulldozer can run in a variety of environments, I like the idea of keeping a prefix here as well to keep things clear and avoid confusion. That would mean this variable would be BULLDOZER_PORT instead of just PORT, which might not work in Heroku.

f1sherman · 2021-02-12T10:48:30Z

@bluekeyes that's a great point. As far as I can tell there is no way to change the environment variable that is used for the port. Doc: https://devcenter.heroku.com/articles/dynos#web-dynos

Maybe another way to solve this would be to allow environment variables to be referenced from within the config file, so we could do something like this:

server:
  port: "${PORT}"

I'm not sure how to do that from Go, but it seems possible.

bluekeyes · 2021-02-19T02:10:24Z

It's kind of unclear to me if this works when using containers rather than processes directly, but it sounds like the .profile file can remap environment variables, allowing you to do something like:

export BULLDOZER_PORT=$PORT

(Note that if this works, we'd still need to modify Bulldozer to set configuration from the environment)

If that doesn't work and we need to modify the container itself, I propose something a bit meta:

Add a BULLDOZER_ENV_PREFIX environment variable. If set, this determines the prefix used for other environment variables. If not set, it defaults to BULLDOZER_
Update Bulldozer to set configuration from the environment using the value from (1) as the prefix.

This means that by default we'll have the prefixes for variables, but in Heroku (or other places where you know things are isolated), you can set BULLDOZER_ENV_PREFIX to the empty string, and use the plain PORT variable.

I think this will be easier to implement and maintain than replacing environment variable references in the configuration file.

f1sherman · 2021-02-19T14:02:46Z

@bluekeyes I love it! I will check in with Heroku support to confirm how .profile works (or doesn't) with containers and report back here. Thank you!

f1sherman · 2021-02-22T16:26:50Z

@bluekeyes after talking with Heroku support it seems that .profile should work just fine in Dockerized-heroku. I went ahead and updated this PR. Thank you!

bluekeyes

Thanks for following up with Heroku support and iterating with me on this! I have one implementation suggestion that will take advantage of existing code and also support some additional variables for free.

bluekeyes · 2021-02-22T17:14:09Z

server/server.go

+	if v, ok := os.LookupEnv("BULLDOZER_PORT"); ok {
+		if i, err := strconv.Atoi(v); err == nil {
+			c.Server.Port = i
+		}
+	}


Instead of only supporting BULLDOZER_PORT here, I'd like to do something that will support additional variables as well:

In the ParseConfig function in server/config.go somewhere around L68, you can add a call to c.Server.SetValuesFromEnv("BULLDOZER_"). This will parse the BULLDOZER_PORT variable, but also add support for configuring the listen address, TLS options, and other properties.

That sounds awesome. I will try to get to this in the coming days. Thanks so much for your help and support!

@bluekeyes sorry for taking so long to get back to you on this! I looked into adding c.Server.SetValuesFromEnv, but it appears that the SetValuesFromEnv method wasn't added until a later version of go-baseapp than what this project is using. I submitted #228 to hopefully unblock us. Thanks again!

Thanks for doing the update as a separate PR. I've merged that, so you should be good to update this.

@bluekeyes of course! I think this should be all set now. I rebased for a clean git history and updated the PR description. Thanks again!

Heroku expects the web process to dynamically set the listen port based on the PORT variable. This, in combination with a [.profile](https://devcenter.heroku.com/articles/dynos#the-profile-file) to set `PORT=BULLDOZER_PORT`, will make it easier to run bulldozer in Heroku, as well as allowing other HTTP-related config to be set via environment variables.

bluekeyes

Looks good! I'll try to include this in a release either later today or tomorrow.

f1sherman · 2021-03-18T20:13:33Z

Fantastic, thanks so much! Would it be too much trouble to update the Docker image as well? Really appreciate the quick response btw!

bluekeyes · 2021-03-18T20:21:44Z

New Docker tags (latest and the release version) are published automatically when we make releases. If you'd like to test before then, the snapshot Docker tag is updated with every commit to develop and already includes this change.

f1sherman · 2021-03-18T20:24:45Z

@bluekeyes I'll give that a try, thanks!

As suggested [here](palantir#220 (comment)), use a `BULLDOZER_ENV_PREFIX` environment variable to override the prefix for server configuration values. This makes it possible to run the Docker container on Heroku, where it is expected that the container will listen on the port specified in the `PORT` environment variable.

bluekeyes requested changes Feb 22, 2021

View reviewed changes

f1sherman force-pushed the allow-port-from-env branch 2 times, most recently from c2ab2e2 to b5e6303 Compare March 18, 2021 19:59

f1sherman force-pushed the allow-port-from-env branch from b5e6303 to a490697 Compare March 18, 2021 20:01

bluekeyes approved these changes Mar 18, 2021

View reviewed changes

bluekeyes merged commit c27b66a into palantir:develop Mar 18, 2021

f1sherman mentioned this pull request Mar 19, 2021

Allow server env var prefix to be overridden #229

Merged

bluekeyes mentioned this pull request Aug 19, 2021

Can I configure a different port? #170

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Allow server port to be set via environment variable #220

Allow server port to be set via environment variable #220

f1sherman commented Feb 12, 2021 •

edited

Loading

palantirtech commented Feb 12, 2021

f1sherman commented Feb 12, 2021

bluekeyes commented Feb 12, 2021

f1sherman commented Feb 12, 2021

bluekeyes commented Feb 19, 2021

f1sherman commented Feb 19, 2021

f1sherman commented Feb 22, 2021

bluekeyes left a comment

bluekeyes Feb 22, 2021

f1sherman Feb 23, 2021

f1sherman Mar 18, 2021

bluekeyes Mar 18, 2021

f1sherman Mar 18, 2021

bluekeyes left a comment

f1sherman commented Mar 18, 2021

bluekeyes commented Mar 18, 2021

f1sherman commented Mar 18, 2021

Allow server port to be set via environment variable #220

Allow server port to be set via environment variable #220

Conversation

f1sherman commented Feb 12, 2021 • edited Loading

palantirtech commented Feb 12, 2021

f1sherman commented Feb 12, 2021

bluekeyes commented Feb 12, 2021

f1sherman commented Feb 12, 2021

bluekeyes commented Feb 19, 2021

f1sherman commented Feb 19, 2021

f1sherman commented Feb 22, 2021

bluekeyes left a comment

Choose a reason for hiding this comment

bluekeyes Feb 22, 2021

Choose a reason for hiding this comment

f1sherman Feb 23, 2021

Choose a reason for hiding this comment

f1sherman Mar 18, 2021

Choose a reason for hiding this comment

bluekeyes Mar 18, 2021

Choose a reason for hiding this comment

f1sherman Mar 18, 2021

Choose a reason for hiding this comment

bluekeyes left a comment

Choose a reason for hiding this comment

f1sherman commented Mar 18, 2021

bluekeyes commented Mar 18, 2021

f1sherman commented Mar 18, 2021

f1sherman commented Feb 12, 2021 •

edited

Loading