Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Put dev secrets into chamber #2089

Merged

Conversation

@chrisgilmerproj
Copy link
Contributor

commented May 7, 2019

Description

NOTE: This does not replace .envrc.local. It's supposed to be a no-op for those that don't have access to chamber.

Many of our newer developers don't have access to GDrive. This means they start the project and cannot get the secrets for their .envrc.local that they need. This change puts all those secrets into chamber so they can be accessed by anyone with AWS access. It also runs an eval statement to grab all those secrets so that users no longer have to fill them into an .envrc.local.

Here are the secrets added:

chamber list app-devlocal

Reviewer Notes

What is the user experience like with this?

Setup

This should remove the need for much of the .envrc.local file (but probably not all of it). So try getting rid of it as a starting point to see how this works. You'll want to bring back parts of it you want after you try things out.

mv .envrc.local .envrc.local.bkp
direnv allow
touch .envrc.local 
# Now add back the things you want

Code Review Verification Steps

  • Request review from a member of a different team.
  • Have the Pivotal acceptance criteria been met for this change?

References

@tinyels
Copy link
Contributor

left a comment

direnv allow is locking up for me because it wants my aws token, but won't let me enter it.

image

@chrisgilmerproj

This comment has been minimized.

Copy link
Contributor Author

commented May 7, 2019

I've added a change that makes this work nicely if you either 1) don't have access to AWS SSM, 2) incorrectly enter your MFA token. It now just warns you instead of trying to eval the error statement.

Here is the output of such a call coupled with missing variables:

direnv allow
direnv: loading .envrc
Enter token for arn:aws:iam::923914045601:mfa/cgilmer: 1234567
direnv: Unable to access app-devlocal variables with chamber.
direnv: Login to chamber with 'chamber list app-devlocal'.
direnv: loading .envrc.local
direnv: loading ~/Projects/transcom/ppp-infra/transcom-ppp/.envrc
direnv: LOGIN_GOV_SECRET_KEY is not set: See https://docs.google.com/document/d/148RzqgaQbhOxXd4z_xuj5Jz8JNETThrn7RVFmMqXFvk
direnv: MOVE_MIL_DOD_TLS_CERT is not set: See https://docs.google.com/document/d/1nvLXLQYz5ax3Ds4n2Y5OeANJhs0AbHtjkrKzI0gN3_o
direnv: MOVE_MIL_DOD_TLS_KEY is not set: See https://docs.google.com/document/d/1nvLXLQYz5ax3Ds4n2Y5OeANJhs0AbHtjkrKzI0gN3_o
direnv: HERE_MAPS_APP_ID is not set: See https://docs.google.com/document/d/16ZomLuR6BPEIK4enfMcqu31oiJYZWNDe9Znyf9e88dg
direnv: HERE_MAPS_APP_CODE is not set: See https://docs.google.com/document/d/16ZomLuR6BPEIK4enfMcqu31oiJYZWNDe9Znyf9e88dg
direnv: GEX_BASIC_AUTH_PASSWORD is not set: See https://docs.google.com/document/d/1nvLXLQYz5ax3Ds4n2Y5OeANJhs0AbHtjkrKzI0gN3_o
direnv: DPS_AUTH_SECRET_KEY is not set: https://docs.google.com/document/d/1HAD9tu9WahzVEam5FFWrgywdMm4aTfVW-Mp3rL7idAo
direnv: DPS_AUTH_COOKIE_SECRET_KEY is not set: https://docs.google.com/document/d/1HAD9tu9WahzVEam5FFWrgywdMm4aTfVW-Mp3rL7idAo
direnv: CSRF_AUTH_KEY is not set: See https://docs.google.com/document/d/1DuWXZLFaW7FXvqh-PStqjZI40niEavXWS5PPtWPlK3w
direnv: EIA_KEY is not set: https://docs.google.com/document/d/1K1-xlYcZaS518PQiaB39gSvqz2tTo0W8eM0wImB7TcI
direnv: Your environment is missing some variables!
direnv: Set the above variables in .envrc.local and try again.
direnv: export +AWS_PROFILE +AWS_S3_BUCKET_NAME +AWS_S3_KEY_NAMESPACE +AWS_S3_REGION +AWS_SDK_LOAD_CONFIG +AWS_SES_DOMAIN +AWS_SES_REGION +AWS_VAULT_KEYCHAIN_NAME +CHAMBER_KMS_KEY_ALIAS +CHAMBER_RETRIES +CHAMBER_USE_PATHS +CLIENT_AUTH_SECRET_KEY +DANGEROUSLY_DISABLE_HOST_CHECK +DB_HOST +DB_NAME +DB_NAME_DEV +DB_NAME_PROD_MIGRATIONS +DB_NAME_TEST +DB_PASSWORD +DB_PORT +DB_PORT_PROD_MIGRATIONS +DB_PORT_TEST +DB_SSL_MODE +DB_USER +DEVLOCAL_CA +DOD_CA_PACKAGE +DPS_COOKIE_EXPIRES_IN_MINUTES +DPS_COOKIE_NAME +DPS_REDIRECT_URL +EIA_URL +GEX_BASIC_AUTH_USERNAME +GEX_URL +GO111MODULE +HERE_MAPS_GEOCODE_ENDPOINT +HERE_MAPS_ROUTING_ENDPOINT +HONEYCOMB_API_HOST +HONEYCOMB_API_KEY +HONEYCOMB_DATASET +HONEYCOMB_DEBUG +HONEYCOMB_ENABLED +HTTP_SDDC_PORT +HTTP_SDDC_PROTOCOL +IWS_RBS_HOST +LOGIN_GOV_CALLBACK_PORT +LOGIN_GOV_CALLBACK_PROTOCOL +LOGIN_GOV_HOSTNAME +LOGIN_GOV_MY_CLIENT_ID +LOGIN_GOV_OFFICE_CLIENT_ID +LOGIN_GOV_TSP_CLIENT_ID +MOVE_MIL_DOD_CA_CERT +MYMOVE_DIR +NO_SESSION_TIMEOUT +PPP_INFRA_PATH +SECURE_MIGRATION_DIR +SERVE_SWAGGER_UI +TZ ~PATH

@chrisgilmerproj chrisgilmerproj requested a review from tinyels May 7, 2019

@tinyels

This comment has been minimized.

Copy link
Contributor

commented May 8, 2019

direnv allow won't work unless I switch from fish to bash. Since several of us use fish, we need to figure out what the issue is. Maybe @jim can help?

@chrisgilmerproj

This comment has been minimized.

Copy link
Contributor Author

commented May 8, 2019

direnv allow won't work unless I switch from fish to bash. Since several of us use fish, we need to figure out what the issue is. Maybe @jim can help?

Does it just not work at all? Because if it acts as a no-op in the fish shell then that's fine and we can work out alternatives later. But if it stops direnv allow from working at all then I need a new solution. It's supposed to fail-over as a no-op, not disable direnv from working.

Personally I'd also like to know how direnv works a little more because our .envrc is bash script itself and even points to /usr/bin/env bash at the top. So I'd expect it to do whatever it does in bash and then for direnv to act appropriately with that information in your desired shell. I find it odd that the output of that file gets swallowed by fish, but honestly I don't know that shell at all. I'll probably dig into this later.

I also want to re-iterate that this doesn't replace .envrc.local, it's just supposed to make life easier for people and otherwise be a no-op where the fallback plan is to use the GDrive links and our old copy-paste method.

The alternative is that I can update the docs and just tell people to put this line of code in their .envrc.local .

In the meantime I'm going to give you and @akostibas access to our chamber secrets and I'll be curious if that changes the behavior at all. You can try chamber list app-devlocal && direnv allow to see if that fixes it later today.

.envrc Outdated
log_error "Login to chamber with 'chamber list app-devlocal'."
else
chamber_env="$(AWS_VAULT_KEYCHAIN_NAME=login aws-vault exec transcom-ppp -- chamber env app-devlocal --retries=1)"
eval "$chamber_env"

This comment has been minimized.

Copy link
@chrisgilmerproj

chrisgilmerproj May 8, 2019

Author Contributor

One issue here may be my use of eval whereas in fish you're supposed to use source. I picked that up looking at this commit:

direnv/direnv@5b4df96

This comment has been minimized.

Copy link
@chrisgilmerproj

chrisgilmerproj May 8, 2019

Author Contributor

The fish docs say this ought to work though: https://fishshell.com/docs/current/commands.html#eval

@chrisgilmerproj

This comment has been minimized.

Copy link
Contributor Author

commented May 8, 2019

@tinyels @jim - can you try this branch with the command direnv export fish | source instead of direnv allow? It looks to me that direnv allow output is swallowed by this function which is why you can't see anything:

https://github.com/direnv/direnv/blob/master/shell_fish.go#L12-L16

So the solution here would be making a script named scripts/direnv-fish with the contents direnv export fish | source.

Btw, I did download the fish shell and this worked for me so I just want to confirm.

@pjdufour-truss
Copy link
Contributor

left a comment

Works great for me, but would be good to get a few testers!

chrisgilmerproj added some commits May 9, 2019

@chrisgilmerproj

This comment has been minimized.

Copy link
Contributor Author

commented May 9, 2019

@tinyels - this is now an opt-in feature that you enable by copying the template over.

@tinyels

This comment has been minimized.

Copy link
Contributor

commented May 10, 2019

I am now able to run without using chamber and with using chamber. (Though will need to remember to call direnv export fish | source instead of direnv allow)

@chrisgilmerproj chrisgilmerproj merged commit 24b3c3f into master May 10, 2019

19 checks passed

Codacy/PR Quality Review Up to standards. A positive pull request.
Details
ci/circleci: acceptance_tests_experimental Your tests passed on CircleCI!
Details
ci/circleci: acceptance_tests_local Your tests passed on CircleCI!
Details
ci/circleci: acceptance_tests_staging Your tests passed on CircleCI!
Details
ci/circleci: build_app Your tests passed on CircleCI!
Details
ci/circleci: build_migrations Your tests passed on CircleCI!
Details
ci/circleci: build_tools Your tests passed on CircleCI!
Details
ci/circleci: client_test Your tests passed on CircleCI!
Details
ci/circleci: integration_tests_api Your tests passed on CircleCI!
Details
ci/circleci: integration_tests_mymove Your tests passed on CircleCI!
Details
ci/circleci: integration_tests_office Your tests passed on CircleCI!
Details
ci/circleci: integration_tests_tsp Your tests passed on CircleCI!
Details
ci/circleci: pre_deps_golang Your tests passed on CircleCI!
Details
ci/circleci: pre_deps_yarn Your tests passed on CircleCI!
Details
ci/circleci: pre_test Your tests passed on CircleCI!
Details
ci/circleci: server_test Your tests passed on CircleCI!
Details
ci/circleci: server_test_coverage Your tests passed on CircleCI!
Details
codecov/patch Coverage not affected when comparing 17e5a49...9277dfa
Details
codecov/project/go 60.12% remains the same compared to 17e5a49
Details

@chrisgilmerproj chrisgilmerproj deleted the cg_164971437_dev_secrets_in_chamber_app_devlocal branch May 10, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.