password skipping config substitution

[EliMorato]

Describe the bug
When using environment variables to provide datasource credentials in great_expectations.yml, example:

    credentials:
      drivername: postgresql+psycopg2
      host: ${GE_REDSHIFT_HOST}
      port: ${GE_REDSHIFT_PORT}
      username: ${GE_REDSHIFT_USER}
      password: ${GE_REDSHIFT_PASSWORD}
      database: ${GE_REDSHIFT_DB}

the password is reading "${GE_REDSHIFT_PASSWORD}" as the password value due to this bugfix.

To Reproduce
Steps to reproduce the behavior:

1. Use an environment variable in your datasource credentials' password
2. Run anything that needs to connect to the datasource. For example, you can create a new suite or edit an old one and run the first Jupyter Notebook cell. You'll get an authentication error like this:

Expected behavior
Not to crash and also read the value of the environment variable (${GE_REDSHIFT_PASSWORD} in this case) as the password value for the datasource credentials.

Environment (please complete the following information):

• OS: All
• GE Version: 0.12.6

Additional context
@anthonyburdi

[whatsrupp]

+1 on this, am currently working around this by generating the yaml file and interpolating in the password from the environment manually.

[martinheld]

+1

[eugmandel]

@EliPino @whatsrupp @martinheld This is indeed a case of a fix breaking something else. We will fix this in a more robust way and cut a release.

[whatsrupp]

@eugmandel Perfect!! While I'm here, just a bit of positive feedback: I have recently set up GE on a small data pipeline at work, it was invigorating how quick it was to scaffold out some meaningful tests that I needed to put together! So, thank you!

[anthonyburdi]

Thank you all for your patience! In this branch anthony/bugfix/issue-2005-password-substitution I have a preliminary fix in place (revert the bugfix #1949 & add escaping for $ characters in actual passwords), and am currently writing / cleaning up tests and documentation.

[martinheld]

🥇 @anthonyburdi thanks a lot!

[anthonyburdi]

Thanks again for your patience! GE v0.12.7 now:

• does not handle the password field differently
• allows for the escaping of $ characters in config values
• allows for multiple substitutions in a single config value

I hope this fixes your issues, please let us know if it does not!