/opt/elasticbeanstalk/bin/get-config doesn't exist for Amazon Linux 2 Platforms

[curtismorte]

Issue:

In a lot of .ebextensions configurations for Amazon Linux 1.X.X platforms, you can get environment configurations using the following command within bash files:

SOME_VARIABLE=$(/opt/elasticbeanstalk/bin/get-config environment -k 'ENV_VARIABLE_KEY')

For Amazon Linux 2, this command has been removed (from what I can tell)

TL;DR

The file /opt/elasticbeanstalk/bin/get-config environment is missing from Amazon Linux 2. To reproduce some functionality like so, we can use the output from calling/opt/aws/bin/cfn-get-metadata.

Possible Solution:

After some inspection in the logs, I was able to deduce that you can still access environment variables, but from using a different command:

• /opt/aws/bin/cfn-get-metadata

There was a specific log entry that tipped me off to this, which I'll omit from showing here but you can quickly find it by looking for the command in the log file /var/log/eb-engine.log.

I was able to get environment variables in the following way:

1. Gather the Environment Region and Stack ID
2. Pass those values to /opt/aws/bin/cfn-get-metadata to get environment variables

# --------------------------------------------------
# HOW TO GET ENVIRONMENT VARIABLES
# --------------------------------------------------

# Get the Region
ENV_REGION="$(</opt/elasticbeanstalk/config/ebenvinfo/region)"

# Get the Stack ID
ENV_STACK_ID="$(</opt/elasticbeanstalk/config/ebenvinfo/stackid)"

# Pass the variables to the CloudFormation Meta Data command.
# Pipe the data to JQ to reduce the environment variables array to an actual key value map
ENV_VARS_KWARGS="$(sudo /opt/aws/bin/cfn-get-metadata -s $ENV_STACK_ID -r AWSEBBeanstalkMetadata --region $ENV_REGION -k "AWS::ElasticBeanstalk::Ext.Parameters.EnvironmentVariables" | jq -c '. |= reduce (.[] | split("=")) as [$key, $value] ({}; .[$key] = $value)')"

# Access any environment variable like so
MY_VARIABLE_VALUE=$(jq -r '.MY_VARIABLE_KEY' <<< $ENV_VARS_KWARGS)

Conclusion

I feel like this isn't an optimal solution, but was able to give me what I needed relative to what I was able to find though my own inspection.

Closing Thoughts

I have some final thoughts for other people that may help to elaborate more on what the optimal solution for this could be:

1. I realized the file /opt/elasticbeanstalk/config/ebenvinfo/env is temporarily written during the build process. Exposing or extending the same script that builds this file may produce the missing functionality from Amazon Linux v1's /opt/elasticbeanstalk/bin/get-config.
2. Wrap my initial solution into a platform prebuild hook (.platform/hooks/prebuild) and take the output from ENV_VARS_KWARGS and write it to a file. You could access the file like so:

   # Using JQ
   MY_VARIABLE_VALUE=$(jq -r '.MY_VARIABLE_KEY' <<< /path/to/env_kwargs_saved.json)
   

[dankhen]

@curtismorte This is great stuff! Thanks for posting.
The Beanstalk team is aware of this gap, and bringing get-config back to AL2 is something they're looking into. This is just one of many cases where we need to adapt either the AL2 platforms or our code/config examples to include AL2. It's work in progress and will take some time.

It would be wonderful if you could post your solution on the AWS Beanstalk forum!

[tomchiverton]

Eh? Why is this closed if AWS know a ton of their own scripts don't work on their own new platform.
Really not looking forward to migrating our Amazon Linux v1 to v2 now!

Maybe I can knock up a compatability script and ship that as a .ebextension file...

[dankhen]

@tomchiverton Thanks for asking. As I wrote above, I closed the issue because we're aware of it and we're tracking it internally, so we didn't need to keep the GitHub issue open. In this specific case, the team indicated to me their intention of bringing back get-config (or a version of it) to the new AL2 platforms.

That said, an .ebextensions file that people can use for now would definitely be welcome. I apologize that we're behind on providing up-to-date examples for the AL2 platforms.

[MatthiasKunnen]

My .ebextensions file. I've started from @curtismorte's solution.

Features:

• get the environment variables as JSON
• export everything to environment variables in your current shell
• fetch single values

# Filename: .ebextensions/000-get-config.config
files:
    "/opt/elasticbeanstalk/bin/get-config-json":
        mode: "000755"
        content: |
            #!/usr/bin/env bash

            # -----------------------------------------------------
            # Get all environment variables as JSON key value pairs
            # -----------------------------------------------------
            # https://github.com/awsdocs/elastic-beanstalk-samples/issues/111#issue-614915898

            # Get the Region
            ENV_REGION="$(</opt/elasticbeanstalk/config/ebenvinfo/region)"

            # Get the Stack ID
            ENV_STACK_ID="$(</opt/elasticbeanstalk/config/ebenvinfo/stackid)"

            # Pass the variables to the CloudFormation Meta Data command.
            # Pipe the data to JQ to reduce the environment variables array to an actual key value map
            echo "$(sudo /opt/aws/bin/cfn-get-metadata -s ${ENV_STACK_ID} -r AWSEBBeanstalkMetadata --region ${ENV_REGION} -k "AWS::ElasticBeanstalk::Ext.Parameters.EnvironmentVariables" | jq -c '. |= reduce (.[] | split("=")) as [$key, $value] ({}; .[$key] = $value)')"

    "/opt/elasticbeanstalk/bin/export-config":
        mode: "000755"
        content: |
            #!/usr/bin/env bash

            # Run: `export $(sudo ./export-config)`
            # https://stackoverflow.com/questions/48512914/exporting-json-to-environment-variables
            echo $(./get-config-json | jq -r "to_entries|map(\"\(.key)=\(.value|tostring)\")|.[]")


    "/opt/elasticbeanstalk/bin/get-config":
        mode: "000755"
        content: |
            #!/usr/bin/env bash

            # Run: `sudo ./get-config DATABASE_URL`
            echo $(./get-config-json | jq -r ".$1")

[lorenzoLectrefy]

@MatthiasKunnen @dankhen

coudn't this be merged in? not a good idea?

[dankhen]

@lorenzoLectrefy I'm open to merging a PR with code like the above, if anyone submitted it into the configuration-files/community-provided directory.

I'd ask that the script name would not be get-config though. As I said above, the Beanstalk team are working on bringing back get-config to the AL2 platforms, so I'd like to make sure that, once this happens, all existing code examples keep using the official program, and not a temporary solution.

Folks who want to use the temporary solution can explicitly change their calls to use the temporary name.

[dankhen]

@lorenzoLectrefy Actually never mind. We already have one:
configuration-files/community-provided/instance-configuration/000_amazon_linux_2_compat_get-config.config

In hindsight, it should have been called something other than get-config.

[MatthiasKunnen]

@dankhen, I believe I tried that script before and it didn't work. I'm afraid I can't remember what exactly failed.

[dankhen]

@MatthiasKunnen In that case, you're welcome to both submit yours as well, and correspond with the other example's submitter to point out issues.
Again, this is temporary, and the team is working to bring the official solution back.

[lorenzoLectrefy]

@dankhen any idea on timescale for release of fix?

[dankhen]

@lorenzoLectrefy Unfortunately we cannot share a specific time. An engineer is working on implementing it, and we'll release it in one of the coming AL2 platform updates.

[DangerDawson]

There are also scripts that use it with the container argument e.g.

get-config container -k app_user

Will this be added as well?

[dankhen]

@DangerDawson Yes, we'll implement the complete set of get-config commands, to be consistent with the Amazon Linux AMI (AL1) version of the tool.

[mahesvaran]

$ sudo ENV_REGION="$(</opt/elasticbeanstalk/config/ebenvinfo/region)"
-bash: /opt/elasticbeanstalk/config/ebenvinfo/region: Permission denied

I got the above error.

[tomchiverton]

Later AL2 based Beanstalk's do have the official version, simply remove this script and upgrade.

[culov]

Any update on this? None of the proposed solutions have worked for me.

[dankhen]

@culov No workaround is necessary anymore. We brought back get-config to AL2 platforms early August. Here are the release notes:
https://docs.aws.amazon.com/elasticbeanstalk/latest/relnotes/release-2020-08-04-al2.html