Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Local environment variables don't work with CloudFormation references #2817

Closed
jeremydaly opened this issue Nov 28, 2016 · 11 comments

Comments

Projects
None yet
4 participants
@jeremydaly
Copy link
Contributor

commented Nov 28, 2016

This is a Bug Report

Description

When specifying CloudFormation references in environment variables (e.g. QUEUE_URL: { "Ref": "queue" }), the values don't load when invoking the function locally.

For bug reports:

  • What went wrong?
    Environment variables that used CloudFormation references displayed [Object object]
  • What did you expect should have happened?
    The CloudFormation references should have been resolved (e.g. arn:aws:sns:us-east-1:XXXXXXXXXXXX:notifications)
  • What was the config you used?
environment:
  NOTIFICATION_TOPIC: { "Ref": "notifications" }
  • What stacktrace or error message from your provider did you see?
    n/a
    Similar or dependent issues:
  • none

Additional Data

  • Serverless Framework Version you're using: 1.2.1
  • Operating System: MacOS
  • Stack Trace: n/a
  • Provider Error messages: n/a

@pmuens pmuens added the bug label Nov 28, 2016

@dougmoscrop

This comment has been minimized.

Copy link
Contributor

commented Nov 29, 2016

Sorry, ignore my other comment, I missed the locally part.

@dougmoscrop

This comment has been minimized.

Copy link
Contributor

commented Nov 30, 2016

Could you elaborate what you would like to happen here? The arn that would be 'resolved' would possibly not exist (unless you were trying to get information about a CF Stack that exists, which would require a --stage). Or, it would have go be faked, which would require very complicated and brittle logic.

Or (and I think I prefer this) it would try to inherit the environment variable from the serverless invoke local command (e.g. NOTIFICATION_TOPIC=x serverless invoke local and maybe throw an error if it's unspecified.

Thoughts?

@jeremydaly

This comment has been minimized.

Copy link
Contributor Author

commented Nov 30, 2016

I'm using the ARNs in my Lambda functions to interact with other AWS services that are generated using the Resources section of the Serverless.yml file. In order for my code to dynamically load the SQS QueueUrl or SNS TopicArn, I need access to those through some shared variable system. The new environment variable support works perfectly when I deploy the service and run it in Lambda. However, when I invoke functions locally, Fn::GetAtt and Ref: references in Serverless.yml obviously don't resolve correctly.

Ideally, a serverless deploy would somehow store a reference to the resources I created (maybe in the .serverless folder) OR invoking locally would call the appropriate CloudFormation stack on AWS and load the outputs into my local ENV variables. Perhaps this could be solved with a plugin instead.

Bottomline is that references to the resources generated via Serverless.yml should be accessible within your Lambda functions when invoking remotely or locally. It would obviously require an initial deploy in order to create the resources (and then invoke locally). Otherwise I need to either hard code the references (which requires me to do an initial deploy anyway) or create a workaround (such as hardcoding the account id so that I can construct ARNs and service URLs manually). Neither of these are ideal since I should be able to deploy this to any AWS account without needing to change any of the code or the Serverless.yml file (other than maybe the provider.profile).

@dougmoscrop

This comment has been minimized.

Copy link
Contributor

commented Nov 30, 2016

Right, so if you had your stuff deployed, serverless invoke local -f foo -s dev would be able to fetch the functions Environment from CloudFormation, I think.

If you did not specify the stage, it would have to throw an error or warning.

@jeremydaly

This comment has been minimized.

Copy link
Contributor Author

commented Nov 30, 2016

If invoked without a stage, it could just use the default in the provider.stage variable.

@dougmoscrop

This comment has been minimized.

Copy link
Contributor

commented Nov 30, 2016

Yeah, I'm on the fence about whether that's ideal, i.e. do you want a round trip to AWS every time? Would it take away the ability to run a function with custom/overriden environment variables?

@jeremydaly

This comment has been minimized.

Copy link
Contributor Author

commented Nov 30, 2016

Hmm, I get what you're saying. I guess this is a development convenience more than anything else. I'd hate to add a bunch of overhead into the invoke local functionality, but at the same time I need a way to invoke it locally without hard coding data in the functions or serverless.yml.

Could there be another way to load local environment variables other than using the stage variables? For example, could serverless invoke local -f foo set a flag that was available via a variable in the serverless.yml so I could use that to load a different set of environment variables?

@dougmoscrop

This comment has been minimized.

Copy link
Contributor

commented Dec 1, 2016

Could do:

sls invoke local -f foo
This would be the new default and would attempt to fetch the environment from CloudFormation based on stage and region.

sls invoke local -f foo --env
This would use the local environment, filtering out any non-string values (code can still use defaults)

sls invoke local -f foo -e X=Y -e FOO=BAR
This would do the same as --env above (-e is just shorthand) and specify two values, these would be merged with the ones from yaml, so you can "fill in" or override the omitted ones that were not resolvable

@jeremydaly

This comment has been minimized.

Copy link
Contributor Author

commented Dec 1, 2016

I like the general idea. You could also use something similar to the --path capability where you could specify a file with stored environment variables for testing. Maybe a deploy script could even create a default file for you?

@eahefnawy

This comment has been minimized.

Copy link
Member

commented Dec 19, 2016

Super interesting use case! I think that can be solved without any changes in core by creative usage of Serverless Variables. Something along the lines of passing an extra option in the CLI that indicate that you're invoking locally, and if it's available use mocked env vars, if not, use real env vars.

Generally agreed with @dougmoscrop though

@pmuens

This comment has been minimized.

Copy link
Member

commented May 19, 2017

Closing since this can now be done with the help of the cf variable (for example) or some other usage of the Serverless Variables system.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.