Skip to content

Commit

Permalink
Merge pull request #1834 from serverless/sls-vars
Browse files Browse the repository at this point in the history
Serverless Variables
  • Loading branch information
flomotlik committed Aug 31, 2016
2 parents 2bc3ac3 + 3016a55 commit e782df7
Show file tree
Hide file tree
Showing 39 changed files with 1,051 additions and 1,678 deletions.
2 changes: 1 addition & 1 deletion README.md
Expand Up @@ -55,7 +55,7 @@ Run `serverless remove` to clean up this function from your account.
- **[Understanding Serverless and its configuration files](docs/understanding-serverless)**
- [Serverless services and functions](docs/understanding-serverless/services-and-functions.md)
- [serverless.yml](docs/understanding-serverless/serverless-yml.md)
- [serverless.env.yml](docs/understanding-serverless/serverless-env-yml.md)
- [serverless variables](docs/understanding-serverless/serverless-variables.md)
- **[How to build your Serverless services](docs/guide)**
- [Installing Serverless](docs/guide/installation.md)
- [Provider account setup](docs/guide/provider-account-setup.md)
Expand Down
4 changes: 2 additions & 2 deletions docs/README.md
Expand Up @@ -30,10 +30,10 @@ You may also take a look at our [code of conduct](/code_of_conduct.md).

## User documentation

- [Understanding Serverless and its configuration files](understanding-serverless)
- [Understanding Serverless and the configuration file](understanding-serverless)
- [Serverless services and functions](understanding-serverless/services-and-functions.md)
- [serverless.yml](understanding-serverless/serverless-yml.md)
- [serverless.env.yml](understanding-serverless/serverless-env-yml.md)
- [Serverless variables](understanding-serverless/serverless-variables.md)
- [How to build your Serverless services](guide)
- [Installing Serverless](guide/installation.md)
- [Provider account setup](guide/provider-account-setup.md)
Expand Down
6 changes: 1 addition & 5 deletions docs/developing-plugins/building-provider-integrations.md
Expand Up @@ -57,8 +57,6 @@ Serverless will automatically exclude the following files / folders to reduce th
- .serverless
- serverless.yaml
- serverless.yml
- serverless.env.yaml
- serverless.env.yml
- .DS_Store

You can always include previously excluded files and folders if you want to.
Expand All @@ -84,9 +82,7 @@ Curious how this works for the Amazon Web Services (AWS) provider integration?
Here are the steps the AWS plugins take to compile and deploy the service on the AWS infrastructure in detail.

#### The steps in detail

1. The [`serverless.yml`](../understanding-serverless/serverless-yml.md) and
[`serverless.env.yml`](../understanding-serverless/serverless-env-yml.md) files are loaded into memory
1. The [`serverless.yml`](../understanding-serverless/serverless-yml.md) configuration file is loaded into memory
2. A default AWS CloudFormation template is loaded and deployed to AWS (A S3 bucket for the service gets created)
(`deploy:setupProviderConfiguration`)
3. The functions of the [`serverless.yml`](../understanding-serverless/serverless-yml.md) file are compiled to lambda
Expand Down
12 changes: 1 addition & 11 deletions docs/guide/creating-a-service.md
Expand Up @@ -13,7 +13,7 @@ To create a service with a `nodejs` runtime running on `aws` just pass the `aws-
serverless create --template aws-nodejs
```

This will create a service and generate `serverless.yml`, `serverless.env.yml` and `handler.js` files in the current
This will create a service and generate `serverless.yml` and `handler.js` files in the current
working directory.

## Open the service inside your editor
Expand All @@ -23,7 +23,6 @@ favorite editor.

You'll see the following files:
- `serverless.yml`
- `serverless.env.yml`
- `handler.js`

### [`serverless.yml`](../understanding-serverless/serverless-yml.md)
Expand All @@ -34,15 +33,6 @@ This is our core service file. You can see the name of our service, the provider
If you want to learn more about the [`serverless.yml`](../understanding-serverless/serverless-yml.md) file you might
want check out our [in depth guide](../understanding-serverless/serverless-yml.md) about it.

### [`serverless.env.yml`](../understanding-serverless/serverless-env-yml.md)

This file includes all the environment variables Serverless uses. It automatically creates environment variables for a
default stage called `dev` and a default region called `us-east-1`. Those variables will be referenced and used as a
default later on for deployment.

If you want to learn more about the [`serverless.env.yml`](../understanding-serverless/serverless-env-yml.md) file you
might want check out our [in depth guide](../understanding-serverless/serverless-env-yml.md) about it.

### `handler.js`

The `handler.js` file includes a function skeleton which returns a simple message. The function definition in
Expand Down
6 changes: 3 additions & 3 deletions docs/guide/custom-provider-resources.md
Expand Up @@ -18,7 +18,7 @@ After initialization, Serverless will try to load the `resources` object from th
It will create an own, empty one if it doesn't exist.

You can use this place to add custom provider resources by writing the resource definition in YAML syntax inside the
`resources` object. You can also use your variables from `serverless.env.yml` in the Values
`resources` object. You can also use Serverless Variables for sensitive data or reusable configuration in your resources templates.

**Note:** You'll have the whole flexibility to overwrite / attach any kind of resource to your CloudFormation stack so
it's no problem to add some new `Resources`, `Outputs` or even overwrite the `Description`. Please be cautious as overwriting
Expand Down Expand Up @@ -54,8 +54,8 @@ resources:
Properties:
# You can also set properties for the resource, based on the CloudFormation properties
BucketName: my-awesome-thumbnails
# Or you could use a variable from your serverless.env.yml
# BucketName: ${bucketname}
# Or you could reference an environment variable
# BucketName: ${env.BUCKET_NAME}
```

### Example custom resources - HTTP Proxy
Expand Down
8 changes: 4 additions & 4 deletions docs/guide/deploying-a-service.md
Expand Up @@ -9,16 +9,16 @@ Make sure that you're still in the service directory.
Run `serverless deploy` to start the deployment process.

Serverless will now deploy the whole service (with all it's functions and events which we'll add soon) to the
services cloud provider. It will use the default `stage` and `region` settings which are defined in
[`serverless.env.yml`](../understanding-serverless/serverless-env-yml.md).
services cloud provider. It will use the default `dev` stage and `us-east-1` region.

The default stage is `dev` and default region is `us-east-1`. You can change the default stage and region in your `serverless.yml` file by setting the `stage` and `region` properties inside a `default` object as the following example shows:
You can change the default stage and region in your `serverless.yml` file by setting the `stage` and `region` properties inside a `provider` object as the following example shows:

```yml
# serverless.yml

service: service-name
defaults:
provider:
name: aws
stage: beta
region: us-west-2
```
Expand Down
7 changes: 2 additions & 5 deletions docs/understanding-serverless/README.md
@@ -1,13 +1,10 @@
# Understanding Serverless

Here we'll take a deep dive into the internals of Serverless and understand what a service is and how functions relate
to a service.

Additionally we look at the Serverless key pieces of each service which are the [`serverless.yml`](serverless-yml.md)
and [`serverless.env.yml`](serverless-env-yml.md) files.
to a service and we'll explore the `serverless.yml` configuration file.

## Table of contents

- [Serverless services and functions](services-and-functions.md)
- [serverless.yml](serverless-yml.md)
- [serverless.env.yml](serverless-env-yml.md)
- [Serverless variables](serverless-variables.md)
25 changes: 0 additions & 25 deletions docs/understanding-serverless/serverless-env-yml.md

This file was deleted.

151 changes: 151 additions & 0 deletions docs/understanding-serverless/serverless-variables.md
@@ -0,0 +1,151 @@
# Serverless Variables
The Serverless framework provides a powerful variable system to give your `serverless.yml` configuration file extra flexibility. With Serverless Variables, you'll be able to do the following:

- Reference & load variables from environment variables
- Reference & load variables from CLI options
- Recursively reference properties of any type from the same `serverless.yml` file
- Recursively reference properties of any type from other YAML/JSON files
- Recursively nest variable references within each other for ultimate flexibility
- Combine multiple variable references to overwrite each other

## Referencing Environment Variables
To reference environment variables, you'll need to use the `${env:SOME_VAR}` syntax in your `serverless.yml` configuration file. Here's an example:

```yml
service: new-service
provider: aws
functions:
hello:
name: ${env:FUNC_PREFIX}-hello
handler: handler.hello
world:
name: ${env:FUNC_PREFIX}-world
handler: handler.world

```

In the previous example you're dynamically adding a prefix to the function names by referencing the `FUNC_PREFIX` env var. So you can easily change that prefix for all functions by changing the `FUNC_PREFIX` env var.

## Referencing CLI Options
To reference CLI options that you passed, you'll need to use the `${opt:some_option}` syntax in your `serverless.yml` configuration file. Here's an example:

```yml
service: new-service
provider: aws
functions:
hello:
name: ${opt:stage}-hello
handler: handler.hello
world:
name: ${opt:stage}-world
handler: handler.world

```

In the previous example you're dynamically adding a prefix to the function names by referencing the `stage` option that you pass in the CLI when you run `serverless deploy --stage dev`. So when you deploy, the function name will always include the stage you're deploying to.

## Recursively Self-Reference serverless.yml Properties
To self-reference properties in `serverless.yml`, you'll need to use the `${self:someProperty}` syntax in your `serverless.yml` configuration file. This functionality is recursive, so you can go as deep in the object tree as you want. Here's an example:

```yml
service: new-service
provider: aws
custom:
globalSchedule: rate(10 minutes)

functions:
hello:
handler: handler.hello
events:
- schedule: ${self:custom.globalSchedule}
world:
handler: handler.world
events:
- schedule: ${self:custom.globalSchedule}

```

In the previous example you're setting a global schedule for all functions by referencing the `globalSchedule` property in the same `serverless.yml` file. This way, you can easily change the schedule for all functions whenever you like.

## Recursively Reference Variables in Other Files
To reference variables in other YAML or JSON files, you'll need to use the `${file(./myFile.yml):someProperty}` syntax in your `serverless.yml` configuration file. This functionality is recursive, so you can go as deep in that file as you want. Here's an example:

```yml

# myCustomFile.yml

globalSchedule: rate(10 minutes)

```


```yml
# serverless.yml
service: new-service
provider: aws
custom: ${file(./myCustomFile.yml)} # You can reference the entire file
functions:
hello:
handler: handler.hello
events:
- schedule: ${file(./myCustomFile.yml):globalSchedule} # Or you can reference a specific property
world:
handler: handler.world
events:
- schedule: ${self:custom.globalSchedule} # This would also work in this case

```

In the previous example you're referencing the entire `myCustomFile.yml` file in the `custom` property. You just need to pass the path relative to your service directory. You can also request specific properties in that file as shown in the `schedule` property. It's completely recursive and you can go as deep as you want.

## Nesting Variable References
The Serverless variable system allows you to nest variable references within each other for ultimate flexibility. So you can reference certain variables based on other variables. Here's an example:

```yml
service: new-service
provider: aws
custom:
myFlexibleArn: ${env:${opt:stage}_arn}

functions:
hello:
handler: handler.hello

```

In the previous example, if you pass `dev` as a stage option, the framework will look for the `dev_arn` environment variable. If you pass `production`, the framework will look for `production_arn`, and so on. This allows you to creatively use multiple variables by using a certain naming pattern without having to update the values of these variables constantly. You can go as deep as you want in your nesting, and can reference variables at any level of nesting from any source (env, opt, self or file).

## Overwriting Variables
The Serverless framework gives you an intuitive way to reference multiple variables as a fallback strategy in case one of the variables is missing. This way you'll be able to use a default value from a certain source, if the variable from another source is missing.

For example, if you want to reference the stage you're deploying to, but you don't want to keep on providing the `stage` option in the CLI. What you can do in `serverless.yml` is:


```yml
service: new-service
provider:
name: aws
stage: dev
custom:
myStage: ${opt:stage, self:provider.stage}

functions:
hello:
handler: handler.hello
```

What this says is to use the `stage` CLI option if it exists, if not, use the default stage (which lives in `provider.stage`). So during development you can safely deploy with `serverless deploy`, but during production you can do `serverless deploy --stage production` and the stage will be picked up for you without having to make any changes to `serverless.yml`.

This overwrite functionality is super powerful. You can have as many variable references as you want, from any source you want, and each of them can be of different type and different name.

# Migrating serverless.env.yml
Previously we used the `serverless.env.yml` file to track Serverless Variables. It was a completely different system with different concepts. To migrate your variables from `serverless.env.yml`, you'll need to decide where you want to store your variables.

* Using a config file: You can still use `serverless.env.yml`, but the difference now is that you can structure the file however you want, and you'll need to reference each variable/property correctly in `serverless.yml`. For more info, you can check the file reference section above.
* Using the same `serverless.yml` file: You can store your variables in `serverless.yml` if they don't contain sensitive data, and then reference them elsewhere in the file using `self:someProperty`. For more info, you can check the self reference section above.
* Using environment variables: You can instead store your variables in environment variables and reference them with `env.someEnvVar`. For more info, you can check the environment variable reference section above.
* Making your variables stage/region specific: `serverless.env.yml` allowed you to have different values for the same variable based on the stage/region you're deploying to. You can achieve the same result by using the nesting functionality of the new variable system. For example, if you have two different ARNs, one for `dev` stage and the other for `prod` stage, you can do the following: `${env:${opt:stage}_arn}`. This will make sure the correct env var is referenced based on the stage provided as an option. Of course you'll need to export both `dev_arn` and `prod_arn` env vars on your local system.

Now you don't need `serverless.env.yml` at all, but you can still use it if you want. It's just not required anymore. Migrating to the new variable system is easy and you just need to know how the new system works and make small adjustments to how you store & reference your variables.


11 changes: 4 additions & 7 deletions docs/understanding-serverless/serverless-yml.md
Expand Up @@ -18,16 +18,14 @@ provider:
subnetIds:
- subnetId1
- subnetId2
stage: beta # Overwrite the default "dev" stage.
region: us-west-2 # Overwite the default "us-east-1" region.
variableSyntax: '\${{([\s\S]+?)}}' # Overwrite the default "${}" variable syntax to be "${{}}" instead

plugins:
- additional_plugin
- another_plugin

defaults: # overwrite defaults
stage: dev
region: us-east-1
variableSyntax: '\${{([\s\S]+?)}}' # change variable syntax to ${{foo}}

package:
# only the following paths will be included in the resulting artifact which will be uploaded. Without specific include everything in the current folder will be included
include:
Expand All @@ -41,8 +39,7 @@ package:

functions:
hello:
# Deployed Lambda name with a prefix
name: ${prefix}-lambdaName # You have to provide that variable in serverless.env.yml
name: ${env.prefix}-lambdaName # Deployed Lambda name with a prefix
handler: handler.hello # Uses the same configuration as your provider. Subdirectories are supported, depending on your language, e.g. subdir/handler.hello if handler.js is in subdir
memorySize: 512 # optional, default is 1024
timeout: 10 # optional, default is 6
Expand Down
27 changes: 4 additions & 23 deletions docs/understanding-serverless/services-and-functions.md
Expand Up @@ -4,21 +4,16 @@ A *Serverless service* is a group of one or multiple functions and any resources
functions together, it's easier to share code and resources between those functions. Services are also designed to
be completely independent, which helps teams develop more quickly without waiting for others.

Each *Serverless service* contains two configuration files:
Each *Serverless service* configuration is managed in the [`serverless.yml`](./serverless-yml.md) file. The main responsibilities of this file are:

### [`serverless.yml`](./serverless-yml.md)
- Declares a Serverless service
- Defines one or multiple functions in the service
- Defines the provider the service will be deployed to (and the runtime if provided)
- Defines custom plugins to be used
- Defines events that trigger each function to execute (e.g. HTTP requests)
- Defines one set of resources (e.g. 1 AWS CloudFormation stack) required by the functions in this service
- Events listed in the `events` section may automatically create the resources required for the event upon deployment

### [`serverless.env.yml`](./serverless-env-yml.md)
- Defines stages for this service
- Defines regions for each stage
- Defines Serverless variables
- Allow flexible configuration using Serverless Variables.

## Example

Expand All @@ -31,7 +26,6 @@ users
|__ lib // contains logic
|__ users.js // single handler file, requires lib
|__ serverless.yml
|__ serverless.env.yml
|__ node_modules
|__ package.json
```
Expand All @@ -43,21 +37,8 @@ service: users
provider:
name: aws
runtime: nodejs4.3
memorySize: ${memoryVar} # reference a Serverless variable
memorySize: ${env.memoryVar} # Serverless Variable referencing an env var
functions:
create:
handler: users.create
```

### [`serverless.env.yml`](./serverless-env-yml.md)

```yml
vars:
memoryVar: 512
stages:
dev:
vars:
regions:
us-east-1:
vars:
```
```

0 comments on commit e782df7

Please sign in to comment.