invoke local does not work with variable substitution in provider

[Omicron7]

This is a Bug Report

Description

When running sls invoke local with variable substitution ${} in the provider configuration, nothing happens and the command exits.

I expected the invoked function to be executed, or if the function could not be found, an error message describing that the function does not exist in this service.

Example using the hello-world template. (serverless create --template hello-world).

service: serverless-hello-world
provider:
  name: aws
  runtime: nodejs6.10
functions:
  helloWorld:
    handler: handler.helloWorld
    events:
      - http:
          path: hello-world
          method: get
          cors: true

Invoke local functions properly:

bash> serverless invoke local --function helloWorld
{
    "statusCode": 200,
    "headers": {
        "Access-Control-Allow-Origin": "*"
    },
    "body": "{\"message\":\"Go Serverless v1.0! Your function executed successfully!\",\"input\":\"\"}"
}

Changing serverless.yml to include variable substitution in the provider:

service: serverless-hello-world
custom:
  provider:
    name: aws
    runtime: nodejs6.10
# The `provider` block is referenced from custom using variable substitution
provider: ${self:custom.provider}
functions:
  helloWorld:
    handler: handler.helloWorld
    events:
      - http:
          path: hello-world
          method: get
          cors: true

results in invoke local doing nothing.

bash> serverless invoke local --function helloWorld

It produces no output or error. Running with SLS_DEBUG=* enabled shows that commands are loaded, invoke:local is executed, but nothing happens. Providing an unknown function name results in the same response, no message or error.

This is just an example that produces the results I'm running into. I'm using similar to change the provider configuration depending on wether serverless is being run in development or on our CI server.

• Serverless Framework Version you're using: 1.26.1
• Operating System: OSX 10.13.3, node v9.3.0, npm 3.10.10
• Stack Trace:
• Provider Error messages:

[MacMcIrish]

Hey, this might be related to an issue I opened earlier today. #4814
What we're doing in the mean time is using

.provider: &provider
  ${self:custom.provider}
provider:
  name: aws
  <<: *provider

Having at least something in the provider and then backfilling the rest of the data makes the script continue.

[Omicron7]

@MacMcIrish This is probably a duplicate of #4814
Like you, I was actually using the ${file()} variable substitution, just simplified it to ${self} for the bug report.

What does serverless print show for you with that syntax. I'm not seeing the yml loaded from an external file getting included in the provider. I believe the YAML merge <<: happens before the variable substitution, so it might just be merging an empty object into provider.

[MacMcIrish]

Running sls print shows me this:

.provider:
  stage: dev
  region: us-west-2
provider:
  name: aws
  runtime: nodejs6.10

But, the deploy and packaging still work even if though the print doesn't have it backfilled

[Omicron7]

I agree, deploy, package and invoke all function with this configuration, as opposed to doing nothing. But the values they are using don't reflect what should be merged from the $file() variable.

I've got additional VPC, bucket name, and tags in the provider configuration that should be used, but are not reflected in the package (.serverless/serverless-state.json) nor in what is actually deployed.

I'd check and make sure your region is indeed set to us-west-2. (Note: --region command line argument will override yaml setting).

[MacMcIrish]

@Omicron7 You're entirely right... That's unfortunate. Thanks for the heads up!

[Omicron7]

I've written a plugin that does post variable substitution merge.
serverless-merge-config

serverless.yml

service: service-name
provider:
  name: aws
  $<<: ${file(path/to/other.yml):provider}

path/to/other.yml

provider:
  runtime: nodejs6.10
  vpc:
    ...

Will merge the provider property from the file path/to/other.yml into the provider in serverless.yml resulting in.

service: service-name
provider:
  name: aws
  runtime: nodejs6.10
  vpc:
    ...

[simlu]

@Omicron7 Doesn't seem to resolve the issue unfortunately. The variables are still resolved incorrectly. Will post details shortly.

[simlu]

@Omicron7 So the problem is that the provider variables are still not picket up, e.g. the region as mentioned in #4814.

[simlu]

@Omicron7 Considering how broken variable resolution + file inclusion currently is in serverless, we're considering implementing our own loading logic - possibly using serverless.js syntax. I'll do a quick draft and hopefully get back here shortly.

[simlu]

We now do the following
(1) rename serverless.yml to _serverless.yml
(2) create serverless.js with content

const path = require("path");
const optimist = require('optimist');
const loadYaml = require("lambda-serverless-api")().loadYaml;

module.exports = loadYaml(path.join(__dirname, "_serverless.yml"), optimist.argv);

Extremely hacky, but at least you know what you get when using file references.

Yaml loading is defined here.

[simlu]

I've now separated this logic into yaml-boost

[medikoo]

Fixed with #8987