Serverles-lift fails to create queues, if stage name is passed from a custom variable

[Abhijith-Nagaraja]

Serverles-lift fails to create queues, if stage name is passed from a custom variable

The following configuration won't work

service: demo

frameworkVersion: '2'

plugins:
  - serverless-lift

custom: ${file(../config/config.yml)}

provider:
  name: aws
  runtime: python3.8
  lambdaHashingVersion: 20201221

  stage: ${self:custom.stage}

  constructs:
    my-queue:
      type: queue
      fifo: true
      worker:
        handler: lambdas/lambda_name.lambda_handler

This fails with exception

Serverless Error ----------------------------------------
 
  ServerlessError: An error occurred: my-queueDlqF097231D - The name of a FIFO queue can only include alphanumeric characters, hyphens, or underscores, must end with .fifo suffix and be 1 to 80 in length. (Service: AmazonSQS; Status Code: 400; Error Code: InvalidParameterValue; Request ID: 472ff810-cd4d-5bdf-acae-a1f36a3ee1fb; Proxy: null).
      at /opt/demo/serverless-script/node_modules/serverless/lib/plugins/aws/lib/monitorStack.js:136:23
      at processTicksAndRejections (internal/process/task_queues.js:97:5)

I tried to find the reason by adding the console.log at this place in Queue.ts

// The default function timeout is 6 seconds in the Serverless Framework
        const functionTimeout = configuration.worker.timeout ?? 6;

        const maxRetries = configuration.maxRetries ?? 3;

        const baseName = `${this.provider.stackName}-${id}`;

        console.log("baseName: " + baseName);

        const dlq = new CdkQueue(this, "Dlq", {
            queueName: configuration.fifo === true ? `${baseName}-dlq.fifo` : `${baseName}-dlq`,
            // 14 days is the maximum, we want to keep these messages for as long as possible
            retentionPeriod: Duration.days(14),
            fifo: configuration.fifo,
        });

The output indicated that custom variable did not get resolved and hence the error

basename: demo-${self:custom.stage}-my-queue

However if I change the stage variable to

stage: test

it works perfectly.

[t-richard]

Hi @Abhijith-Nagaraja

I was able to reproduce with a setup similar to yours.

The error you have is linked to your config.yml file being outside of your project directory (referenced with ..).

You should even see that serverless gives you a warning about that

Variables resolver reports following resolution errors:
- Cannot resolve variable at "custom": Cannot load file from outside of a project directory (configure "projectDir" to extend project boundary)
From a next major this will be communicated with a thrown error.

Moving the config file inside the project directory or setting projectDir: .. should solve the issue.

Let me know if that works for you.

[Abhijith-Nagaraja]

Hi @t-richard,

Thanks for the quick reply. Yes you are correct. Once I moved it inside project directory, it worked.

However I still feel this is a bug. Here is my reasoning.

Configs files can be anywhere, we should be able to refer to it as long we have system file access for the user. I don't experience this issue w.r.t my own plugins or some other plugin I use. I guess serverless-lift is restricting it to project directory.

Saying that, for me it make sense to keep configs at one level higher. As I have multiple small projects which I combine in one repo. Here is my structure

root
|
- Project 1
-- serverless.yml
- Project 2
-- serverless.yml
...
- Config
-- Project 1
--- dev.yml
--- test.yml
--- prod.yml
-- Project 2
--- dev.yml
--- test.yml
--- prod.yml
....

So in my case, although it's doable to bring configs under each project, that just not preferred

If you don't feel this is a bug, please feel free to close this issue.

[t-richard]

This warning is not coming from Lift but from the Serverless framework itself.

See https://github.com/serverless/serverless/blob/master/lib/configuration/variables/sources/file.js#L51-L57

The docs states the following

Files need to be referenced by relative paths, which should not reach out beyond project directory (by default service directory). If you work with multi-service project, you can change project directory boundary with projectDir setting (e.g. set projectDir: ../ if you're service is nested in top level service-x directory)

Not sure why you don't encounter this in your other projects though. Maybe you are using an older version of the framework in those projects ? Or you defined the projectDir configuration option ?

In any way, we can't really do something about it in Lift.