AWS Lambda Image Function: Error: ENOENT: no such file or directory, mkdir '/home/sbx_user1051/.serverless'

[ariel-frischer]

I'm trying to deploy my full nodejs/express backend application using the serverless framework. Doing any normal serverless deploy does not work because it exceeds the 200Mb limit. So I've wrapped the application with a docker image and I've deployed a lambda function using this image. I've followed this article as a loose guide: https://espressocoder.com/2021/01/05/creating-a-serverless-docker-image/ with the goal of using the serverless offline command to start this lambda function.

This does not seem like a plugin issue, locally the plugins don't cause any errors so my serverless.yml is as follows.

service: initiate-backend
app: initiate-web
org: arielf

frameworkVersion: '2'
useDotenv: true

plugins:
  - '@kingdarboja/serverless-plugin-typescript'
  - serverless-offline

custom:
  serverless-offline:
    useDocker: true
    dockerReadOnly: false

provider:
  name: aws
  runtime: nodejs12.x
  lambdaHashingVersion: 20201221
  stage: dev
  region: us-west-1
  environment:
    AWS_NODEJS_CONNECTION_REUSE_ENABLED: 1


functions:
  app:
    handler: ./server/server.handler
    events:
      - http: ANY /
      - http: 'ANY /{proxy+}'

yarn serverless offline output

command deploy
Serverless: Load command deploy:function
Serverless: Load command deploy:list
Serverless: Load command deploy:list:functions
Serverless: Load command invoke
Serverless: Load command invoke:local
Serverless: Load command info
Serverless: Load command logs
Serverless: Load command metrics
Serverless: Load command print
Serverless: Load command remove
Serverless: Load command rollback
Serverless: Load command rollback:function
Serverless: Load command slstats
Serverless: Load command plugin
Serverless: Load command plugin
Serverless: Load command plugin:install
Serverless: Load command plugin
Serverless: Load command plugin:uninstall
Serverless: Load command plugin
Serverless: Load command plugin:list
Serverless: Load command plugin
Serverless: Load command plugin:search
Serverless: Load command config
Serverless: Load command config:credentials
Serverless: Load command rollback
Serverless: Load command rollback:function
Serverless: Load command upgrade
Serverless: Load command uninstall
Serverless: Load command offline
Serverless: Load command offline:start
Serverless: Load command login
Serverless: Load command logout
Serverless: Load command generate-event
Serverless: Load command test
Serverless: Load command dashboard
Serverless: Load command output
Serverless: Load command output:get
Serverless: Load command output:list
Serverless: Load command param
Serverless: Load command param:get
Serverless: Load command param:list
Serverless: Load command studio
Serverless: Error: EROFS: read-only file system, open '/home/sbx_user1051/.serverlessrc.1568800362'
    at Object.openSync (fs.js:462:3)
    at Function.writeFileSync [as sync] (/backend/node_modules/write-file-atomic/index.js:205:13)
    at storeConfig (/backend/node_modules/@serverless/utils/config.js:50:21)
    at createDefaultGlobalConfig (/backend/node_modules/@serverless/utils/config.js:72:3)
    at getGlobalConfig (/backend/node_modules/@serverless/utils/config.js:129:10)
    at getConfig (/backend/node_modules/@serverless/utils/config.js:134:24)
    at Object.getLoggedInUser (/backend/node_modules/@serverless/utils/config.js:188:18)
    at module.exports (/backend/node_modules/@serverless/enterprise-plugin/lib/isAuthenticated.js:5:44)
    at ServerlessEnterprisePlugin.asyncInit (/backend/node_modules/@serverless/enterprise-plugin/lib/plugin.js:541:7)
    at /backend/node_modules/serverless/lib/classes/PluginManager.js:648:82
    at Array.map (<anonymous>)
    at PluginManager.asyncPluginInit (/backend/node_modules/serverless/lib/classes/PluginManager.js:648:39)
    at PluginManager.loadAllPlugins (/backend/node_modules/serverless/lib/classes/PluginManager.js:125:17)
    at Serverless.init (/backend/node_modules/serverless/lib/Serverless.js:166:30)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
    at /backend/node_modules/serverless/scripts/serverless.js:73:7
Serverless: Unable to store serverless config: /home/sbx_user1051/.serverlessrc due to EROFS error
 
  Error --------------------------------------------------
 
  Error: ENOENT: no such file or directory, mkdir '/home/sbx_user1051/.serverless'
 
  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Issues:        forum.serverless.com
 
  Your Environment Information ---------------------------
     Operating System:          linux
     Node Version:              12.20.2
     Framework Version:         2.25.2 (local)
     Plugin Version:            4.4.3
     SDK Version:               2.3.2
     Components Version:        3.7.0
 
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
END RequestId: bf435cce-a6ed-4f1f-a7f3-a285779f6404
REPORT RequestId: bf435cce-a6ed-4f1f-a7f3-a285779f6404	Duration: 14190.71 ms	Billed Duration: 14191 ms	Memory Size: 5000 MB	Max Memory Used: 230 MB	
RequestId: bf435cce-a6ed-4f1f-a7f3-a285779f6404 Error: Runtime exited with error: exit status 1
Runtime.ExitError

Installed version

Framework Core: 2.25.2 (local)
Plugin: 4.4.3
SDK: 2.3.2
Components: 3.7.0

[pgrzesik]

Hello @ariel-frischer, thanks for reporting. In the case you're listing above it seems like there's a problem with using a read-only filesystem and .serverlessrc file with config cannot be created. It looks like the whole command is executed in the context of a read-only container - how are you executing the yarn serverless offline command? Is it via docker somehow?

[ariel-frischer]

Yes using docker my entrypoint /usr/bin/dumb-init using cmd: yarn serverless offline I need to figure out how to allow lambda to use a writeable container I guess.

[pgrzesik]

@ariel-frischer What base image are you using when running yarn serverless offline?

[ariel-frischer]

jrottenberg/ffmpeg:4.3.1-ubuntu1804
https://hub.docker.com/r/jrottenberg/ffmpeg
https://github.com/jrottenberg/ffmpeg/blob/master/docker-images/4.3/ubuntu1804/Dockerfile

[ariel-frischer]

If I could set the writable directory for serverless-offline to /tmp that might work also.

[pgrzesik]

Are you setting the container to be nonwritable somehow locally? I'm afraid it's not a problem with the Framework itself but rather with your local setup or with serverless-offline plugin, but it's hard to diagnose without more information. As for the image you provided - are you using it as a base for an image that you use to run yarn serverless offline?

[ariel-frischer]

Yes that is the base image where I run the command. What other files do you need me to provide? The dockerfile and handler? I can do that but I'm not sure if that will help.

[pgrzesik]

Thanks @ariel-frischer - I'm honestly a bit puzzled here because the error suggests that the command is running in one of lambci/lambda images, not in one based on jrottenberg/ffmpeg:4.3.1-ubuntu1804 image. Do you have the option to run serverless commands outside of docker container?

[ariel-frischer]

Yea I've tried it coming across multiple different bugs when I try it that way as well. I've found a useful docker image to try to emulate here: adieuadieu/serverless-chrome#282 (comment) . I guess you can close the issue for now it seems to be more of an aws issue.

[pgrzesik]

What other bugs did you run into when running outside of docker container?

As you suggested, I'm going to close this issue, but feel free to report back if you feel there's a bug that should be addressed on Framework side.