Which is best folder/file structure to organize multiple functions of service in v1.0

[nirmalgoswami]

This is related to the serverlessv1.0 . I am opening this for discussion only.

How we can organize multiple function in proper manner

1. folder structure of multiple functions.
   2, serverless.yml sample
2. depedency of node_modules for particular function,some node module is only require for specific function.
3. Optimize size of function to load only require files/folder

Which is best approach to do above things ?

[horike37]

I made src directory in my Serverless project root, and add multiple functions to it as follow.
also node_modules add to it.

├── serverless.yml
└── src
     ├── function1.js
     ├── function2.js
     ├── function3.js
     └── node_modules

serverless.yml setting as follow:

functions:
  function1:
    handler: src/function1.handler
    events:
      - http:
          path: function1
          method: get
      - http:
          path: function1
          method: post
  function2:
    handler: src/function2.handler
    events:
      - http:
          path: function2
          method: get
      - http:
          path: function2
          method: post
  function3:
    handler: src/function3.handler
    events:
      - http:
          path: function3
          method: get
      - http:
          path: function3
          method: post

[nirmalgoswami]

@horike37 thanks for reply,how you are dealing with include/exclude with function because some node module only require for specific function,we cant upload everything with single function zip,if you will check your function folder after download from aws lambda console you will see some unnecessary file in zip like node module or other function's file

[horike37]

@nirmalgoswami

I see. How about bellow?

Serverless project tree

├── serverless.yml
├── function1
│      ├── node_modules
│      └── index.js
└── function2
       ├── node_modules
       └── index.js

serverless.yml setting

package:
   individually: true
functions:
  function1:
    handler: function1/index.handler
    package:
      exclude:
        - function2/**
  function2:
    handler:  function2/index.handler
    package:
      exclude: 
        - function1/**

https://serverless.com/framework/docs/guide/packaging/

But Every time you add a function, adding exclude file/folder the statements in the exclude.
I think a best way to be implemented in serverless as follow

For example

• 1. Add include files/folder the statements in the include
• 2. Deploy only added it.

functions:
  function1:
    handler: function1/index.handler
    package:
      include:
        - function1/**
        - node_modules/RequireModule1
        - node_modules/RequireModule2
  function2:
    handler:  function2/index.handler
    package:
      include: 
        - function2/**
        - node_modules/RequireModule3

[nirmalgoswami]

But Every time you add a function, adding exclude file/folder the statements in the exclude.

yes this was main issue in serverless 0.5

├── serverless.yml
├── function1
│ ├── node_modules
│ └── index.js
└── function2
├── node_modules
└── index.js

this is little problematic because it is not good to make node_modules in each folder.we can make common node_modules folder and add specific module in include

Your approach related include seems perfect 💯

[horike37]

@pmuens @flomotlik
What do you think to add include statement in each functions in serverless.yml?
I think that is good.

[rowanu]

I think having a per-function directory (each with it's own node_modules directory) is the way to go. While there may be some duplication between modules in each node_modules directory, disk is cheap 😄

[andrewoh531]

@horike37 do you know how your suggestion works with plugins? I'm trying to structure my project like:

├── serverless.yml
└── src
     ├── function1.js
     ├── function2.js
     ├── function3.js
     └── node_modules
     └── package.json

and within my serverless.yml I'm trying to use serverless-webpack and have defined:

plugins:
  - serverless-webpack

but am getting the error: Cannot find module 'serverless-webpack'

[goncaloneves]

Running into the same scenario that I want to use include inside each function to avoid bloat in the artifact.
I agree about one parent package.json and choosing modules per function package include/exclude.

[acorbel]

with

functions:
  function1:
    handler: function1/index.handler
    package:
      include:
        - function1/**
        - node_modules/RequireModule1
        - node_modules/RequireModule2
  function2:
    handler:  function2/index.handler
    package:
      include: 
        - function2/**
        - node_modules/RequireModule3

Do you strictly use npm2 or is there a way to use npm3 (which has a flat structure)?

[ecstasy2]

+1 on adding include to cherry pick which modules to include in the package.
One potential problem is that some function could use a given package that is not referenced in the include and all test would pass but the function would fail once packaged and deployed.

[danawoodman]

I'm doing the following and I like it thus far:

├── serverless.yml
└── functions
     ├── handler.js
     └── users
          ├── create.js
          ├── delete.js
          ├── read.js
          └── update.js
     └── emails
          ├── forgot-password.js
          └── welcome.js  

[jamiechong]

Checkout this plugin https://github.com/doapp-ryanp/serverless-plugin-browserify which handles efficient bundling for you. My folder structure is like this:

- serverless.yml
- package.json
- functions
    - function1.js
    - function2.js
- node_modules
- includes
    - util.js

In each functionsX.js require the modules you need. In serverless.yml just point the handler to appropriate file like so (and don't worry about includes or excludes):

functions:
  function1:
    handler: functions/function1.handler
  function2:
    handler: functions/function2.handler

The browserfy plugin will create the optimal package using only the dependencies required.

[pmuens]

Closing this issue since we have some great examples already.

Feel free to continue the discussion.

Another good place to find some inspiration is the official Serverless Examples repository: https://github.com/serverless/examples

[DmitryEfimenko]

I didn't really see in the examples from the link above any use of node_modules at all. Some official guidance regarding this topic would be great.

[pmuens]

Hey @DmitryEfimenko thanks for the response.

You might also want to look at this list of real world projects for inspiration: https://github.com/JustServerless/awesome-serverless/blob/master/README.md#projects--services

[btburton42]

@pmuens I'm using serverless-webpack and I haven't seen any project in the examples that optimizes for each function to only contain the modules it needs and no more. I'm fine with either a central node_modules folder or one per function for deps and using the root level folder for dev-deps. Is there an example that really solves for the cases listed above that you (or anyone) can point to?

[HyperBrain]

@btburton42 serverless-webpack v3 will support individual packaging, i.e. you enable package individually on your project (just keep the central node_modules) and the plugin will fully take care of the packaging. It will just add exactly the needed dependencies to your functions. A preversion is already available in the plugin's v3.0.0 branch.
In general, Serverless (the framework) cannot know which external node modules are needed as it does not do any code analysis - but Webpack does.

[btburton42]

@HyperBrain thanks for the quick reply. I'll give v3 a spin tomorrow and post any errors to your PR thread here serverless-heaven/serverless-webpack#175

[alexcroox]

@horike37 I like your second example, except when deployed the handler is hello-world/index.handler and that handler path is then wrong when deployed in an isolated remote single lambda function

"errorMessage": "Cannot find module '/var/task/hello-world/index'",

It would be expecting just index.handler

[paulalex]

I'm doing the following and I like it thus far:

├── serverless.yml
└── functions
     ├── handler.js
     └── users
          ├── create.js
          ├── delete.js
          ├── read.js
          └── update.js
     └── emails
          ├── forgot-password.js
          └── welcome.js  

A question on this, would this structure mean your services are more prone to cold starts? If all of your CRUD operations were in a single file I was told this would mean the container will have more chance of staying up meaning a fast start time.