cdk deploy: JavaScript heap out of memory

[benjicarpy19]

Describe the bug

I am running into a "JavaScript heap out of memory" error when running cdk deploy in my CI. I am using a t3.large instance which has 8GiB memory as shown here.

I searched for this issue and it was mentioned that setting the NODE_OPTIONS=--max_old_space_size prior to running cdk deploy should fix the memory issue however when I amend the cdk deploy command to be like so:

NODE_OPTIONS=--max_old_space_size=6000 && cdk deploy

I still get the JavaScript heap out of memory issue even though I am limiting it to 6000 when my available memory should be 8000

Expected Behavior

I would expect with my defined node memory limit it should not run out of memory

Current Behavior

It currently runs out of memory while it appears to be creating the cloud formation template. Please see image below for reference in my CI

Reproduction Steps

I am unable to easily provide this i'm afraid, I am also not sure it is required to identify the issue. What I can provide is my general lambda configuration that causes the memory issue. I believe it is likely due to node_modules across 40 or so different lambdas creating large memory consumption. Unfortunately due to my setup it would be very difficult to migrate to using a lambda llayer to abstract the node_modules away from the lambda function asset bundles.

const nodeJsFunctionProps: NodejsFunctionProps = {
  environment: {
    AWS_NODEJS_CONNECTION_REUSE_ENABLED: '1',
    USER_POOL_ID: props.userPoolId,
    USER_POOL_CLIENT_ID: props.userPoolClientId,
    FRONTEND_URL: process.env.FRONTEND_URL as string,
    DATABASE_URL: process.env.DATABASE_URL as string,
    HUBSPOT_API_KEY: process.env.HUBSPOT_API_KEY as string,
    API_BASE_URL: domains[props.environment].apiDomain as string,
    STAGE: props.environment,
    UPLOAD_PRICE_LIST_BUCKET: priceListBucket.bucketName,
    UPLOAD_ORDER_PDF_BUCKET: orderPdfBucket.bucketName,
    SENTRY_DSN: process.env.SENTRY_DSN as string
  },
  vpc: props.vpc,
  role: props.lambdaRole,
  runtime: runTime,
  securityGroups: [props.defaultSecurityGroup],
  memorySize: 1024,
  timeout: Duration.seconds(900),
  bundling: {
    nodeModules: [
      '@middy/http-json-body-parser',
      '@middy/http-error-handler',
      '@middy/core',
      '@middy/http-cors',
      '@middy/http-header-normalizer',
      '@middy/util',
      '@aws-sdk/client-cognito-identity-provider',
      '@aws-sdk/client-secrets-manager',
      '@prisma/client',
      '@sentry/node',
      '@hubspot/api-client',
      'aws-jwt-verify',
      'aws-sdk',
      'prisma',
      'read-excel-file',
      'pdf-creator-node',
      'cookie',
      'dayjs',
      'nodemailer',
      'handlebars'
    ],
    commandHooks: {
      beforeBundling(_inputDir: string, _outputDir: string) {
        return [];
      },
      beforeInstall(inputDir: string, outputDir: string) {
        const commands = [
          `cp -R ${inputDir}/prisma ${outputDir}/`,
          `mkdir -p ${outputDir}/src/lambda/templates`,
          `cp -R ${inputDir}/src/lambda/templates/pdf ${outputDir}/src/lambda/templates`
        ];
        if (process.env.NODE_ENV !== 'CI') {
          commands.push(`cp -R ${inputDir}/.env ${outputDir}/`);
        }
        return commands;
      },
      afterBundling(_inputDir: string, outputDir: string) {
        return [
          `cd ${outputDir}`,
          `prisma generate`,
          `rm -rf node_modules/@prisma/engines`,
          `rm -rf node_modules/@prisma/client/node_modules node_modules/.bin node_modules/prisma`
        ];
      }
    }
  }
};

Possible Solution

I did try using the -no-asset-parallelism flag however this makes deployment painfully slow as no assest run in parallel at all.

Additional Information/Context

No response

CDK CLI Version

2.59.0

Framework Version

No response

Node.js Version

16.3.0

OS

Linux

Language

Typescript

Language Version

No response

Other information

No response

[benjicarpy19]

Note: upgrading CDK to 2.64.0 does not resolve the issue

[khushail]

Hi @benjicarpy19 , thanks for reaching out. You might want to try the suggested solution provided in here.

Similar to what has been asked in the linked issue, could you share the repro code and if that's not feasible, it would be great to answer about stacks and resources.

[benjicarpy19]

Hi,

As mentioned I have already tried the solution in the link you suggested and I am still seeing the issue persist. Here is the full construct that is likely causing te memory issue. As you can see there are many lambdas all with node_modules which I believe is what is likely cuasing the memory outage. Just to re-iterate setting NODE_OPTIONS=--max_old_space_size=6000 before running cdk deploy does not resolve the issue

import { Duration, RemovalPolicy } from 'aws-cdk-lib';
import { ContentHandling, LambdaIntegration } from 'aws-cdk-lib/aws-apigateway';
import { IRole } from 'aws-cdk-lib/aws-iam';
import { Vpc } from 'aws-cdk-lib/aws-ec2';
import * as s3 from 'aws-cdk-lib/aws-s3';
import * as lambdaEventSources from 'aws-cdk-lib/aws-lambda-event-sources';
import { Runtime } from 'aws-cdk-lib/aws-lambda';
import { RestApiBase } from 'aws-cdk-lib/aws-apigateway';
import {
  NodejsFunction,
  NodejsFunctionProps
} from 'aws-cdk-lib/aws-lambda-nodejs';
import { Construct } from 'constructs';
import domains from '../config/domains';
import { ISecurityGroup } from 'aws-cdk-lib/aws-ec2';
import {
  aws_events as events,
  aws_events_targets as targets
} from 'aws-cdk-lib';
import { setRemovalPolicy } from '../utils/cdk-funcs';

interface LambdasApiProps {
  environment: string;
  userPoolId: string;
  userPoolClientId: string;
  defaultSecurityGroup: ISecurityGroup;
  lambdaRole: IRole;
  api: RestApiBase;
  vpc: Vpc;
}

export default class LambdasApi extends Construct {
  constructor(scope: Construct, id: string, props: LambdasApiProps) {
    super(scope, id);

    const { api } = props;

    const runTime = Runtime.NODEJS_16_X;

    const eventRule = new events.Rule(this, 'LambdaSchedule', {
      schedule: events.Schedule.rate(Duration.minutes(1))
    });

    const priceListBucket = new s3.Bucket(this, 'OurBucket', {
      autoDeleteObjects: true,
      removalPolicy: RemovalPolicy.DESTROY
    });

    const orderPdfBucket = new s3.Bucket(this, 'OrderPDFBucket', {
      removalPolicy: setRemovalPolicy(props.environment)
    });

    const s3PutEventSource = new lambdaEventSources.S3EventSource(
      priceListBucket,
      {
        events: [s3.EventType.OBJECT_CREATED_PUT]
      }
    );

    const nodeJsFunctionProps: NodejsFunctionProps = {
      environment: {
        AWS_NODEJS_CONNECTION_REUSE_ENABLED: '1',
        USER_POOL_ID: props.userPoolId,
        USER_POOL_CLIENT_ID: props.userPoolClientId,
        FRONTEND_URL: process.env.FRONTEND_URL as string,
        DATABASE_URL: process.env.DATABASE_URL as string,
        HUBSPOT_API_KEY: process.env.HUBSPOT_API_KEY as string,
        API_BASE_URL: domains[props.environment].apiDomain as string,
        STAGE: props.environment,
        UPLOAD_PRICE_LIST_BUCKET: priceListBucket.bucketName,
        UPLOAD_ORDER_PDF_BUCKET: orderPdfBucket.bucketName,
        SENTRY_DSN: process.env.SENTRY_DSN as string
      },
      vpc: props.vpc,
      role: props.lambdaRole,
      runtime: runTime,
      securityGroups: [props.defaultSecurityGroup],
      memorySize: 1024,
      timeout: Duration.seconds(900),
      bundling: {
        nodeModules: [
          '@middy/http-json-body-parser',
          '@middy/http-error-handler',
          '@middy/core',
          '@middy/http-cors',
          '@middy/http-header-normalizer',
          '@middy/util',
          '@aws-sdk/client-cognito-identity-provider',
          '@aws-sdk/client-secrets-manager',
          '@prisma/client',
          '@sentry/node',
          '@hubspot/api-client',
          'aws-jwt-verify',
          'aws-sdk',
          'prisma',
          'read-excel-file',
          'pdf-creator-node',
          'cookie',
          'dayjs',
          'nodemailer',
          'handlebars'
        ],
        commandHooks: {
          beforeBundling(_inputDir: string, _outputDir: string) {
            return [];
          },
          beforeInstall(inputDir: string, outputDir: string) {
            const commands = [
              `cp -R ${inputDir}/prisma ${outputDir}/`,
              `mkdir -p ${outputDir}/src/lambda/templates`,
              `cp -R ${inputDir}/src/lambda/templates/pdf ${outputDir}/src/lambda/templates`
            ];
            if (process.env.NODE_ENV !== 'CI') {
              commands.push(`cp -R ${inputDir}/.env ${outputDir}/`);
            }
            return commands;
          },
          afterBundling(_inputDir: string, outputDir: string) {
            return [
              `cd ${outputDir}`,
              `prisma generate`,
              `rm -rf node_modules/@prisma/engines`,
              `rm -rf node_modules/@prisma/client/node_modules node_modules/.bin node_modules/prisma`
            ];
          }
        }
      }
    };

    // status
    const statusLambda = new LambdaIntegration(
      new NodejsFunction(this, 'statusFunction', {
        entry: 'dist/src/lambda/resolvers/status.js',
        functionName: 'status',
        ...nodeJsFunctionProps
      })
    );

    // admin
    const adminGetUserLambda = new LambdaIntegration(
      new NodejsFunction(this, 'adminGetUserFunction', {
        entry: 'dist/src/lambda/resolvers/admin/user/get-user.js',
        functionName: 'admin_user_get-user',
        ...nodeJsFunctionProps
      })
    );

    const adminGetProductsUploadLambda = new LambdaIntegration(
      new NodejsFunction(this, 'adminGetProductsUploadLambdaFunction', {
        entry:
          'dist/src/lambda/resolvers/admin/products-upload/get-products-upload.js',
        functionName: 'admin_products-upload_get-products-upload',
        ...nodeJsFunctionProps
      })
    );

    const adminPostProductsUploadExecuteLambda = new NodejsFunction(
      this,
      'adminPostProductsUploadExecuteLambdaFunction',
      {
        entry:
          'dist/src/lambda/resolvers/admin/products-upload/post-products-upload-execute.js',
        functionName: 'admin_products-upload_post-products-upload-execute',
        ...nodeJsFunctionProps
      }
    );

    const adminPostProductsUploadLambda = new LambdaIntegration(
      new NodejsFunction(this, 'adminPostProductsUploadLambdaFunction', {
        entry:
          'dist/src/lambda/resolvers/admin/products-upload/post-products-upload.js',
        functionName: 'admin_products-upload_post-products-upload',
        ...nodeJsFunctionProps
      })
    );

    adminPostProductsUploadExecuteLambda.addEventSource(s3PutEventSource);

    // Auth
    const loginLambda = new LambdaIntegration(
      new NodejsFunction(this, 'loginFunction', {
        entry: 'dist/src/lambda/resolvers/auth/login.js',
        functionName: 'auth_login',
        ...nodeJsFunctionProps
      })
    );
    const registerConfirmLambda = new LambdaIntegration(
      new NodejsFunction(this, 'registerConfirmFunction', {
        entry: 'dist/src/lambda/resolvers/auth/register-confirm.js',
        functionName: 'auth_register-confirm',
        ...nodeJsFunctionProps
      })
    );
    const forgotPasswordLambda = new LambdaIntegration(
      new NodejsFunction(this, 'forgotPasswordLambda', {
        entry: 'dist/src/lambda/resolvers/auth/forgot-password.js',
        functionName: 'auth_forgot-password',
        ...nodeJsFunctionProps
      })
    );
    const confirmForgotPasswordLambda = new LambdaIntegration(
      new NodejsFunction(this, 'confirmForgotPasswordLambda', {
        entry: 'dist/src/lambda/resolvers/auth/confirm-forgot-password.js',
        functionName: 'auth_confirm-forgot-password',
        ...nodeJsFunctionProps
      })
    );
    const resendConfirmationCodeLambda = new LambdaIntegration(
      new NodejsFunction(this, 'resendConfirmationCodeLambda', {
        entry: 'dist/src/lambda/resolvers/auth/resend-confirmation-code.js',
        functionName: 'auth_resend-confirmation-code',
        ...nodeJsFunctionProps
      })
    );
    const logoutLambda = new LambdaIntegration(
      new NodejsFunction(this, 'logoutLambda', {
        entry: 'dist/src/lambda/resolvers/auth/logout.js',
        functionName: 'auth_logout',
        ...nodeJsFunctionProps
      })
    );

    // order
    const createOrderLambda = new LambdaIntegration(
      new NodejsFunction(this, 'createOrderFunction', {
        entry: 'dist/src/lambda/resolvers/order/create-order.js',
        functionName: 'order_create-order',
        ...nodeJsFunctionProps
      })
    );
    const updateOrderLambda = new LambdaIntegration(
      new NodejsFunction(this, 'updateOrderFunction', {
        entry: 'dist/src/lambda/resolvers/order/update-order.js',
        functionName: 'order_update-order',
        ...nodeJsFunctionProps
      })
    );
    const deleteOrderLambda = new LambdaIntegration(
      new NodejsFunction(this, 'deleteOrderFunction', {
        entry: 'dist/src/lambda/resolvers/order/delete-order.js',
        functionName: 'order_delete-order',
        ...nodeJsFunctionProps
      })
    );
    const getOrderLambda = new LambdaIntegration(
      new NodejsFunction(this, 'getOrderFunction', {
        entry: 'dist/src/lambda/resolvers/order/get-order.js',
        functionName: 'order_get-order',
        ...nodeJsFunctionProps
      })
    );

    // wholesaler order
    const createWholesalerOrderLambda = new LambdaIntegration(
      new NodejsFunction(this, 'createWholesalerOrderFunction', {
        entry:
          'dist/src/lambda/resolvers/wholesaler-order/create-wholesaler-order.js',
        functionName: 'wholesaler-order_create-wholesaler-order',
        ...nodeJsFunctionProps
      })
    );
    const updateWholesalerOrderLambda = new LambdaIntegration(
      new NodejsFunction(this, 'updateWholesalerOrderFunction', {
        entry:
          'dist/src/lambda/resolvers/wholesaler-order/update-wholesaler-order.js',
        functionName: 'wholesaler-order_update-wholesaler-order',
        ...nodeJsFunctionProps
      })
    );

    // PDF order
    const getPdfWholesalerOrderLambda = new LambdaIntegration(
      new NodejsFunction(this, 'getPdfWholesalerOrderFunction', {
        entry:
          'dist/src/lambda/resolvers/wholesaler-order/pdf/get-wholesaler-order-pdf.js',
        functionName: 'wholesaler-order_pdf_get-wholesaler-order-pdf',
        ...nodeJsFunctionProps
      })
    );

    // orders
    const getOrdersLambda = new LambdaIntegration(
      new NodejsFunction(this, 'getOrdersFunction', {
        entry: 'dist/src/lambda/resolvers/orders/get-orders.js',
        functionName: 'orders_get-orders',
        ...nodeJsFunctionProps
      })
    );

    // product-filters
    const getProductFiltersLambda = new LambdaIntegration(
      new NodejsFunction(this, 'getProductFiltersFunction', {
        entry:
          'dist/src/lambda/resolvers/product-filters/get-product-filters.js',
        functionName: 'product-filters_get-product-filters',
        ...nodeJsFunctionProps
      })
    );

    // user
    const createMeLambda = new LambdaIntegration(
      new NodejsFunction(this, 'createMeFunction', {
        entry: 'dist/src/lambda/resolvers/user/me/create-me.js',
        functionName: 'user_me_create-me',
        ...nodeJsFunctionProps
      })
    );
    const getMeLambda = new LambdaIntegration(
      new NodejsFunction(this, 'getMeLambda', {
        entry: 'dist/src/lambda/resolvers/user/me/get-me.js',
        functionName: 'user_me_get-me',
        ...nodeJsFunctionProps
      })
    );
    const updateMeLambda = new LambdaIntegration(
      new NodejsFunction(this, 'updateMeLambda', {
        entry: 'dist/src/lambda/resolvers/user/me/update-me.js',
        functionName: 'user_me_update-me',
        ...nodeJsFunctionProps
      })
    );

    //venue
    const createVenueLambda = new LambdaIntegration(
      new NodejsFunction(this, 'createVenueLambda', {
        entry: 'dist/src/lambda/resolvers/venue/create-venue.js',
        functionName: 'venue_create-venue',
        ...nodeJsFunctionProps
      })
    );
    const updateVenueLambda = new LambdaIntegration(
      new NodejsFunction(this, 'updateVenueLambda', {
        entry: 'dist/src/lambda/resolvers/venue/update-venue.js',
        functionName: 'venue_update-venue',
        ...nodeJsFunctionProps
      })
    );
    const getVenueLambda = new LambdaIntegration(
      new NodejsFunction(this, 'getVenueLambda', {
        entry: 'dist/src/lambda/resolvers/venue/get-venue.js',
        functionName: 'venue_get-venue',
        ...nodeJsFunctionProps
      })
    );
    const deleteVenueLambda = new LambdaIntegration(
      new NodejsFunction(this, 'deleteVenueLambda', {
        entry: 'dist/src/lambda/resolvers/venue/delete-venue.js',
        functionName: 'venue_delete-venue',
        ...nodeJsFunctionProps
      })
    );
    const getVenuesLambda = new LambdaIntegration(
      new NodejsFunction(this, 'getVenuesLambda', {
        entry: 'dist/src/lambda/resolvers/venues/get-venues.js',
        functionName: 'venues_get-venues',
        ...nodeJsFunctionProps
      })
    );
    const getVenuesCountLambda = new LambdaIntegration(
      new NodejsFunction(this, 'getVenuesCount', {
        entry: 'dist/src/lambda/resolvers/venues/count/get-venues-count.js',
        functionName: 'venues_count_get-venues-count',
        ...nodeJsFunctionProps
      })
    );

    // wholesaler
    const createLinkedWholesalerLambda = new LambdaIntegration(
      new NodejsFunction(this, 'createLinkedWholesalerLambda', {
        entry:
          'dist/src/lambda/resolvers/wholesaler/linked/create-linked-wholesaler.js',
        functionName: 'wholesaler_linked_create-linked-wholesaler',
        ...nodeJsFunctionProps
      })
    );
    const updateLinkedWholesalerLambda = new LambdaIntegration(
      new NodejsFunction(this, 'updateLinkedWholesalerLambda', {
        entry:
          'dist/src/lambda/resolvers/wholesaler/linked/update-linked-wholesaler.js',
        functionName: 'wholesaler_linked_update-linked-wholesaler',
        ...nodeJsFunctionProps
      })
    );
    const deleteLinkedWholesalerLambda = new LambdaIntegration(
      new NodejsFunction(this, 'deleteLinkedWholesalerLambda', {
        entry:
          'dist/src/lambda/resolvers/wholesaler/linked/delete-linked-wholesaler.js',
        functionName: 'wholesaler_linked_delete-linked-wholesaler',
        ...nodeJsFunctionProps
      })
    );
    const getLinkedWholesalersLambda = new LambdaIntegration(
      new NodejsFunction(this, 'getLinkedWholesalersLambda', {
        entry:
          'dist/src/lambda/resolvers/wholesalers/linked/get-linked-wholesalers.js',
        functionName: 'wholesalers_linked_get-linked-wholesalers',
        ...nodeJsFunctionProps
      })
    );
    const getWholesalersLambda = new LambdaIntegration(
      new NodejsFunction(this, 'getWholesalersLambda', {
        entry: 'dist/src/lambda/resolvers/wholesalers/get-wholesalers.js',
        functionName: 'wholesalers_get-wholesalers',
        ...nodeJsFunctionProps
      })
    );

    // product
    const getProductsFunction = new NodejsFunction(this, 'getProductsLambda', {
      entry: 'dist/src/lambda/resolvers/products/get-products.js',
      functionName: 'products_get-products',
      ...nodeJsFunctionProps
    });
    const getProductsLambda = new LambdaIntegration(getProductsFunction);
    // below keeps the lambda warm
    eventRule.addTarget(new targets.LambdaFunction(getProductsFunction));

    // product savings
    const getProductsSavingsLambda = new LambdaIntegration(
      new NodejsFunction(this, 'getProductsSavingsLambda', {
        entry:
          'dist/src/lambda/resolvers/products/savings/get-products-savings.js',
        functionName: 'products_savings_get-products-savings',
        ...nodeJsFunctionProps
      })
    );
    const createProductSavingsLambda = new LambdaIntegration(
      new NodejsFunction(this, 'createProductSavingsLambda', {
        entry:
          'dist/src/lambda/resolvers/product/savings/create-product-savings.js',
        functionName: 'product_savings_create-product-savings',
        ...nodeJsFunctionProps
      })
    );
    const deleteProductSavingsLambda = new LambdaIntegration(
      new NodejsFunction(this, 'deleteProductSavingsLambda', {
        entry:
          'dist/src/lambda/resolvers/product/savings/delete-product-savings.js',
        functionName: 'product_savings_delete-product-savings',
        ...nodeJsFunctionProps
      })
    );
    const updateProductSavingsLambda = new LambdaIntegration(
      new NodejsFunction(this, 'updateProductSavingsLambda', {
        entry:
          'dist/src/lambda/resolvers/product/savings/update-product-savings.js',
        functionName: 'product_savings_update-product-savings',
        ...nodeJsFunctionProps
      })
    );

    const authResource = api.root.addResource('auth');

    const adminResource = api.root.addResource('admin');
    const adminUserResource = adminResource.addResource('user');
    const adminProductsUploadResource =
      adminResource.addResource('products-upload');
    const adminUserResourceWithId = adminUserResource.addResource('{id}');

    const loginResource = authResource.addResource('login');
    const registerConfirmResource =
      authResource.addResource('register-confirm');
    const forgotPasswordResource = authResource.addResource('forgot-password');
    const confirmForgotPasswordResource = authResource.addResource(
      'confirm-forgot-password'
    );
    const resendConfirmationCodeResource = authResource.addResource(
      'resend-confirmation-code'
    );
    const logoutResource = authResource.addResource('logout');

    const productFiltersResource = api.root.addResource('product-filters');

    const userResource = api.root.addResource('user');
    const meResource = userResource.addResource('me');

    const venueResource = api.root.addResource('venue');
    const venueResourceWithId = venueResource.addResource('{id}');

    const venuesResource = api.root.addResource('venues');
    const venuesResourceCount = venuesResource.addResource('count');

    const orderResource = api.root.addResource('order');
    const orderResourceWithId = orderResource.addResource('{id}');
    const wholesalerOrderResource =
      orderResourceWithId.addResource('wholesaler-order');

    const wholesalerOrderResourceWithId = wholesalerOrderResource.addResource(
      '{wholesalerOrderId}'
    );

    const wholesalerOrderPdfResource =
      wholesalerOrderResource.addResource('pdf');

    const ordersResource = api.root.addResource('orders');

    const wholesalerResource = api.root.addResource('wholesaler');
    const linkedWholesalerResource = wholesalerResource.addResource('linked');
    const linkedWholesalerResourceWithId =
      linkedWholesalerResource.addResource('{id}');
    const wholesalersResource = api.root.addResource('wholesalers');
    const linkedWholesalersResource = wholesalersResource.addResource('linked');

    const productsResource = api.root.addResource('products');
    const productsSavingsResource = productsResource.addResource('savings');
    const productResource = api.root.addResource('product');
    const productSavingsResource = productResource.addResource('savings');
    const productSavingsResourceWithId =
      productSavingsResource.addResource('{id}');

    api.root.addMethod('GET', statusLambda);
    adminUserResourceWithId.addMethod('GET', adminGetUserLambda);
    adminProductsUploadResource.addMethod('GET', adminGetProductsUploadLambda);
    adminProductsUploadResource.addMethod(
      'POST',
      adminPostProductsUploadLambda
    );

    if (process.env.STAGE === 'local') {
      const localAdminProductsUploadResource =
        adminProductsUploadResource.addResource('local');
      localAdminProductsUploadResource.addMethod(
        'POST',
        new LambdaIntegration(adminPostProductsUploadExecuteLambda)
      );
    }
    loginResource.addMethod('POST', loginLambda);
    registerConfirmResource.addMethod('POST', registerConfirmLambda);
    forgotPasswordResource.addMethod('POST', forgotPasswordLambda);
    confirmForgotPasswordResource.addMethod(
      'POST',
      confirmForgotPasswordLambda
    );
    resendConfirmationCodeResource.addMethod(
      'POST',
      resendConfirmationCodeLambda
    );
    logoutResource.addMethod('POST', logoutLambda);

    productFiltersResource.addMethod('GET', getProductFiltersLambda);

    meResource.addMethod('POST', createMeLambda);
    meResource.addMethod('GET', getMeLambda);
    meResource.addMethod('PUT', updateMeLambda);

    venueResource.addMethod('POST', createVenueLambda);
    venueResourceWithId.addMethod('GET', getVenueLambda);
    venueResourceWithId.addMethod('PUT', updateVenueLambda);
    venueResourceWithId.addMethod('DELETE', deleteVenueLambda);

    venuesResource.addMethod('GET', getVenuesLambda);
    venuesResourceCount.addMethod('GET', getVenuesCountLambda);

    orderResource.addMethod('POST', createOrderLambda);
    orderResourceWithId.addMethod('DELETE', deleteOrderLambda);
    orderResourceWithId.addMethod('PUT', updateOrderLambda);
    orderResourceWithId.addMethod('GET', getOrderLambda);

    wholesalerOrderResource.addMethod('POST', createWholesalerOrderLambda);
    wholesalerOrderResourceWithId.addMethod('PUT', updateWholesalerOrderLambda);

    wholesalerOrderPdfResource.addMethod('GET', getPdfWholesalerOrderLambda);

    ordersResource.addMethod('GET', getOrdersLambda);

    linkedWholesalerResource.addMethod('POST', createLinkedWholesalerLambda);
    linkedWholesalerResourceWithId.addMethod(
      'PUT',
      updateLinkedWholesalerLambda
    );
    linkedWholesalerResourceWithId.addMethod(
      'DELETE',
      deleteLinkedWholesalerLambda
    );

    linkedWholesalersResource.addMethod('GET', getLinkedWholesalersLambda);
    wholesalersResource.addMethod('GET', getWholesalersLambda);

    productsResource.addMethod('GET', getProductsLambda);

    productsSavingsResource.addMethod('GET', getProductsSavingsLambda);
    productSavingsResource.addMethod('POST', createProductSavingsLambda);
    productSavingsResourceWithId.addMethod(
      'DELETE',
      deleteProductSavingsLambda
    );
    productSavingsResourceWithId.addMethod('PUT', updateProductSavingsLambda);
  }
}

[RomainMuller]

Is NODE_OPTIONS=--max_old_space_size=6000 && cdk deploy the actual command you ran? If so, you actually haven't changed the max old space size (your screenshot suggests the Node VM hit heap limit at 2G, which is the default, and would be consistent with this observation), due to the presence of the && there... I'm actually surprised this didn't err out (but this is beyond the point)...

You could try:

$ NODE_OPTIONS=--max_old_space_size=6000 cdk deploy

or

$ export NODE_OPTIONS=--max_old_space_size=6000
$ cdk deploy

[benjicarpy19]

Oh interesting, you could well be right. Yes that is the actual command I was running. Is there a way to log overall memory usage after the cdk deploy process?

[RomainMuller]

No I don't think there is such a provision in place today.

Your app could actually just periodically dump JS heap usage information out to STDERR or to a file, using a combination of setInterval and node:v8.getHeapSpaceStatistics(). You might want to call .unref() on the object returned by setInterval so that it does not prevent the Node VM from exiting once other work is done.

[benjicarpy19]

Thanks, that could be worth running alongside the deploy operation. I am curious though as to why it is/ was timing out. If the default max_old_space_size is 2G as you mentioned and my runner has 8G of memory, how was it running out of memory? Surely it should have never surpassed 2G if the max_old_space_size was not set properly and therefore not run out of memory as it would have never surpsassed 8G?