Migrate Infrastructure from Serverless Framework to AWS CDK by sannidhishukla · Pull Request #68 · newjersey/feedback-api

sannidhishukla · 2025-06-10T22:21:04Z

Overview

This PR contains a number of changes related to migrating the Feedback Widget's AWS infrastructure from Serverless Framework to AWS CDK and moving it into our new Platform-specific AWS accounts

Building out the resources using AWS
Updating calls to the SSM parameter store to happen within the code instead of during resource creation so that plaintext values are not stuck in the CloudFormation template
Removing the summary endpoint which would return an AI-generated summary of Feedback Widget comments. This function has not been used and its functionality can be replaced with using the NJ AI Assistant
Upgrading the project from Node v18 to Node v22
Greatly simplifying the file structure
Update the README to be more up-to-date

For more in depth documentation on the changes and resources migrated over, see this document

Next Steps

Create a GitHub action that handles deployment to the new AWS accounts
Add more tests - many parts of this repository are completely untested. We will work on adding more tests as we make the switch over to our new RDS database
Potentially setting up a connection to a dev Google Sheet to use in the interim while the databases are being set up. Not sure if this is strictly necessary at this point, though.

sannidhishukla · 2025-06-10T22:22:00Z

README.md

-### Project structure
-
-The project code base is mainly located within the `src` folder. This folder is divided in:
-
- `functions` - containing code base and configuration for your lambda functions
- `libs` - containing shared code base between your lambdas
-
-```
-.
-├── src
-│   ├── functions               # Lambda configuration and source code folder
-│   │   ├── rating
-│   │   │   ├── handler.ts      # `rating` lambda source code
-│   │   │   ├── index.ts        # `rating` lambda Serverless configuration
-│   │   │   ├── mock.json       # `rating` lambda input parameter, if any, for local invocation
-│   │   │   └── schema.ts       # `rating` lambda input event JSON-Schema
-│   │   │
-│   │   └── index.ts            # Import/export of all lambda configurations
-│   │
-│   └── libs                    # Lambda shared code
-│       └── apiGateway.ts       # API Gateway specific helpers
-│       └── handlerResolver.ts  # Sharable library for resolving lambda handlers
-│       └── lambda.ts           # Lambda middleware
-│
-├── package.json
-├── serverless.ts               # Serverless service file
-├── tsconfig.json               # Typescript compiler configuration
-├── tsconfig.paths.json         # Typescript paths
-└── webpack.config.js           # Webpack configuration
-```
-
-This project also contains a `scripts` folder, which contains post-processing Python scripts that help analyze the feedback data.


removing this for now. it will be hard to keep up-to-date. we can replace it if needed once all of the major changes forecasted for the feedback widget are complete.

sannidhishukla · 2025-06-10T22:23:29Z

src/utils/google-sheets.ts

-export async function getLastNComments(
-  sheetsClient: sheets_v4.Sheets,
-  n: number,
-  pageURL: string,
-  tabInfo
-): Promise<string[][]> {
-  const { tabName, totalRowsRange, columnMap, isDefault } = tabInfo;
-  try {
-    const totalRows = await getTotalRows(sheetsClient, totalRowsRange);
-    if (totalRows < 2) return [];
-    let accumulatedComments = [];
-    let commentBatchEnd = totalRows;
-    const rightmostColumn = Object.keys(columnMap).reduce((a, b) =>
-      columnMap[a].index > columnMap[b].index ? a : b
-    );
-    while (accumulatedComments.length < n && commentBatchEnd > 1) {
-      const commentBatchStart = Math.max(commentBatchEnd - n + 1, 2);
-      const result = await sheetsClient.spreadsheets.values.get({
-        spreadsheetId: process.env.SHEET_ID,
-        range: `${tabName}!A${commentBatchStart}:${columnMap[rightmostColumn].column}${commentBatchEnd}`
-      });
-      let commentBatch = result.data.values ?? [];
-      if (commentBatch.length > 0) {
-        if (isDefault) {
-          commentBatch = commentBatch.filter(
-            (v) =>
-              v[columnMap.pageUrl.index].includes(pageURL) &&
-              v[columnMap.comment.index] !== undefined
-          );
-        } else {
-          commentBatch = commentBatch.filter(
-            (v) => !!v[columnMap.comment.index]
-          );
-        }
-        accumulatedComments = commentBatch.concat(accumulatedComments);
-      }
-      commentBatchEnd = commentBatchStart - 1;
-    }
-    return accumulatedComments;
-  } catch (e) {
-    throw Error(`Google Sheets API failed to get input data: ${e.message}`);
-  }
-}


removed as this was only used by the deprecated /summary endpoint

sannidhishukla · 2025-06-10T22:23:36Z

src/utils/google-sheets.ts

-// only used within getLastNComments function
-async function getTotalRows(sheetsClient: sheets_v4.Sheets, totalRowRange) {
-  try {
-    const result = await sheetsClient.spreadsheets.values.get({
-      spreadsheetId: process.env.SHEET_ID,
-      range: totalRowRange
-    });
-    return parseInt(result.data.values[0][0]);
-  } catch (e) {
-    throw Error(`Google Sheets API failed to get data size: ${e.message}`);
-  }
-}


removed as this was only used by the deprecated /summary endpoint

sannidhishukla · 2025-06-10T22:30:20Z

src/utils/awsUtils.test.ts

+// eslint-disable-next-line @typescript-eslint/no-var-requires
+const { getSsmParam } = require('./awsUtils');


not sure why but the next line breaks everything if it's switched from require to import

package.json

casewalker

I just wanted to do a quick CDK check, things look pretty great! Left some comments, but great work regardless!

AnJuHyppolite · 2025-06-13T15:03:50Z

src/functions/comment.ts

+  const { feedbackId, comment, pageURL, rating } = JSON.parse(
+    event.body
+  ) as Comment;
+


[boulder] Since comment is a required property, we should validate it before entering the try block by adding a guard clause that returns early if it's null or undefined. What do you think?

just added this!

src/functions/email.ts

AnJuHyppolite · 2025-06-13T15:28:50Z

src/functions/rating.ts

+  event: APIGatewayProxyEvent
+): Promise<FeedbackResponse> => {
+  const { pageURL, rating } = JSON.parse(event.body) as Rating;
+


[boulder] Since pageURL and rating are required properties, we should validate them before entering the try block by adding a guard clause that returns early if they're null or undefined. What do you think?

AnJuHyppolite

Left a few comments. Great work, @sannidhishukla!

README.md

ezhangy

awesome job on this refactor! left some dusts, mostly on the README to clarify how to deploy the API and test it locally

README.md

src/shared/utils/awsUtils.test.ts

ezhangy · 2025-06-16T21:03:55Z

README.md

+2. Run `npm install` (on Node 22, as listed in `.nvmrc`) to install Node dependencies
+3. Save the credentials from the `Innov-Platform-dev` AWS account to your `~/.aws/credentials` file
+4. Run the API locally
+5. In another terminal window, you can test sending commands like the following:


[dust] is it possible to add instructions on how to query the dev deployment (in Innov-Platform-dev) instead of from localhost:3000? i feel like being able to do so will be an important manual testing step before promoting to prod while we're still missing test coverage

yes! i added instructions on sending requests through the lambda console to test (which is what i've been doing)

tsconfig.json

…DK (#68)" This reverts commit 8c240f8.

Merges the following PRs to main: - #68 - #70 - #69 - #72 - #71 - #73 - #75 - #77

sannidhishukla added 3 commits June 6, 2025 10:32

start migrating stack + rating function

f91b622

progress

a809913

updates

3c497f6

sannidhishukla commented Jun 10, 2025

View reviewed changes

add newline to end

1be623d

sannidhishukla commented Jun 10, 2025

View reviewed changes

sannidhishukla added 5 commits June 10, 2025 16:24

add newline to end of file

a1898f2

add newline to end of file

deed770

add newline to end of file

374f3e1

format

2519a7a

add eslint ignore

196d60e

sannidhishukla commented Jun 10, 2025

View reviewed changes

sannidhishukla added 11 commits June 10, 2025 17:43

updates

6e7010e

formatting

5e68deb

update imports

e72148b

update imports

cf36909

update key names

3661a54

update timeout

56794af

update imports

c286888

remove unneeded setup file

57b6dc2

fix test

feb7817

update tests

29184cf

update filenames

cd69daf

sannidhishukla changed the base branch from main to dev June 11, 2025 14:39

sannidhishukla added 3 commits June 11, 2025 08:41

update package lock json

dc0b574

node 18 -> 20

953b62b

some readme updates

7caee18

sannidhishukla requested review from AnJuHyppolite and williamavasquez June 11, 2025 15:03

casewalker reviewed Jun 12, 2025

View reviewed changes

package.json Show resolved Hide resolved

casewalker reviewed Jun 12, 2025

View reviewed changes

package.json Show resolved Hide resolved

casewalker reviewed Jun 12, 2025

View reviewed changes

AnJuHyppolite reviewed Jun 13, 2025

View reviewed changes

src/functions/email.ts Outdated Show resolved Hide resolved

AnJuHyppolite reviewed Jun 13, 2025

View reviewed changes

AnJuHyppolite self-requested a review June 13, 2025 15:38

AnJuHyppolite reviewed Jun 13, 2025

View reviewed changes

sannidhishukla added 6 commits June 13, 2025 12:13

review changes

c4349fb

node 20 -> node 22

6aa5411

add outputs

a42b17d

add error handling for missing fields

9ed8c7e

update param name!

0d92f0a

move error handling to try clause

d23fb48

sannidhishukla requested a review from AnJuHyppolite June 13, 2025 19:14

AnJuHyppolite reviewed Jun 13, 2025

View reviewed changes

README.md Outdated Show resolved Hide resolved

sannidhishukla added 2 commits June 13, 2025 14:32

add comma

dcce0e2

add new type

a771b54

ezhangy approved these changes Jun 16, 2025

View reviewed changes

ezhangy reviewed Jun 16, 2025

View reviewed changes

tsconfig.json Outdated Show resolved Hide resolved

sannidhishukla added 3 commits June 16, 2025 15:38

fix typo

d553463

update README

76154ab

add newline to end

a4482ac

sannidhishukla merged commit 3fe9fb7 into dev Jun 16, 2025

sannidhishukla deleted the migrate-to-aws-cdk branch June 16, 2025 21:47

ezhangy mentioned this pull request Aug 20, 2025

2025-08-20 Release #76

Merged

ezhangy pushed a commit that referenced this pull request Aug 26, 2025

Migrate AWS infrastructure from Serverless Framework to AWS CDK (#68)

8c240f8

ezhangy added a commit that referenced this pull request Aug 26, 2025

Revert "Migrate AWS infrastructure from Serverless Framework to AWS C…

e43b67e

…DK (#68)" This reverts commit 8c240f8.

ezhangy mentioned this pull request Aug 26, 2025

2025-08-25 Release #81

Merged

ezhangy added a commit that referenced this pull request Aug 26, 2025

2025-08-25 Release (#81)

58f7f27

Merges the following PRs to main: - #68 - #70 - #69 - #72 - #71 - #73 - #75 - #77

		// eslint-disable-next-line @typescript-eslint/no-var-requires
		const { getSsmParam } = require('./awsUtils');

Conversation

sannidhishukla commented Jun 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview

Next Steps

Uh oh!

sannidhishukla Jun 10, 2025

Choose a reason for hiding this comment

Uh oh!

sannidhishukla Jun 10, 2025

Choose a reason for hiding this comment

Uh oh!

sannidhishukla Jun 10, 2025

Choose a reason for hiding this comment

Uh oh!

sannidhishukla Jun 10, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

casewalker left a comment

Choose a reason for hiding this comment

Uh oh!

AnJuHyppolite Jun 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sannidhishukla Jun 13, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

AnJuHyppolite Jun 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AnJuHyppolite left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ezhangy left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ezhangy Jun 16, 2025

Choose a reason for hiding this comment

Uh oh!

sannidhishukla Jun 16, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

sannidhishukla commented Jun 10, 2025 •

edited

Loading

AnJuHyppolite Jun 13, 2025 •

edited

Loading

AnJuHyppolite Jun 13, 2025 •

edited

Loading