feat: add searches and aggregation to express backend #28

jordan-smith721 · 2025-11-06T16:37:57Z

Adds MongoDB Search, Vector Search, and Aggregation endpoints to the express API. Also adds unit testing for these endpoints and cleans up the unit test files.
This PR also fixes up an issue with the vector search results pagination not working correctly

dacharyc

There are a couple of route issues we should fix to match the client, and a tsconfig Q related to a client file included here. Everything else is just a Q to consider.

dacharyc · 2025-11-06T19:46:05Z

client/tsconfig.json

    "resolveJsonModule": true,
    "isolatedModules": true,
-    "jsx": "react-jsx",
+    "jsx": "preserve",


When I run the client, it's automatically switching this value back to "react-jsx" with this message:

We detected TypeScript in your project and reconfigured your tsconfig.json file for you. The following mandatory changes were made to your tsconfig.json: - jsx was set to react-jsx (next.js uses the React automatic runtime)

Curious why you're changing it here, and if we need it to be "preserve", what we might need to change in the client to force it to respect this config field?

That's interesting, every time I run it it's been automatically changing it to preserve. I'm not sure what's causing the change but I'll look into it. For now I'll revert to react-jsx since that's what I originally had in there before it started changing it

Interestingly it's almost the same message too:

We detected TypeScript in your project and reconfigured your tsconfig.json file for you. The following mandatory changes were made to your tsconfig.json: - jsx was set to preserve (next.js implements its own optimized jsx transform)

Ahhh I see, it's a versioning issue. From Claude:

The Problem
The issue occurs because Next.js automatically modifies your tsconfig.json file when it starts up. Different versions of Next.js have different preferences for the jsx setting:

Your current setup: jsx: "preserve" (which Next.js 15+ prefers)
Others' setup: jsx: "react-jsx" (which older versions or different configurations prefer)
Why This Happens
Version Differences: Different Next.js versions have different default TSConfig preferences
Auto-Configuration: Next.js automatically "fixes" what it thinks are incorrect settings
Lock File Issues: Different package-lock.json or yarn.lock states can cause different dependency resolutions

Since this is using the latest version of Next, I think we should actually stick with preserve since later versions prefer it

That doesn't track with the usage, though. I ran npm install on the client in the last few days so it's pulling in the most recent version of Next.js. (Actually I checked and I've got 16.0.0 and it looks like Next released 16.0.1 recently.)

It may be a versioning difference issue, but that doesn't really hold up if we're both on the latest version of Next. So I think it's more likely it's some sort of configuration difference, or maybe a Node or other version issue, and we should probably try to get to the bottom of this as a fast follow.

Interesting. Okay yeah I'll add it to the list of fast follows and try and get to the bottom of what's going on here

dacharyc · 2025-11-06T20:05:21Z

server/express/src/controllers/movieController.ts

+  }
+
+  // Validate and set limit (default: 10, min: 1, max: 50)
+  const limitNum = Math.min(Math.max(parseInt(limit as string) || 10, 1), 50);


Curious about this. When performing vector search, the client provides a default limit of 20 - should the backend default match the client default, or are we intentionally defaulting to a different value of 10 here?

Yeah it should match, good catch!

dacharyc · 2025-11-06T20:07:43Z

server/express/src/controllers/movieController.ts

+          index: "vector_index",
+          path: "plot_embedding_voyage_3_large",
+          queryVector: queryVector,
+          numCandidates: limitNum * 15, // Search more candidates for better results


Is there something we should say about why we're multiplying the limit by 15 here? 15 feels like an arbitrary/magic number and I wonder if there's some way we arrived at this number or some guidance we should provide about balancing performance/quality here when setting this multiple?

Turns out the number is actually supposed to be 20! From the docs here: https://www.mongodb.com/docs/atlas/atlas-vector-search/vector-search-stage/#numcandidates-selection

I clarified the comment to explain better

Awesome - thanks for the doc link! Good to know what the guidance is. I haven't checked the other two backends but we should probably check what the others are using as part of the cleanup pass tomorrow, and update it if we're using different values there.

Definitely, I'll check!

dacharyc · 2025-11-06T20:16:40Z

server/express/src/controllers/movieController.ts

+ * Find similar movies using vector search based on a movie ID.
+ * Demonstrates finding semantically similar movies using plot embeddings.
+ */
+export async function findSimilarMovies(req: Request, res: Response): Promise<void> {


Maybe I missed it but I can't find anywhere in the frontend that we're using this API. Are we supporting future development here? I wonder if we can/should omit this from the v1 of the app if we haven't implemented the corresponding frontend feature yet.

Oh yep, this was a separate, earlier implementation of vector search that wasn't working out. I definitely thought I got rid of it but I guess not! Removed now.

dacharyc · 2025-11-06T20:35:28Z

server/express/src/routes/movies.ts

+ * Aggregate movies with their most recent comments.
+ * Demonstrates MongoDB $lookup aggregation to join collections.
+ */
+router.get("/aggregations/comments", asyncHandler(movieController.getMoviesWithMostRecentComments));


Unfortunately, these aggregation routes don't match the frontend. We need some adjustments to match the client:

Suggested change

router.get("/aggregations/comments", asyncHandler(movieController.getMoviesWithMostRecentComments));

router.get("/aggregations/reportingByComments", asyncHandler(movieController.getMoviesWithMostRecentComments));

dacharyc · 2025-11-06T20:36:49Z

server/express/src/routes/movies.ts

+ * Aggregate directors with the most movies.
+ * Demonstrates MongoDB $unwind and $group for array aggregation.
+ */
+router.get("/aggregations/directors", asyncHandler(movieController.getDirectorsWithMostMovies));


Suggested change

router.get("/aggregations/directors", asyncHandler(movieController.getDirectorsWithMostMovies));

router.get("/aggregations/reportingByDirectors", asyncHandler(movieController.getDirectorsWithMostMovies));

dacharyc · 2025-11-06T20:37:03Z

server/express/src/routes/movies.ts

+router.get("/aggregations/years", asyncHandler(movieController.getMoviesByYearWithStats));
+
+/**
+ * GET /api/movies/aggregations/directors


Suggested change

* GET /api/movies/aggregations/directors

* GET /api/movies/aggregations/reportingByDirectors

dacharyc · 2025-11-06T20:38:35Z

server/express/src/controllers/movieController.ts

+    title: result.title,
+    year: result.year,
+    genres: result.genres,
+    imdb: result.imdbRating ? { rating: result.imdbRating } : undefined,


This doesn't seem to match what the client is expecting:

Suggested change

imdb: result.imdbRating ? { rating: result.imdbRating } : undefined,

imdbRating: result.imdbRating,

dacharyc · 2025-11-06T20:39:47Z

server/express/src/types/index.ts

+  imdb?: {
+    rating?: number;
+    votes?: number;
+  };


Related to the mismatch with what the client is expecting:

Suggested change

imdb?: {

rating?: number;

votes?: number;

};

imdbRating?: number;

dacharyc · 2025-11-06T20:43:10Z

server/express/src/routes/movies.ts

+ * Find similar movies using vector search based on a movie ID.
+ * Demonstrates finding semantically similar movies using plot embeddings.
+ */
+router.get("/find-similar-movies", asyncHandler(movieController.findSimilarMovies));


Noted in the controller, but it looks like the frontend isn't using this route currently. Should we omit this until the frontend has implemented it?

dacharyc

Awesome - thanks for the fixups, everything seems to be working well! ✅

feat: add searches and aggregation to express backend

2755c63

jordan-smith721 temporarily deployed to testing November 6, 2025 16:38 — with GitHub Actions Inactive

jordan-smith721 added 2 commits November 6, 2025 11:49

Merge remote-tracking branch 'upstream/development' into express-updates

3ee207f

fix vector search implementation

dbb1dbf

jordan-smith721 temporarily deployed to testing November 6, 2025 17:42 — with GitHub Actions Inactive

jordan-smith721 had a problem deploying to testing November 6, 2025 17:42 — with GitHub Actions Failure

jordan-smith721 temporarily deployed to testing November 6, 2025 17:42 — with GitHub Actions Inactive

dacharyc requested changes Nov 6, 2025

View reviewed changes

Dachary feedback

24b407a

jordan-smith721 temporarily deployed to testing November 7, 2025 01:38 — with GitHub Actions Inactive

jordan-smith721 had a problem deploying to testing November 7, 2025 01:38 — with GitHub Actions Failure

jordan-smith721 temporarily deployed to testing November 7, 2025 01:38 — with GitHub Actions Inactive

jordan-smith721 requested a review from dacharyc November 7, 2025 01:38

dacharyc approved these changes Nov 7, 2025

View reviewed changes

dacharyc merged commit d7c83d3 into mongodb:development Nov 7, 2025
3 of 4 checks passed

	router.get("/aggregations/comments", asyncHandler(movieController.getMoviesWithMostRecentComments));
	router.get("/aggregations/reportingByComments", asyncHandler(movieController.getMoviesWithMostRecentComments));

	router.get("/aggregations/directors", asyncHandler(movieController.getDirectorsWithMostMovies));
	router.get("/aggregations/reportingByDirectors", asyncHandler(movieController.getDirectorsWithMostMovies));

	* GET /api/movies/aggregations/directors
	* GET /api/movies/aggregations/reportingByDirectors

	imdb: result.imdbRating ? { rating: result.imdbRating } : undefined,
	imdbRating: result.imdbRating,

feat: add searches and aggregation to express backend #28

feat: add searches and aggregation to express backend #28

Conversation

jordan-smith721 commented Nov 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dacharyc left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jordan-smith721 Nov 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jordan-smith721 Nov 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dacharyc left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jordan-smith721 commented Nov 6, 2025 •

edited

Loading

jordan-smith721 Nov 7, 2025 •

edited

Loading

jordan-smith721 Nov 7, 2025 •

edited

Loading