perf: Pagination memory optimization and code quality improvements#53
perf: Pagination memory optimization and code quality improvements#53devondragon merged 13 commits intomainfrom
Conversation
- Upgrade zod from ^3.25.76 to ^4.3.6 - Upgrade vitest from ^3.2.4 to ^4.0.18 - Reformat keywords array for better readability
Zod v4 requires explicit key type for z.record(). Update all z.record() calls to z.record(z.string(), ...) for customFieldValues and autoScheduled fields.
Add API_CONFIG with TIMEOUT_MS (30s) and CONNECT_TIMEOUT_MS (10s) for centralized request timeout management.
- Add isCodedError type guard for coded error detection - Add extractErrorInfo to uniformly extract message, code, context - Enhance formatMcpError to handle unknown error types - Include parameter info for validation errors in context
Add explicit debug log when workspace validation is skipped, making it clearer that the API will validate access when the ID is used.
Replace custom hasTextProperty helper with inline 'in' operator check, reducing code complexity while maintaining type safety.
- Add comprehensive JSDoc documentation to all handler methods - Add interface documentation for parameter types - Improve priority validation with explicit error message - Validates priority before applying to update payload
Add comprehensive JSDoc comments documenting the registry class, its methods, and usage examples for tool management.
Prevent excessive memory usage by stopping pagination early when limits are reached, rather than fetching all pages then slicing. Changes: - Add pre-fetch early termination in paginationNew.ts - Add limit parameter to getProjects() and getRecurringTasks() - Optimize searchTasks/searchProjects with overfetch multiplier (3x) - Optimize getAllUncompletedTasks with adaptive fetch limits - Remove redundant .slice() in getTasks() when maxItems enforced - Track remaining items needed and stop searching when satisfied This reduces memory usage for paginated operations and prevents fetching unnecessary data when user-specified limits are provided.
…hedules Add comprehensive unit tests for: - CustomFieldHandler: list, create, delete, add/remove from projects/tasks - RecurringTaskHandler: list, create, delete operations - ScheduleHandler: list operation with workspace resolution
Code Review: Pagination Optimization and Code Quality ImprovementsThank you for this comprehensive PR! I've reviewed the changes and here's my detailed feedback: ✅ Strengths1. Excellent Pagination Optimization (src/utils/paginationNew.ts:52-56)The pre-fetch early termination is a smart optimization that prevents memory exhaustion: if (maxItems && allItems.length >= maxItems) {
hasMore = false;
break;
}This will significantly reduce memory usage for large datasets. 2. Comprehensive Test CoverageGreat addition of unit tests for CustomFieldHandler, RecurringTaskHandler, and ScheduleHandler. The tests are well-structured with proper mocking and cover success and error cases. 3. Improved Error Handling (src/utils/errorHandling.ts:65-96)The new 4. Better DocumentationJSDoc additions to TaskHandler and ToolRegistry significantly improve code maintainability. 5. Proper Zod v4 MigrationThe schema updates to use 🔍 Issues & Concerns1. Critical: Type Check Failures ❗The PR currently has TypeScript compilation errors:
These should be resolved before merging. Run 2. Potential Bug: Overfetch Multiplier Logic 🐛In pagination optimizations, the overfetch multiplier (3x) could cause issues:
Recommendation: Add a cap like: const effectiveLimit = Math.min(limit * overfetchMultiplier, LIMITS.MAX_SEARCH_RESULTS);3. Memory Safety: MAX_PAGE_SIZE Truncation
|
There was a problem hiding this comment.
Pull request overview
This PR introduces significant performance optimizations for pagination handling, adds comprehensive unit test coverage for three handler modules, improves code documentation with JSDoc comments, and upgrades core dependencies (Zod v4, Vitest v4). The changes focus on preventing memory exhaustion through intelligent pagination limits and early termination.
Changes:
- Pagination memory optimization with pre-fetch early termination and configurable limits in API methods
- Comprehensive unit tests for CustomFieldHandler, RecurringTaskHandler, and ScheduleHandler
- JSDoc documentation for ToolRegistry and TaskHandler methods
- Zod v4 compatibility by adding explicit key types to z.record schemas
- Dependency updates (Zod v4.3.6, Vitest v4.0.18) with corresponding package-lock updates
Reviewed changes
Copilot reviewed 13 out of 14 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/handlers.schedule.spec.ts | New comprehensive unit tests for ScheduleHandler covering list operations, date validation, and error handling |
| tests/handlers.recurring.spec.ts | New comprehensive unit tests for RecurringTaskHandler covering CRUD operations and validation |
| tests/handlers.customfield.spec.ts | New comprehensive unit tests for CustomFieldHandler covering all operations including field attachment to tasks/projects |
| src/utils/paginationNew.ts | Added pre-fetch early termination check to prevent unnecessary API calls when maxItems limit is reached |
| src/utils/errorHandling.ts | Enhanced error extraction with new extractErrorInfo utility and improved formatMcpError with error codes |
| src/utils/constants.ts | Added API_CONFIG with timeout configuration constants |
| src/utils/workspaceResolver.ts | Added debug logging when skipping workspace validation |
| src/tools/ToolRegistry.ts | Added comprehensive JSDoc documentation for all methods and class |
| src/services/motionApi.ts | Updated getProjects and getRecurringTasks to accept options object with limit parameter; optimized search methods with overfetch multiplier; improved error handling; removed redundant slice calls |
| src/schemas/motion.ts | Added explicit string key type to all z.record() calls for Zod v4 compatibility |
| src/handlers/TaskHandler.ts | Added JSDoc documentation and priority validation to update operation |
| src/handlers/CommentHandler.ts | Simplified pagination cursor type check using inline type guard |
| package.json | Updated zod to v4.3.6 and vitest to v4.0.18, formatted keywords array |
| package-lock.json | Updated dependency tree for Zod v4 and Vitest v4 upgrades |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
|
||
| while (hasMore && pageCount < absoluteMaxPages) { | ||
| // Pre-fetch early termination: skip fetch if we've already reached maxItems | ||
| if (maxItems && allItems.length >= maxItems) { |
There was a problem hiding this comment.
The early termination check on line 53 uses if (maxItems && allItems.length >= maxItems), but maxItems has a default value of LIMITS.MAX_PAGE_SIZE * 10 on line 41, so it will never be undefined/falsy. The check if (maxItems && ...) is unnecessary since maxItems will always have a numeric value. Consider simplifying to if (allItems.length >= maxItems) for clarity.
| if (maxItems && allItems.length >= maxItems) { | |
| if (allItems.length >= maxItems) { |
|
|
||
| const res = await handler.handle(args); | ||
|
|
||
| expect(motionService.getRecurringTasks).toHaveBeenCalledWith('ws1'); |
There was a problem hiding this comment.
The API signature has changed from getRecurringTasks(workspaceId?: string, maxPages: number = 10) to getRecurringTasks(workspaceId?: string, options?: { maxPages?: number; limit?: number }). However, the test is mocking it with only the workspaceId parameter. While this works because the options parameter is optional, the test expectation on line 45 should verify the call is made with the correct parameter count to match the actual implementation more precisely. Consider updating to expect(motionService.getRecurringTasks).toHaveBeenCalledWith('ws1', undefined); to be explicit about the second parameter.
| expect(motionService.getRecurringTasks).toHaveBeenCalledWith('ws1'); | |
| expect(motionService.getRecurringTasks).toHaveBeenCalledWith('ws1', undefined); |
maxItems always has a value from the default parameter, so the truthy check was unnecessary. Simplifies the early termination condition.
Pull Request Review: Pagination Memory Optimization and Code Quality ImprovementsOverviewThis PR makes significant improvements to pagination handling, adds comprehensive test coverage for three handler classes, and enhances documentation. Overall, the changes demonstrate thoughtful engineering with a strong focus on memory safety and maintainability. ✅ Strengths1. Excellent Pagination Memory Safety (src/utils/paginationNew.ts:52-56)The pre-fetch early termination is a smart optimization that prevents unnecessary API calls and is more efficient than checking after fetch. The dual-layer protection (pre-fetch + post-fetch) provides robust memory safety. 2. Comprehensive Test CoverageThe three new test files (CustomFieldHandler, RecurringTaskHandler, ScheduleHandler) follow excellent patterns with consistent structure, good use of mocks, coverage of both happy paths and error cases, and clear test naming. 3. Improved Error Handling (src/utils/errorHandling.ts:76-96)The extractErrorInfo function now properly handles multiple error types with appropriate fallbacks, making error reporting more consistent across the codebase. 4. Excellent Documentation (src/handlers/TaskHandler.ts & src/tools/ToolRegistry.ts)The JSDoc additions are thorough, well-structured, and provide real value with parameter descriptions, return type documentation, usage examples, and clear explanations of complex logic. 5. API Optimization (src/services/motionApi.ts)Adding limit parameters to getProjects() and getRecurringTasks() prevents over-fetching and gives callers more control.
|
Fix calculateFetchLimit() in searchTasks, searchProjects, and getAllUncompletedTasks to handle edge cases where remaining items could be zero or negative. - Add early return (0) when remaining <= 0 to prevent negative values - Add defense-in-depth checks before API calls to skip when limit is 0 - Store fetchLimit in variable to avoid redundant function calls This prevents potential issues where: - Zero limit would fall through to default maxItems (2000) - Negative limit could cause unexpected pagination behavior
Pull Request Review: Pagination Memory Optimization and Code Quality ImprovementsThank you for this comprehensive PR! This is a well-structured set of improvements addressing performance, testing, and code quality. Below is my detailed feedback: ✅ Strengths1. Excellent Pagination Memory OptimizationThe core performance improvement is solid:
2. Strong Test Coverage
3. Documentation Improvements
4. Dependency Updates
|
…lidation Address PR review feedback: 1. DRY violation: Extract shared calculateAdaptiveFetchLimit() utility to paginationNew.ts, replacing 3 duplicate inline implementations 2. Edge case handling: Use Math.max(0, remaining) for cleaner guard against negative values (defense-in-depth) 3. Security: Add limit parameter validation in getTasks() and getProjects() to reject negative or non-integer values 4. Documentation: Add comprehensive JSDoc explaining the overfetch multiplier (3x) rationale - compensates for client-side filtering while balancing API calls vs memory usage
PR Review: Pagination Memory Optimization and Code Quality ImprovementsSummaryThis is a well-structured PR that addresses important performance and code quality concerns. The changes demonstrate solid software engineering practices with a focus on preventing resource exhaustion and improving maintainability. 🟢 Strengths1. Excellent Pagination OptimizationThe core improvement to prevent memory exhaustion is well-implemented:
2. Strong Test CoverageNew test files add significant value:
3. Documentation Improvements
4. Code Quality
5. Dependency Updates
🟡 Areas for Improvement1. TypeScript Compilation Issues
|
Summary
Changes
Performance
paginationNew.tslimitparameter togetProjects()andgetRecurringTasks()searchTasks(),searchProjects(),getAllUncompletedTasks()with overfetch multiplier.slice()callsTesting
Documentation
Refactoring
Dependencies
Test plan