Knowledge table optimization - #2202
Merged
mfortman11 merged 30 commits intoAug 4, 2026
Merged
Conversation
…pagination past 100 files
- file size sort, added check for file_size and chunk_count -10k limit -> 500 file limit -changed hardcoded page size to be controlled by React State; removed client side pagination from AG Grid and wired up custom UI mimicing AG Grid UI
- Add sortBy/sortOrder state wired to useListFiles (T1-4) - Map AG Grid colId to backend sort field names (size->file_size, etc.) - onSortChanged resets to page 1 on sort change via paginationGoToFirstPage() - Restore AG Grid native pagination UI with pageSize:10000 (T1-6) - T1-2: le=100 backend cap already removed from files.py
Switched to composite aggregation pagination under v2
…pagination past 100 files
- file size sort, added check for file_size and chunk_count -10k limit -> 500 file limit -changed hardcoded page size to be controlled by React State; removed client side pagination from AG Grid and wired up custom UI mimicing AG Grid UI
- Add sortBy/sortOrder state wired to useListFiles (T1-4) - Map AG Grid colId to backend sort field names (size->file_size, etc.) - onSortChanged resets to page 1 on sort change via paginationGoToFirstPage() - Restore AG Grid native pagination UI with pageSize:10000 (T1-6) - T1-2: le=100 backend cap already removed from files.py
Switched to composite aggregation pagination under v2
… pagination, mark total as approximate
…on-dong03/openrag into knowledge-table-optimization # Conflicts: # frontend/app/api/queries/useListFiles.ts # frontend/app/knowledge/page.tsx # src/services/file_service_v2.py
Frontend: switch to /api/v2/files, include is_approximate in ListFilesResponse, and disable pagination buttons; removed a debug console.log. Backend API: add _parse_after_key helper to validate JSON-encoded after_key and return 400 for invalid values; use FastAPI Query pattern parameter. FileServiceV2: add embedding_model to composite sort fields, make _get_file_count return (count, is_approximate) and log on failure, use raw bucket count to detect final page, and propagate is_approximate in responses so the UI can display approximate totals.
-moved custom footer into its own component
…on-dong03/openrag into knowledge-table-optimization
…3/openrag into pr/jason-dong03/2165
…earch_files page_size cap, reset page on search change
…on-dong03/openrag into knowledge-table-optimization
…ion exposure (git advanced security fix)
github-actions
Bot
added
the
backend
Contributor
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
mfortman11
merged commit Aug 4, 2026
b245948
into
langflow-ai:knowledge-table-pagination
15 of 29 checks passed
VisveshJ
pushed a commit
that referenced
this pull request
Aug 4, 2026
* Knowledge table optimization (#2202) * fix: remove page_size cap and fetch all files to fix Knowledge table pagination past 100 files * T1-1 to T1-3 deliverable - file size sort, added check for file_size and chunk_count -10k limit -> 500 file limit -changed hardcoded page size to be controlled by React State; removed client side pagination from AG Grid and wired up custom UI mimicing AG Grid UI * feat: T1-4 server-side sort + T1-6 native AG Grid pagination - Add sortBy/sortOrder state wired to useListFiles (T1-4) - Map AG Grid colId to backend sort field names (size->file_size, etc.) - onSortChanged resets to page 1 on sort change via paginationGoToFirstPage() - Restore AG Grid native pagination UI with pageSize:10000 (T1-6) - T1-2: le=100 backend cap already removed from files.py * v2 API Migration Switched to composite aggregation pagination under v2 * Update page.tsx * fix: remove page_size cap and fetch all files to fix Knowledge table pagination past 100 files * T1-1 to T1-3 deliverable - file size sort, added check for file_size and chunk_count -10k limit -> 500 file limit -changed hardcoded page size to be controlled by React State; removed client side pagination from AG Grid and wired up custom UI mimicing AG Grid UI * feat: T1-4 server-side sort + T1-6 native AG Grid pagination - Add sortBy/sortOrder state wired to useListFiles (T1-4) - Map AG Grid colId to backend sort field names (size->file_size, etc.) - onSortChanged resets to page 1 on sort change via paginationGoToFirstPage() - Restore AG Grid native pagination UI with pageSize:10000 (T1-6) - T1-2: le=100 backend cap already removed from files.py * v2 API Migration Switched to composite aggregation pagination under v2 * Address review feedback: log _get_file_count failures, restore search pagination, mark total as approximate * Git merge fix issues * fix git HEAD * v2 files: add approximate counts & cursor parse Frontend: switch to /api/v2/files, include is_approximate in ListFilesResponse, and disable pagination buttons; removed a debug console.log. Backend API: add _parse_after_key helper to validate JSON-encoded after_key and return 400 for invalid values; use FastAPI Query pattern parameter. FileServiceV2: add embedding_model to composite sort fields, make _get_file_count return (count, is_approximate) and log on failure, use raw bucket count to detect final page, and propagate is_approximate in responses so the UI can display approximate totals. * Refactored component -moved custom footer into its own component * fix merge issues * PR Fixes -changed styling to match Tailwind - footer is now always active, showing regardless of servertotal or wildcardquery -global sort fix * Fix search pagination: route all data through useListFiles, restore search_files page_size cap, reset page on search change * Update page.tsx * update page.tsx * style: ruff autofix (auto) * Lint backend fix * Fix: remove exception detail from error responses to prevent information exposure (git advanced security fix) --------- Co-authored-by: Visvesh Jegadheesh <visveshjega@ibm.com> Co-authored-by: Mike Fortman <mfortman11@gmail.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * Fix: missing_bucket for composite agg, is_approximate consistency, page guard, search_files page_size cap * style: ruff autofix (auto) * fixes: PR fixes -Improved exception handling by chaining the original error. -Refactored FileServiceV2 to use dependency injection instead of creating it in routes -Optimized cursor cache initialization with lazy Map creation -Fixed infinite re-render caused by search reset effect * Update page.tsx --------- Co-authored-by: Visvesh Jegadheesh <visveshjega@ibm.com> Co-authored-by: Mike Fortman <mfortman11@gmail.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.