refactor: refactor ingestion output, remove unused exception and simplify cloning & FileSystem logic #223
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…emove unused exceptions, and fix partial clone subpath handling. - Consolidate `format_directory` and `format_single_file` into a single `format_node` function - Remove unused exceptions (`MaxFilesReachedError`, `MaxFileSizeReachedError`, `AlreadyVisitedError`) - Update partial clone logic to correctly handle single-file paths by stripping the filename from subpath when `blob` is True - Improve docstrings and clean up code for better readability
cyclotruc
approved these changes
Mar 7, 2025
Member
cyclotruc
left a comment
cyclotruc
left a comment
There was a problem hiding this comment.
The more i looked at the new sorting implementation, the more I liked it
impressive work thank you
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR unifies the ingestion output flow for both directories and single files into a single function,
format_node. It also removes several unused exceptions (MaxFilesReachedError,MaxFileSizeReachedError, andAlreadyVisitedError) and refines the partial clone subpath logic.Key Changes
Reduced Duplication in
output_formatters.pyConsolidated
format_directoryandformat_single_fileinto a singleformat_nodefunction, eliminating parallel code paths and clarifying how content is assembled and displayed.Removed an Unused Exception
Deleted
MaxFilesReachedError,MaxFileSizeReachedError, andAlreadyVisitedErrorfromexceptions.pysince they were no longer being used.Simplified Cloning Logic
Streamlined partial clone handling and removed extraneous steps in the
clone_repofunction.Refactored
FileSystemNodeSimplified the logic for the
sort_childrenmethod ofFileSystemNodeclass, itscontentandcontent_stringpropertiesAdditional Improvements