This repository was archived by the owner on May 13, 2026. It is now read-only.
docs(sample-data): mark coding-projects auto-skip + settings.json skip as aspirational - #875
Merged
Merged
Conversation
…p as aspirational Reconcile sample_data/README.md with the actual smart-folder behavior. The README claimed: - "Auto-skip coding projects (`coding_projects/`) — directories containing manifest files like `package.json`, `Cargo.toml`, or `pyproject.toml` are skipped entirely before LLM classification" - "Skip config files (.bashrc, settings.json)" Code trace shows neither is implemented: - `src/ingestion/smart_folder/scanner.rs::scan_directory_recursive` only filters by extension and skips dotfiles. There's no manifest-aware logic anywhere in the smart-folder pipeline. An explicit test (`test_scan_includes_coding_project_files`, mod.rs:489) asserts the opposite — coding_projects/* IS scanned. - `apply_heuristic_filtering` in classify.rs marks `.json`/`.toml`/`.py`/ `.rs`/`.js` as personal_data/code with should_ingest=true, so the LLM is never even asked about coding-project files. - `.bashrc` is skipped only incidentally (scanner drops dotfiles), and `settings.json` is not skipped at all — its `.json` extension matches the heuristic's text/data class. Per task instructions: code-vs-spec drift defaults to Case B. README is the spec, code is missing the feature. This PR updates the README to describe the *current* behavior and marks the unimplemented bits as aspirational with a TODO pointing to follow-up kanban task `1e2c7`. Implementation of the auto-skip is intentionally out of scope here.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
sample_data/README.mdwith actual smart-folder behavior. The README claimed manifest-based auto-skip (package.json/Cargo.toml/pyproject.toml) and config-file skipping (settings.json); the code does neither.1e2c7.Trace evidence
Discovery during 2026-05-05 dogfood scan over
sample_data/: 79 files scanned, 59 ingest / 20 skipped.coding_projects/my_website/(3 files),coding_projects/rust_cli/(2),coding_projects/data_analysis/(3) all queued aspersonal_data.config/settings.jsonqueued aspersonal_datatoo.Code trace (read-only):
src/ingestion/smart_folder/scanner.rs::scan_directory_recursive— only filters by extension (is_ingestible_file) and skips dotfiles (file_name.starts_with('.')). No manifest awareness anywhere in this file.src/ingestion/smart_folder/scanner.rs:489—test_scan_includes_coding_project_filesis an explicit test asserting the opposite of the README: coding-project files MUST be infile_paths. That's the contract the code defends today.src/ingestion/smart_folder/mod.rs::perform_smart_folder_scan_with_progress— orchestrates scan → image-dir classify → dedup → heuristic → LLM. No directory-skip step.src/ingestion/smart_folder/classify.rs::apply_heuristic_filtering—.json/.toml/.py/.rs/.js→should_ingest=true, categorypersonal_dataorcode. Heuristic resolves coding-project files instantly, so they never reach the LLM where contextual judgment might have skipped them..bashrcIS skipped — incidentally, by the scanner's dotfile filter, not by any config-aware rule.settings.jsonis NOT skipped —.jsonis heuristic-classified as personal data.Why Case B (not Case A)
The discovery prompt's tiebreaker: "Default to Case B unless the spec is clearly broken — manifest-based auto-skip is a useful UX primitive for new users." It is — coding-project files dwarf personal data on most laptops, and the README's behavior is the right product. So spec stays, code gets the follow-up.
What this PR does NOT do
src/ingestion/smart_folder/*edits — the scanner's contract stays as-is until the follow-up task lands.Follow-up
Filed as kanban task
1e2c7—feat(smart-folder): auto-skip directories with package.json / Cargo.toml / pyproject.toml manifests before LLM classification. The follow-up prompt includes the trace above plus an explicit plan to:constnext tois_ingestible_file.scan_directory_recursivepeek for the manifests before recursing; skip the entire subtree.test_scan_includes_coding_project_fileswithtest_scan_skips_coding_project_directories.The
settings.jsondrift is not part of that follow-up — it's an LLM-judgment problem rather than a manifest one. Flagged in the README; will need its own task if anyone wants it.Test plan
scanner.rs,mod.rs,classify.rsconfirming no auto-skip logic exists (covered above).sample_data/README.mdchanges; no Rust code touched.🤖 Generated with Claude Code