Add splitting for big messy refactors

[eonist]

Proposed Solutions for Diff Grouping Feature.md
Can you look at this request [https___www.producth-3.md
Can you look at this request [https___www.producth-2.md
Can you look at this request [https___www.producth.md

[eonist]

How --split Works

Three-Phase Process

Phase 1: Intelligent Grouping

• Collects metadata for all staged files (path, status, lines changed, file type, directory)
• Sends compact file list to local LLM with grouping prompt: "Organize these into logical commit groups"
• LLM returns JSON with 3-5 semantic groups (e.g., "API changes", "Tests", "Documentation")
• Fits in 4k tokens since only metadata is sent, not full diffs

Phase 2: Per-Group Message Generation

• Loops through each group sequentially
• Generates actual diff for just that group's files
• Calls LLM again with group-specific diff to create tailored commit message
• Shows message to user (or auto-approves if --nopopup)

Phase 3: Sequential Commits

• Unstages all files temporarily
• Re-stages only current group's files
• Commits with generated message
• Repeats for next group

Example Flow

diffsense --split

1. Analyzes: 15 staged files → LLM suggests 3 groups
2. Group 1: "User authentication" (5 files) → generates message → commits
3. Group 2: "Test coverage" (8 files) → generates message → commits
4. Group 3: "Config updates" (2 files) → generates message → commits

Result: One messy 15-file PR becomes 3 clean, semantic commits.

With --nopopup

Same process, but zero interaction - fully automated for CI/CD and agents.

[dkargatzis]

@eonist the flow you outlined makes a lot of sense.

One thought that might fit well with this design, especially for big or messy refactors:
treat the staged diff as the single source of truth and replay it per group.

Concretely:

• Snapshot the staged state once (git diff --cached → temp patch), preserving exactly what the user staged (including partial hunks).
• Phase 1 stays the same: send only file metadata to the LLM and get back logical groups.
• For Phase 2 + 3, instead of unstaging/restaging:
  • reset to a clean baseline
  • apply only the relevant paths from the original patch per group
  • generate the commit message from the group-specific diff
  • commit sequentially

This keeps the LLM focused purely on grouping and messaging, while the actual code changes come from a deterministic patch. In practice that tends to be safer for large refactors and helps avoid edge cases or “creative” interpretations of what should go into each commit.

Just a thought - overall the proposal looks solid and this might be a low-risk way to implement the splitting mechanics.

[eonist]

@dkargatzis

can you read this comment? #49 (comment) does he mean this? staged files -> base of what will be split into multiple commits, with multiple files in each commit. or dos he mean something else?

Yes, your interpretation is correct. The commenter (dkargatzis) means exactly what you described: staged files are the base of what will be split into multiple commits, with multiple files in each commit.

What the Commenter Proposes

The commenter is suggesting an alternative implementation approach where:

• The staged diff (all files staged via git add) becomes the single source of truth
• This entire staged state is captured once as a patch file using git diff --cached
• The LLM groups these staged files into logical commit groups (e.g., "API changes", "Tests", "Documentation")
• Each group contains multiple files from the original staged set
• Sequential commits are created, with each commit containing multiple files from its assigned group

Key Difference from Original Proposal

The commenter's suggestion differs in the commit creation mechanism:

• Original approach: Unstage all files, then re-stage files per group
• Proposed approach: Capture the staged diff once as a patch, then apply only relevant paths from that patch per group

The commenter argues this is "safer for large refactors" because it ensures the exact changes the user staged (including partial hunks) are preserved deterministically, rather than relying on unstaging/restaging which could introduce edge cases.

Both approaches ultimately achieve the same goal: splitting a large set of staged changes into multiple logical commits, with each commit containing multiple related files.
¹

⁂

---

can you show me messy refactor of files in like example. that becomes multiple commits. simulate this in a simple markdown example please

I'll create a concrete example showing how a messy refactor with multiple staged files gets split into logical commits.

Messy Refactor Example

Initial Staged State (15 files)

All these files are staged together in one big mess:

git status
Changes to be committed:
  modified:   src/auth/login.js
  modified:   src/auth/logout.js
  modified:   src/auth/session.js
  modified:   src/auth/password-reset.js
  modified:   src/api/users-endpoint.js
  modified:   src/api/posts-endpoint.js
  modified:   src/api/comments-endpoint.js
  modified:   tests/auth/login.test.js
  modified:   tests/auth/session.test.js
  modified:   tests/api/users.test.js
  modified:   tests/api/posts.test.js
  modified:   docs/API.md
  docs/AUTH.md
  modified:   config/database.yml
  modified:   config/redis.yml

The Problem: These changes represent multiple unrelated features/fixes mixed together. Committing as one would create an unmaintainable history.

---

After diffsense --split

The LLM analyzes the files and creates 3 logical commit groups:

Commit 1: Authentication System Refactor

# Group contains 6 files
git commit -m "refactor: modernize authentication flow

- Migrate from sessions to JWT tokens
- Add password reset functionality  
- Update session management logic
- Improve logout cleanup process"

Files in this commit:

• src/auth/login.js
• src/auth/logout.js
• src/auth/session.js
• src/auth/password-reset.js
• tests/auth/login.test.js
• tests/auth/session.test.js

---

Commit 2: API Endpoints Update

# Group contains 5 files
git commit -m "feat: add pagination and filtering to API endpoints

- Implement cursor-based pagination for users/posts/comments
- Add query parameter validation
- Update test coverage for new parameters"

Files in this commit:

• src/api/users-endpoint.js
• src/api/posts-endpoint.js
• src/api/comments-endpoint.js
• tests/api/users.test.js
• tests/api/posts.test.js

---

Commit 3: Infrastructure Configuration

# Group contains 4 files
git commit -m "chore: update database and cache configuration

- Bump PostgreSQL connection pool size
- Configure Redis for session storage
- Update API and authentication documentation"

Files in this commit:

• config/database.yml
• config/redis.yml
• docs/API.md
• docs/AUTH.md

---

Result

Before: 1 messy commit with 15 unrelated files

After: 3 clean, semantic commits:

• Each commit has a clear purpose
• Related files are grouped together
• Easy to review, revert, or cherry-pick
• Git history tells a coherent story

This is exactly what the commenter means: the 15 staged files are the base that gets intelligently split into multiple commits, with each commit containing multiple related files.

Footnotes

1. https://github.com/edgeleap/diffsense/issues/49 ↩