Problem
The current history digest process can consume a large amount of tokens even for a relatively small repository.
In an observed repository with approximately 80 commits, digesting the Git history quickly used a substantial amount of the available token budget. Development repositories commonly contain hundreds or thousands of commits, so the current consumption pattern makes digest difficult to run at larger scales.
Observed behavior
The digest process appears to inspect commits individually and at a similar level of depth, even though not every commit contains historical work that is equally valuable to preserve.
Token usage increases quickly as the history is processed. The cost is amplified when sub-agents are used to analyze and segment the history. While sub-agent analysis may produce better semantic results, a single digest session using gpt-5.6-sol medium can consume a noticeable portion of the weekly allowance.
The process does not currently expose a clear distinction between commits that receive lightweight inspection and commits that receive deeper semantic analysis. As a result, routine, low-value, or otherwise less relevant commits may contribute to token consumption at a level similar to more meaningful feature or experiment commits.
Environment
- research-git version: 0.0.7
- Repository history: approximately 80 commits
- Agent/model context: sub-agent digest analysis using
gpt-5.6-sol medium
Impact
Digest can exhaust practical token budgets before finishing even modest repository histories. On repositories with hundreds or thousands of commits, the cost can make full history digestion impractical and limits the feature's scalability for established development projects.
Problem
The current history digest process can consume a large amount of tokens even for a relatively small repository.
In an observed repository with approximately 80 commits, digesting the Git history quickly used a substantial amount of the available token budget. Development repositories commonly contain hundreds or thousands of commits, so the current consumption pattern makes digest difficult to run at larger scales.
Observed behavior
The digest process appears to inspect commits individually and at a similar level of depth, even though not every commit contains historical work that is equally valuable to preserve.
Token usage increases quickly as the history is processed. The cost is amplified when sub-agents are used to analyze and segment the history. While sub-agent analysis may produce better semantic results, a single digest session using
gpt-5.6-sol mediumcan consume a noticeable portion of the weekly allowance.The process does not currently expose a clear distinction between commits that receive lightweight inspection and commits that receive deeper semantic analysis. As a result, routine, low-value, or otherwise less relevant commits may contribute to token consumption at a level similar to more meaningful feature or experiment commits.
Environment
gpt-5.6-sol mediumImpact
Digest can exhaust practical token budgets before finishing even modest repository histories. On repositories with hundreds or thousands of commits, the cost can make full history digestion impractical and limits the feature's scalability for established development projects.