You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I ran a no-write Sverklo proof pass against OpenMontage because it is trending today and it is an agent-heavy repo, so it is a better sanity check than a toy project.
Indexed 1,284 files, 20,750 chunks, and 150,672 symbol references
Selected ToolResult at tools/base_tool.py:120
Found 475 references across 81 files
Generated next agent prompt: Use sverklo impact on ToolResult and tell me what would break if I changed its signature.
Legacy token-efficiency demo:
grep + reading the top 10 matched files over 5 exploratory questions: 219,549 estimated input tokens
sverklo_search: 10,026 estimated input tokens
Aggregate: 21.9x fewer estimated tokens
Important caveat: this is the legacy token-efficiency benchmark, not a correctness-gated benchmark. One query, auth_middleware, returned no Sverklo result, so I would not treat this as a quality win. The honest claim is only that repo-memory search can cut token input sharply on exploratory codebase lookup, while still needing human/project feedback on whether the selected symbol is meaningful.
What I would like from OpenMontage maintainers/users:
Is ToolResult actually a useful high-blast-radius symbol to inspect?
Did this miss a more important abstraction in the repo?
Is this kind of no-write proof useful enough to run on agent-heavy repos, or is it mostly noise?
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
-
I ran a no-write Sverklo proof pass against OpenMontage because it is trending today and it is an agent-heavy repo, so it is a better sanity check than a toy project.
Pinned repo:
calesthio/OpenMontage@9066dcb2e319727789820c5bcd28274695f2a18asverklo@0.29.2Proof run:
sverklo prove --no-write --guided --markdown <repo>ToolResultattools/base_tool.py:120Use sverklo impact on ToolResult and tell me what would break if I changed its signature.Legacy token-efficiency demo:
grep+ reading the top 10 matched files over 5 exploratory questions: 219,549 estimated input tokenssverklo_search: 10,026 estimated input tokensImportant caveat: this is the legacy token-efficiency benchmark, not a correctness-gated benchmark. One query,
auth_middleware, returned no Sverklo result, so I would not treat this as a quality win. The honest claim is only that repo-memory search can cut token input sharply on exploratory codebase lookup, while still needing human/project feedback on whether the selected symbol is meaningful.What I would like from OpenMontage maintainers/users:
ToolResultactually a useful high-blast-radius symbol to inspect?Sverklo repo: https://github.com/sverklo/sverklo
Beta Was this translation helpful? Give feedback.
All reactions