fix: complete label rename, export prompt bugs, docs + 0.1.5#57
Merged
fix: complete label rename, export prompt bugs, docs + 0.1.5#57
Conversation
Three export paths still said "Overall Score" or bare "Score" after the rename that was supposed to disambiguate KSM from the LLM strategy assessment. The terminal output, text report, and markdown report all had unrenamed labels — exactly the places where users see both numbers side-by-side and get confused. Also fixed the formula explainer that claimed KSM is a simple multiplication of three terms. It isn't — efficacy acts as a gate with a cap at 30 when zero and a sliding multiplier below 50. Saying "x × y × z" when the code does something materially different is worse than saying nothing. Closes #54
The post-run export (PR #51) shipped with writeFileSync unwrapped, a guard that made the no-analysis path completely unreachable, and Ctrl+C surfacing as a benchmark failure. Probably shouldn't ship interactive features that crash on predictable user behavior. Also updated the test that was still asserting "Overall Score" after we renamed it in #49. Tests only work if you keep them current. Closes #55
KSM now has three factors instead of two. The docs should probably reflect that before someone reads the spec and wonders why their score doesn't match the formula on the page. Updated KSM-SCORING.md with the full token efficiency section, realistic examples showing the cost difference between efficient and wasteful models, and bumped the spec version to 1.2. README scoring section now documents all three factors. CHANGELOG covers everything from #44 through #55. Version bump to 0.1.5.
The hand-calculated efficiency was 0.871 but the actual formula gives 0.867 for 2698 tokens/step. KSM rounds accordingly: 84.1 not 84.5. Spec documents should have correct math.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR #49 renamed "Overall Score" to "Strategy Score" in about 70% of the output paths. This finishes the job — terminal analysis, text reports, and markdown reports all say the right thing now. The formula explainer also stopped pretending KSM is a simple multiplication when it's actually a piecewise function with efficacy gating.
PR #51's export prompt had three bugs: writeFileSync with no error handling (permission denied = crash), a guard that made the no-analysis export path unreachable, and Ctrl+C during the prompt surfacing as "Benchmark failed." All fixed.
Docs updated to reflect token efficiency as the third KSM factor. CHANGELOG covers #44 through #55. Version bumped to 0.1.5.
363 tests passing, all output formats verified locally (terminal, markdown, HTML, share card, clipboard).
Closes #54, closes #55